Download to read offline
Uploaded byCeph Community
Ceph Day San Jose - Red Hat Storage Acceleration Utlizing Flash Technology
The document discusses three ways to accelerate application performance with flash storage using Ceph software defined storage: 1) utilizing all flash storage to maximize performance, 2) using a hybrid configuration with flash and HDDs to balance performance and capacity, and 3) using all HDD storage for maximum capacity but lowest performance. It also examines using NVMe SSDs versus SATA SSDs, and how to optimize Linux settings and Ceph configuration to improve flash performance for applications.
More Related Content
Similar to Ceph Day San Jose - Red Hat Storage Acceleration Utlizing Flash Technology
![Vibe Coding vs. Spec-Driven Development [Free Meetup]](https://cdn.slidesharecdn.com/ss_thumbnails/vibecodingvsspecdrivendevelopment-251209105622-43f455e7-thumbnail.jpg?width=640&height=640&fit=bounds)
Ceph Day San Jose - Red Hat Storage Acceleration Utlizing Flash Technology
- 1. RED HAT CEPHSTORAGE ACCELERATION UTILIZING FLASH TECHNOLOGY Applications and Ecosystem Solutions Development Rick Stehno Ceph Day SJC 2017 1
- 2. Seagate Confidential 2 •Utilize flash caching features to accelerate critical data. Caching methods can be write-back for writes, write-thru for disk/cache transparency, read cache, etc.. • Utilize storage tiering capabilities. Performance critical data resides on flash storage, colder data resides on HDD • Utilize all flash storage to accelerate performance when all application data is performance critical or when the application does not provide the features or capabilities to cache or to migrate the data Three ways to accelerate application performance with flash Flash Acceleration for Applications
- 3. Seagate Confidential 3 Configurations: •All flash storage - Performance • Highest performance per node • Less maximum capacity per node • Hybrid HDD and flash storage - Balanced • Balances performance, capacity and cost • Application and workload suitable for • Performance critical data on flash • Utilize host software caching or tiering on flash • All HDD storage - Capacity • Maximum capacity per node, lowest cost • Lower performance per node Ceph Software Defined Storage (SDS) Acceleration
- 4. Seagate Confidential 4 –Higherperformance in half the rack space –28% less power and cooling –Higher MTBF inherent with reduced component count –Reduced OSD recovery time per Ceph node –Lower TCO Why 1U server with 10 NVMe SSDs may be better choice vs. 2U Server with 24 SATA SSDs Storage - NVMe vs SATA SSD
- 5. Seagate Confidential 5 •4.5x increase for 128k sequential reads • 3.5x increase for 128k sequential writes • 3.7x increase for 4k random reads • 1.4x increase for 4k random 70/30 RR/RW • Equal performance for 4k random writes Why 1U server with 10 NVMe SSDs may be better choice vs. 2U Server with 24 SATA SSDs All Flash Storage - NVMe vs SATA SSD cont’d FIO Benchmarks (1x represents 24 SATA SSD baseline)
- 6. Seagate Confidential 6 Why1U server with 10 NVMe SSDs may be better choice vs. 2U Server with 24 SATA SSDs All Flash Storage - NVMe vs SATA SSD cont’d Increasing the load to extend NVMe advantage over and above the 128 thread SATA SSD Test: • 5.8x increase for Random Writes at 512 threads • 3.1x increase for 70/30 RR/RW at 512 threads • 4.2x increase for Random Reads at 790 threads • 8.2x increase for Sequential Reads at 1264 threads 10 NVMe SSDs support higher workloads and more users 3x 5.8x 1.4x 3.1x 1.0x 4.2x 1.3x 8.2x 128 Theads 512 Theads 128 Threads 512 Threads 128 threads 790 threads 128 threads 1264 threads Gains Random Write 70/30 RR/RW Random Reads Sequential Reads Ceph RBD NVMe Performance Gains over SATA SSD Random Writes 70/30 RR/RW Random Reads Sequential Reads 128k FIO RBD IOEngine Benchmark
- 7. Seagate Confidential 7 Priceper MB/s: Cost of ((Retail Cost of SSD) / MB/s for each test) SSD Total SSD Price Price MB/s 128k Random Writes 128 threads Price MB/s 128k Random Writes 512 threads 24 - SATA SSD 960G $7,896 24 - SATA SSD 960G $15.00 10 - NVMe 2TB $10,990 10 - NVMe 2TB $7.00 10 – NVMe 2TB $3.00 These prices do not include savings from electrical/cooling costs, reducing datacenter floor space, from the reduction of SATA SSD Note: 128k random write FIO RBD benchmark: SATA SSD averaged 85% busy, NVMe averaged 80% busy with 512 threads FIO RBD Maximum Threads Random Write Performance for NVMe Ceph Storage Costs Seagate SATA SSD vs. Seagate NVMe SSD
- 8. Seagate Confidential 8 MySQL •MySQL is the most popular and the most widely used open-source database in the world • MySQL is both feature rich in the areas of performance, scalability and reliability • Database users demand high OLTP performance - Small random reads/writes Ceph • Most popular Software Defined Storage system • Scalable • Reliable Does it make sense implementing Ceph into a MySQL Database environment? Ceph was not designed to provide high performance for OLTP environments OLTP entails small random reads/writes
- 9. Seagate Confidential 9 MySQLSetup: Release 5.7 45,000,000 rows 6GB Buffer 4G logfiles RAID 0 over 18 HDD Ceph Setup: 3 Nodes each containing: Jewel Using Filestore 4 NVMe SSDs 1 Pool over 12 NVMe SSDs Replica 2 40G private and public network For all tests, all MySQL files were local on local server except the database file, this file was moved to the Ceph cluster. MySQL - Comparing Local HDD to Ceph Cluster Threads
- 10. Seagate Confidential 10 MySQL- Comparing Local NVMe SSD to Ceph Cluster MySQL Setup: Release 5.7 45,000,000 rows 6GB Buffer 4G logfiles RAID 0 over 4 NVMe SSDs Ceph Setup: 3 Nodes each containing: Jewel Using Filestore 4 NVMe SSDs 1 Pool over 12 NVMe SSDs Replica 1 40G private and public network For all tests, all MySQL files were local on local server except the database file, this file was moved to the Ceph cluster.
- 11. Seagate Confidential 11 AllSSD Case-1: Case-2: Case-3: 2 SSDs 2 SSDs 1 PCIe flash 1 OSD/SSD 4 OSDs/SSD 4 OSDs/SSD 8 OSD journals on PCIe flash 0 100000 200000 300000 400000 500000 600000 700000 800000 0 200000 400000 600000 800000 1000000 1200000 2 ssd, 2 osd 2 ssd, 8 osd 2 ssd, 8 osd, +journal IOPS KB/s FIO Random Write - 200 Threads - 128k Data Seagate SSD and Seagate PCIe Storage Ceph All Flash Storage Acceleration
- 12. Seagate Confidential 12 CephAll Flash Storage Acceleration 4K FIO RBD Benchmarks 1M FIO RBD Benchmarks 3 node Ceph cluster 100G Public and Private Networks 4 - Seagate NVMe SSD per node 12 Seagate NVMe SSD per cluster Benchmark 1: 1 - OSD per NVMe SSD Benchmark 2: 4 - OSD per NVMe 4k Random Write 12 NVMe 12 OSD MB/s 12 NVMe 48 OSD MB/s 12 NVMe 12 OSD IO/s 12 NVMe 48 OSD IO/s Total: 174 291 43885 72634 4k Random Read 12 NVMe 12 OSD MB/s 12 NVMe 48 OSD MB/s 12 NVMe 12 OSD IO/s 12 NVMe 48 OSD IO/s Total: 580 2537 188801 634739 1M Random Write 12 NVMe 12 OSD MB/s 12 NVMe 48 OSD MB/s 12 NVMe 12 OSD IO/s 12 NVMe 48 OSD IO/s Total: 1335 2706 1300 2636 1M Random Read 12 NVMe 12 OSD MB/s 12 NVMe 48 OSD MB/s 12 NVMe 12 OSD IO/s 12 NVMe 48 OSD IO/s Total: 19434 39743 19609 39734
- 13. Seagate Confidential 13 •Use RAW device or create 1st partition on 1M boundary (sector 2048 for 512B sectors, sector 256 for 4k sectors) • Ceph-deploy uses the optimal alignment when creating an OSD • Use blk-mq/scsi-mq if kernel supports it • rq_affinity = 1 for NVMe, rq_affinity = 2 for non-NVMe • rotational = 0 • blockdev --setra 256 (for 4k sectors, 4096 for 512B sectors) Linux tuning is still a requirement to get optimum performance out of a SSD Linux Flash Storage Tuning
- 14. Seagate Confidential 14 •If using an older kernel that doesn’t support BLK-MQ, use: • “deadline” IO-Scheduler with supporting variables: • fifo-batch • front-merges • writes-starved • XFS Mount options: • nobarrier,discard,noatime,attr2,inode64,noquota • MySQL – when using flash, configure both innodb_io_capacity and innodb_lru_scan_depth • Modify Linux read ahead on mapped RBD image on client • echo 1024 > /sys/class/block/rbd0/queue/read_ahead_kb Linux tuning is still a requirement to get optimum performance out of a SSD Linux Flash Storage Tuning cont’d
- 15. Seagate Confidential 15 FlashStorage Device Configuration Ceph tuning options can make a difference: • RBD Cache • If using a smaller number of SSD/NVMe SSD, try creating multiple OSD’s per SSD/NVMe SSD. Have seen good performance increases using 4 OSD per SSD/NVMe SSD 128k Random Writes 12 NVMe MB/s 30 NVMe MB/s disabled Total: 365 1107 enabled Total: 432 1067 Gain/Loss 18% 0% 128k Sequential Reads 12 NVMe MB/s disabled Total: 9691 enabled Total: 6359 Gain/Loss -34% 1M Random Reads 12 NVMe MB/s disabled Total: 38899 enabled Total: 42677 Gain/Loss 10% 128k Random Reads 12 NVMe MB/s disabled Total: 8915 enabled Total: 5669 Gain/Loss -37%
- 16. Seagate Confidential 16 FlashStorage Device Configuration If the NVMe SSD or SAS/SATA SSD device can be configured to use a 4k sector size, this could increase performance for certain applications like databases. For my FIO tests with the RBD engine and for all of my MySQL tests, I saw up to a 3x improvement (depending on the test) when using 4k sector sizes compared to using 512 byte sectors. Precondition all SSD before running benchmarks. Have seen over a 3x gain in performance after preconditioning Storage devices used for all of the above benchmarks/tests: • Seagate Nytro XF1440 NVMe SSD • Seagate Nytro XF1230 SATA SSD • Seagate 1200.2 SAS SSD • Seagate XP6500 PCIe Flash Accelerator Card
- 17. Seagate Confidential 17 SeagateBroadest PCIe, SAS and SATA Portfolio
- 18. Seagate Confidential 18SeagateConfidential Thank You! Questions? Learn how Seagate accelerates storage with one of the broadest SSD and Flash portfolios in the market