The document discusses optimizing Ceph performance by leveraging Intel Optane and 3D NAND TLC SSDs. It provides an overview of a Ceph cluster configuration with Intel Optane SSDs for journal/metadata and Intel 3D NAND SSDs for data storage. The document analyzes the performance of this all-flash Ceph cluster, finding over 2.8 million IOPS with low latency. It also examines various optimizations that could further improve Ceph performance on Intel Optane-based storage.
Intel - optimizing ceph performance by leveraging intel® optane™ and 3 d nand...inwin stack
Kenny Chang (張任伯) (Storage Solution Architect, Intel)
With the trend that Solid State Drive (SSD) becomes more affordable, more and more cloud providers are trying to provide high performance, highly reliable storage for their customers with SSDs. Ceph is becoming one of most open source scale-out storage solutions in worldwide market. More and more customers have strong demands that using SSD in Ceph to build high performance storage solutions for their Openstack clouds.
The disrupted Intel® Optane SSDs based on 3D Xpoint technology fills the performance gap between DRAM and NAND based SSD while the Intel® 3D NAND TLC is reducing cost gap between SSD and traditional spindle hard drive and makes it possible for all flash storage. In this session, we will
1) Discuss OpenStack storage Ceph reference design on the first Intel Optane (3D Xpoint) and P4500 TLC NAND based all-flash Ceph cluster, it delivers multi-million IOPS with extremely low latency as well as increase storage density with competitive dollar-per-gigabyte costs
2) Share Ceph bluestore tunings and optimizations, latency analysis, TCO model, IOPS/TB, IOPS/$ based on the reference architecture to demonstrate this high performance, cost effective solution.
Intel - optimizing ceph performance by leveraging intel® optane™ and 3 d nand...inwin stack
Kenny Chang (張任伯) (Storage Solution Architect, Intel)
With the trend that Solid State Drive (SSD) becomes more affordable, more and more cloud providers are trying to provide high performance, highly reliable storage for their customers with SSDs. Ceph is becoming one of most open source scale-out storage solutions in worldwide market. More and more customers have strong demands that using SSD in Ceph to build high performance storage solutions for their Openstack clouds.
The disrupted Intel® Optane SSDs based on 3D Xpoint technology fills the performance gap between DRAM and NAND based SSD while the Intel® 3D NAND TLC is reducing cost gap between SSD and traditional spindle hard drive and makes it possible for all flash storage. In this session, we will
1) Discuss OpenStack storage Ceph reference design on the first Intel Optane (3D Xpoint) and P4500 TLC NAND based all-flash Ceph cluster, it delivers multi-million IOPS with extremely low latency as well as increase storage density with competitive dollar-per-gigabyte costs
2) Share Ceph bluestore tunings and optimizations, latency analysis, TCO model, IOPS/TB, IOPS/$ based on the reference architecture to demonstrate this high performance, cost effective solution.
Yesterday's thinking may still believe NVMe (NVM Express) is in transition to a production ready solution. In this session, we will discuss how the evolution of NVMe is ready for production, the history and evolution of NVMe and the Linux stack to address where NVMe has progressed today to become the low latency, highly reliable database key value store mechanism that will drive the future of cloud expansion. Examples of protocol efficiencies and types of storage engines that are optimizing for NVMe will be discussed. Please join us for an exciting session where in-memory computing and persistence have evolved.
Ceph: Open Source Storage Software Optimizations on Intel® Architecture for C...Odinot Stanislas
Après la petite intro sur le stockage distribué et la description de Ceph, Jian Zhang réalise dans cette présentation quelques benchmarks intéressants : tests séquentiels, tests random et surtout comparaison des résultats avant et après optimisations. Les paramètres de configuration touchés et optimisations (Large page numbers, Omap data sur un disque séparé, ...) apportent au minimum 2x de perf en plus.
How does Ceph perform when used in high-performance computing? This talk will cover a year of running Ceph on a (small) Cray supercomputer. I will describe how Ceph was configured to perform in an all-NVME configuration, and the process of analysis and optimisation of the configuration. I'll also give details on the efforts underway to adapt Ceph's messaging to run over high performance network fabrics, and how this work could become the next frontier in the battle against storage performance bottlenecks.
IBM POWER8 processor is the fastest available on the market, redefining Open Source performance. With this amazing processor, IBM and members of the OpenPower Foundation design innovative and cost-effective systems, delivering the infrastructure of choice for the most demanding workloads, in terms of throughput, scalability and reliability.
In this talk in english, Thibaud Besson will browse the key characteristics of Power Systems, why they are the most relevant for today's challenges, both from a technical and economical standpoint. Finally, we will review the possibilities you have to get your hands on one of these outstanding plateforms for your Open Source applications.
Yesterday's thinking may still believe NVMe (NVM Express) is in transition to a production ready solution. In this session, we will discuss how the evolution of NVMe is ready for production, the history and evolution of NVMe and the Linux stack to address where NVMe has progressed today to become the low latency, highly reliable database key value store mechanism that will drive the future of cloud expansion. Examples of protocol efficiencies and types of storage engines that are optimizing for NVMe will be discussed. Please join us for an exciting session where in-memory computing and persistence have evolved.
Ceph: Open Source Storage Software Optimizations on Intel® Architecture for C...Odinot Stanislas
Après la petite intro sur le stockage distribué et la description de Ceph, Jian Zhang réalise dans cette présentation quelques benchmarks intéressants : tests séquentiels, tests random et surtout comparaison des résultats avant et après optimisations. Les paramètres de configuration touchés et optimisations (Large page numbers, Omap data sur un disque séparé, ...) apportent au minimum 2x de perf en plus.
How does Ceph perform when used in high-performance computing? This talk will cover a year of running Ceph on a (small) Cray supercomputer. I will describe how Ceph was configured to perform in an all-NVME configuration, and the process of analysis and optimisation of the configuration. I'll also give details on the efforts underway to adapt Ceph's messaging to run over high performance network fabrics, and how this work could become the next frontier in the battle against storage performance bottlenecks.
IBM POWER8 processor is the fastest available on the market, redefining Open Source performance. With this amazing processor, IBM and members of the OpenPower Foundation design innovative and cost-effective systems, delivering the infrastructure of choice for the most demanding workloads, in terms of throughput, scalability and reliability.
In this talk in english, Thibaud Besson will browse the key characteristics of Power Systems, why they are the most relevant for today's challenges, both from a technical and economical standpoint. Finally, we will review the possibilities you have to get your hands on one of these outstanding plateforms for your Open Source applications.
COLLEGE BUS MANAGEMENT SYSTEM PROJECT REPORT.pdfKamal Acharya
The College Bus Management system is completely developed by Visual Basic .NET Version. The application is connect with most secured database language MS SQL Server. The application is develop by using best combination of front-end and back-end languages. The application is totally design like flat user interface. This flat user interface is more attractive user interface in 2017. The application is gives more important to the system functionality. The application is to manage the student’s details, driver’s details, bus details, bus route details, bus fees details and more. The application has only one unit for admin. The admin can manage the entire application. The admin can login into the application by using username and password of the admin. The application is develop for big and small colleges. It is more user friendly for non-computer person. Even they can easily learn how to manage the application within hours. The application is more secure by the admin. The system will give an effective output for the VB.Net and SQL Server given as input to the system. The compiled java program given as input to the system, after scanning the program will generate different reports. The application generates the report for users. The admin can view and download the report of the data. The application deliver the excel format reports. Because, excel formatted reports is very easy to understand the income and expense of the college bus. This application is mainly develop for windows operating system users. In 2017, 73% of people enterprises are using windows operating system. So the application will easily install for all the windows operating system users. The application-developed size is very low. The application consumes very low space in disk. Therefore, the user can allocate very minimum local disk space for this application.
Immunizing Image Classifiers Against Localized Adversary Attacksgerogepatton
This paper addresses the vulnerability of deep learning models, particularly convolutional neural networks
(CNN)s, to adversarial attacks and presents a proactive training technique designed to counter them. We
introduce a novel volumization algorithm, which transforms 2D images into 3D volumetric representations.
When combined with 3D convolution and deep curriculum learning optimization (CLO), itsignificantly improves
the immunity of models against localized universal attacks by up to 40%. We evaluate our proposed approach
using contemporary CNN architectures and the modified Canadian Institute for Advanced Research (CIFAR-10
and CIFAR-100) and ImageNet Large Scale Visual Recognition Challenge (ILSVRC12) datasets, showcasing
accuracy improvements over previous techniques. The results indicate that the combination of the volumetric
input and curriculum learning holds significant promise for mitigating adversarial attacks without necessitating
adversary training.
Water scarcity is the lack of fresh water resources to meet the standard water demand. There are two type of water scarcity. One is physical. The other is economic water scarcity.
Explore the innovative world of trenchless pipe repair with our comprehensive guide, "The Benefits and Techniques of Trenchless Pipe Repair." This document delves into the modern methods of repairing underground pipes without the need for extensive excavation, highlighting the numerous advantages and the latest techniques used in the industry.
Learn about the cost savings, reduced environmental impact, and minimal disruption associated with trenchless technology. Discover detailed explanations of popular techniques such as pipe bursting, cured-in-place pipe (CIPP) lining, and directional drilling. Understand how these methods can be applied to various types of infrastructure, from residential plumbing to large-scale municipal systems.
Ideal for homeowners, contractors, engineers, and anyone interested in modern plumbing solutions, this guide provides valuable insights into why trenchless pipe repair is becoming the preferred choice for pipe rehabilitation. Stay informed about the latest advancements and best practices in the field.
Vaccine management system project report documentation..pdfKamal Acharya
The Division of Vaccine and Immunization is facing increasing difficulty monitoring vaccines and other commodities distribution once they have been distributed from the national stores. With the introduction of new vaccines, more challenges have been anticipated with this additions posing serious threat to the already over strained vaccine supply chain system in Kenya.
Student information management system project report ii.pdfKamal Acharya
Our project explains about the student management. This project mainly explains the various actions related to student details. This project shows some ease in adding, editing and deleting the student details. It also provides a less time consuming process for viewing, adding, editing and deleting the marks of the students.
Courier management system project report.pdfKamal Acharya
It is now-a-days very important for the people to send or receive articles like imported furniture, electronic items, gifts, business goods and the like. People depend vastly on different transport systems which mostly use the manual way of receiving and delivering the articles. There is no way to track the articles till they are received and there is no way to let the customer know what happened in transit, once he booked some articles. In such a situation, we need a system which completely computerizes the cargo activities including time to time tracking of the articles sent. This need is fulfilled by Courier Management System software which is online software for the cargo management people that enables them to receive the goods from a source and send them to a required destination and track their status from time to time.
Hybrid optimization of pumped hydro system and solar- Engr. Abdul-Azeez.pdffxintegritypublishin
Advancements in technology unveil a myriad of electrical and electronic breakthroughs geared towards efficiently harnessing limited resources to meet human energy demands. The optimization of hybrid solar PV panels and pumped hydro energy supply systems plays a pivotal role in utilizing natural resources effectively. This initiative not only benefits humanity but also fosters environmental sustainability. The study investigated the design optimization of these hybrid systems, focusing on understanding solar radiation patterns, identifying geographical influences on solar radiation, formulating a mathematical model for system optimization, and determining the optimal configuration of PV panels and pumped hydro storage. Through a comparative analysis approach and eight weeks of data collection, the study addressed key research questions related to solar radiation patterns and optimal system design. The findings highlighted regions with heightened solar radiation levels, showcasing substantial potential for power generation and emphasizing the system's efficiency. Optimizing system design significantly boosted power generation, promoted renewable energy utilization, and enhanced energy storage capacity. The study underscored the benefits of optimizing hybrid solar PV panels and pumped hydro energy supply systems for sustainable energy usage. Optimizing the design of solar PV panels and pumped hydro energy supply systems as examined across diverse climatic conditions in a developing country, not only enhances power generation but also improves the integration of renewable energy sources and boosts energy storage capacities, particularly beneficial for less economically prosperous regions. Additionally, the study provides valuable insights for advancing energy research in economically viable areas. Recommendations included conducting site-specific assessments, utilizing advanced modeling tools, implementing regular maintenance protocols, and enhancing communication among system components.
Forklift Classes Overview by Intella PartsIntella Parts
Discover the different forklift classes and their specific applications. Learn how to choose the right forklift for your needs to ensure safety, efficiency, and compliance in your operations.
For more technical information, visit our website https://intellaparts.com
NO1 Uk best vashikaran specialist in delhi vashikaran baba near me online vas...Amil Baba Dawood bangali
Contact with Dawood Bhai Just call on +92322-6382012 and we'll help you. We'll solve all your problems within 12 to 24 hours and with 101% guarantee and with astrology systematic. If you want to take any personal or professional advice then also you can call us on +92322-6382012 , ONLINE LOVE PROBLEM & Other all types of Daily Life Problem's.Then CALL or WHATSAPP us on +92322-6382012 and Get all these problems solutions here by Amil Baba DAWOOD BANGALI
#vashikaranspecialist #astrologer #palmistry #amliyaat #taweez #manpasandshadi #horoscope #spiritual #lovelife #lovespell #marriagespell#aamilbabainpakistan #amilbabainkarachi #powerfullblackmagicspell #kalajadumantarspecialist #realamilbaba #AmilbabainPakistan #astrologerincanada #astrologerindubai #lovespellsmaster #kalajaduspecialist #lovespellsthatwork #aamilbabainlahore#blackmagicformarriage #aamilbaba #kalajadu #kalailam #taweez #wazifaexpert #jadumantar #vashikaranspecialist #astrologer #palmistry #amliyaat #taweez #manpasandshadi #horoscope #spiritual #lovelife #lovespell #marriagespell#aamilbabainpakistan #amilbabainkarachi #powerfullblackmagicspell #kalajadumantarspecialist #realamilbaba #AmilbabainPakistan #astrologerincanada #astrologerindubai #lovespellsmaster #kalajaduspecialist #lovespellsthatwork #aamilbabainlahore #blackmagicforlove #blackmagicformarriage #aamilbaba #kalajadu #kalailam #taweez #wazifaexpert #jadumantar #vashikaranspecialist #astrologer #palmistry #amliyaat #taweez #manpasandshadi #horoscope #spiritual #lovelife #lovespell #marriagespell#aamilbabainpakistan #amilbabainkarachi #powerfullblackmagicspell #kalajadumantarspecialist #realamilbaba #AmilbabainPakistan #astrologerincanada #astrologerindubai #lovespellsmaster #kalajaduspecialist #lovespellsthatwork #aamilbabainlahore #Amilbabainuk #amilbabainspain #amilbabaindubai #Amilbabainnorway #amilbabainkrachi #amilbabainlahore #amilbabaingujranwalan #amilbabainislamabad
Cosmetic shop management system project report.pdfKamal Acharya
Buying new cosmetic products is difficult. It can even be scary for those who have sensitive skin and are prone to skin trouble. The information needed to alleviate this problem is on the back of each product, but it's thought to interpret those ingredient lists unless you have a background in chemistry.
Instead of buying and hoping for the best, we can use data science to help us predict which products may be good fits for us. It includes various function programs to do the above mentioned tasks.
Data file handling has been effectively used in the program.
The automated cosmetic shop management system should deal with the automation of general workflow and administration process of the shop. The main processes of the system focus on customer's request where the system is able to search the most appropriate products and deliver it to the customers. It should help the employees to quickly identify the list of cosmetic product that have reached the minimum quantity and also keep a track of expired date for each cosmetic product. It should help the employees to find the rack number in which the product is placed.It is also Faster and more efficient way.
4. 4
NVM Solutions Group Intel Confidential
Intel®3DNANDSSDsandOPTANESSDTransformStorage
Expand the reach of Intel® SSDs. Deliver disruptive value to the data center.
11
Refer to appendix for footnotes
Capacity
for Less
Performance
for Less
Optimized STORAGE
5. 5
InnovationforCloudSTORAGE: Intel® Optane™ + Intel® 3D NAND SSDs
New Storage Infrastructure: enable high
performance and cost effective storage:
Journal/Log/Cache Data
Openstack/Ceph:
‒ Intel Optane™ as Journal/Metadata/WAL (Best write
performance, Lowest latency and Best QoS)
‒ Intel 3D NAND TLC SSD as data store (cost effective storage)
‒ Best IOPS/$, IOPS/TB and TB/Rack
Ceph Node (Yesterday)
P3520
2TB
P3520
2TB
P3520
2TB
P3520
2TB
P3700
U.2 800GB
Ceph Node (Today)
Intel® Optane™ P4800X (375GB)
3D NAND
P4500
4TB
3D XPoint™
P4500
4TB
P4500
4TB
P4500
4TB
P4500
4TB
P4500
4TB
Transition to
+
P4500
4TB
P4500
4TB
6. Suggested Configurations for Ceph* Storage Node
Standard/good (baseline):
Use cases/Applications: that need high capacity storage with high
throughput performance
NVMe*/PCIe* SSD for Journal + Caching, HDDs as OSD data
drive
Better IOPS
Use cases/Applications: that need higher performance especially for
throughput, IOPS and SLAs with medium storage capacity requirements
NVMe/PCIe SSD as Journal, High capacity SATA SSD for data
drive
Best Performance
Use cases/Applications: that need highest performance (throughput
and IOPS) and low latency/QoS (Quality of Service).
All NVMe/PCIe SSDs
More information at Ceph.com (new RAs update soon!)
http://tracker.ceph.com/projects/ceph/wiki/Tuning_for_All_Flash_
Deployments
*Other names and brands may be claimed as the property of others.
6
Ceph* storage node --Good
CPU Intel(R) Xeon(R) CPU E5-2650v4
Memory 64 GB
NIC 10GbE
Disks 1x 1.6TB P3700 + 12 x 4TB HDDs (1:12 ratio)
P3700 as Journal and caching
Caching software Intel(R) CAS 3.0, option: Intel(R) RSTe/MD4.3
Ceph* Storage node --Better
CPU Intel(R) Xeon(R) CPU E5-2690v4
Memory 128 GB
NIC Duel 10GbE
Disks 1x Intel(R) DC P3700(800G) + 4x Intel(R) DC S3510 1.6TB
Or 1xIntel P4800X (375GB) + 8x Intel® DC S3520 1.6TB
Ceph* Storage node --Best
CPU Intel(R) Xeon(R) CPU E5-2699v4
Memory >= 128 GB
NIC 2x 40GbE, 4x dual 10GbE
Disks 1xIntel P4800X (375GB) + 6x Intel® DC P4500 4TB
7. 7
Who is using Ceph?
Searchable Examples: http://www.slideshare.net/inktank_ceph
Telcom
CSP/IPDC
OEM/ODM
Enterprise, FSI,
Healthcare,
Retailers
8. 8
Ceph* @ PRC
Ceph* is very hot in PRC
Redevelopment based on the upstream code
More and more companies move to OpenSource storage solutions
Intel/Redhat held Three Ceph days at Beijing and Shanghai
1000+ attendees from 500+ companies
Self media-reports and re-clippings
Next one: Ceph Day Beijing on June, 10th
More and more PRC code contributors
ZTE, XSKY, H3C, LETV, UnitedStack, AliYun, Ebay, EasyStack
9. 9
Ceph* on all-flash array
Storage providers are struggling to achieve the required high performance
There is a growing trend for cloud providers to adopt SSD
– CSP who wants to build EBS alike service for their OpenStack* based public/private cloud
Strong demands to run enterprise applications
OLTP workloads running on Ceph, tail latency is critical
high performance multi-purpose Ceph cluster is a key advantage
Performance is still an important factor
SSD price continue to decrease
*Other names and brands may be claimed as the property of others.
11. 11
Ceph* performance trend with SSD – 4K Random Write
38x performance improvement in Ceph all-flash array!
1.98x
3.7x
1.66x
1.23x
1.19x
12. CEPH2
12
Ceph* All Flash Optane configuration
8x Client Node
• Intel® Xeon™ processor E5-2699 v4 @ 2.3GHz,
64GB mem
• 1x X710 40Gb NIC
8x Storage Node
• Intel Xeon processor E5-2699 v4 @ 2.3 GHz
• 256GB Memory
• 1x 400G SSD for OS
• 1x Intel® DC P4800 375G SSD as WAL and DB
• 8x 2.0TB Intel® SSD DC P4500 as data drive
• 2 OSD instances one each P4500 SSD
• Ceph 12.0.0 with Ubuntu 16.10
2x40Gb NIC
Test Environment
CEPH1
MON
OSD1 OSD16
…
FIO FIO
CLIENT 1
1x40Gb NIC
FIO FIO
CLIENT 2
FIO FIO
CLIENT 3
FIO FIO
…..
FIO FIO
CLIENT8
*Other names and brands may be claimed as the property of others.
CEPH3 … CEPH8
Workloads
• Fio with librbd
• 20x 30 GB volumes each client
• 4 test cases: 4K random read & write; 64K
Sequential read & write
13. 13
Excellent performance on Optane cluster
– random read & write hit CPU bottleneck
Ceph* Optane Performance overview
Throughput Latency (avg.) 99.99% latency
4K Random Read 2876K IOPS 0.9 ms 2.25
4K Random Write 610K IOPS 4.0 ms 25.435
64K Sequential Read 27.5 GB/s 7.6 ms 13.744
64K Sequential Write 13.2 GB/s 11.9 ms 215
14. 14
• Good Node Scalability, poor disk scalability for 4K block workloads (CPU throttled!)
• NUMA & HT helps a lot on the Performance
• Fine tune the # of OSD per node and Drive per Node.
Ceph* Optane Performance – Tunings
15. 15
Ceph* Optane Performance – Performance improvement
• The breakthrough high performance of Optane eliminated the WAL & rocksdb bottleneck
• 1 P4800X or P3700 covers up to 8x P4500 data drivers as both WAL and rocksdb
16. 16
Ceph* Optane Performance – Rocksdb improvement
• Eliminate rocksdb write stall with Optane drive
• Added event listener in rocksdb to provide real-time compaction&&flush information
• Write stalls when flush or compaction can’t keep up with the incoming write rate
• rocksdb_compaction matches submit_transaction latency increase points
17. 17
Ceph* Optane Performance – latency improvement
• Significant tail latency improvement with Optane
• 20x latency reduction for 99.99% latency
5.7 ms
14.7 ms
18.3 ms
317.8 ms
19. 19
Ceph* Optane Performance Analysis
- CPU utilizations
• Random Read & Write performance are throttled by CPU
• Unbalanced CPU utilization caused by HT efficiency for random workloads
• Limiting # of drive scalability for small block random workloads
• Need to optimize CPU utilization
20. Ceph* Optane Performance Analysis
- CPU profiling for 4K RR
• Perf record for 30 second
• Ceph-osd: 34.1%, tp_osd_tp:
65.6%
• Heavy network messenger
overhead
AsyncMsg
20
22. Ceph* Optane Performance Analysis
- CPU profiling for 4K RR
• The top three
consumers of
tp_osd_tp thread are
• KernelDevice::read,
• OSD::ShardedOpWQ::_
process
• PrimaryLogPG::do_op.
• Perf record 30s.
22
23. Ceph* Optane Performance Analysis
- CPU profiling for 4K RW
Wal remove thread(~1.85%)
RocksDB(about ~6%-~7% )
AsyncMsg(~28.9%)
23
25. Ceph* Optane Performance Analysis
- WAL tunings and optimizations
• Key takeaways:
• rocksdb memTable overhead lead to 50000 IOPS difference. (tuning2 vs. tuning3)
• If we don’t clean bluestore WAL data in rocksdb, rocksdb overhead increase dramatically with the # of
metadata increase:
• Use an external WAL device to store WAL data, and just write WAL metadata into rocksdb!
Tunings 4K RW IOPS comments
Default: Baseline(NVMe as DB&& WAL drive) 340000 Separated DB&&WAL device
Tuning1: DB on NVMe && WAL on Ramdisk 360000 Move WAL to Ramdisk
Tuning2: Tuning1+disable rocksdb WAL 360000 RocksDB tuning
Tuning3: Tuning2+omit WAL in deferred write mode 410000 Don’t write WAL in deferred write mode
Tuning4: Tuning1+write WAL but don’t remove WAL
from rocksdb
240000 Write WAL before write metadata into rocksdb, but will not clean WAL
after write data to data device in deferred write mode
Tuning5: Tuning1+external WAL 380000 Write WAL to an external WAL device, and write its metadata to rocksdb
25
26. 26
Ceph* Optane Performance Analysis
- Direct I/O vs Buffered I/O
• Key takeaways:
• On P3700 setup, with bulefs_buffered_io, the performance improved 9%, 99.99% latency improved ~8x
• However on Optane setup, we find bluefs_direct_io could improve the 10% performance, 99.99% latency
also got improved ~1x
0.00x
0.02x
0.06x
0.25x
1.00x
0.00x
0.20x
0.40x
0.60x
0.80x
1.00x
1.20x
1.40x
bluefs_bufferd_io bluefs_direct_io
99.99%_lat(ms)
IOPS
4k rand write performance: buffered vs direct
P3700 iops Optane iops P3700 99.99%_lat Optane 99.99%_lat
27. 27
Ceph* Optane Performance Analysis
- Metadata plane I/O pattern
DB
WAL
• Key takeaways:
• Sync writes!
• Large sequential write w/ offset-overlapping on WAL/DB device
• Small disk reads on DB device due to metadata lookup missing on writes
• BlueFS will save tail part of previous unaligned writes and merge with current writes
• Medium-sized requests due to db transaction batch in BlueStore
29. 29
Summary & Next
Summary
• Ceph* is awesome!
• Strong demands for all-flash array Ceph* solutions
• Optane based all-flash array Ceph* cluster is capable of delivering over 2.8M IOPS
with very low latency!
• Let’s work together to make Ceph* more efficient with all-flash array!
Next
• Client side cache on Optane with SQL workloads!
*Other names and brands may be claimed as the property of others.