Jack Zhang is a Senior Enterprise Architect at Intel Corp. This document discusses Ceph storage configurations using Intel SSDs and discusses benchmark results. Tuning Ceph for all-flash storage can significantly improve performance, with up to 16x better random read performance and 7.6x better random write performance achieved. Using SSDs instead of HDDs provides much higher performance, needing 58x fewer drives for the same write performance and 175x fewer for the same read performance. The document also outlines several suggested Ceph storage node configurations using different ratios of SSDs and HDDs.
Ceph Day Tokyo - Bit-Isle's 3 years footprint with Ceph Ceph Community
Bit-isle has been using Ceph storage with OpenStack for 3 years, starting with a proof of concept in 2013. They have three Ceph environments - a development environment using OpenStack Havana and Ceph Dumpling, a staging environment using OpenStack Juno and Ceph Giant, and a production customer environment using OpenStack Kilo and Ceph Hammer. They chose Ceph because it provides high performance scalable storage without the need for expensive dedicated storage appliances or many storage engineers. Their initial POC was successful and showed Ceph could provide fault tolerance and cooperate well with OpenStack.
Ceph Day Seoul - AFCeph: SKT Scale Out Storage Ceph Ceph Community
SK Telecom is optimizing Ceph for all-flash storage to improve performance and efficiency. Recent work includes enhancing BlueStore, implementing quality of service controls, and exploring data deduplication techniques. Looking ahead, SKT aims to further leverage NVRAM/SSD technologies and expand use of all-flash Ceph in its cloud infrastructure.
Ceph Day Taipei - Delivering cost-effective, high performance, Ceph cluster Ceph Community
An all-NVMe Ceph cluster was configured with 5 storage nodes, each containing 20 Intel P3700 SSDs providing a total of 80 object storage daemons (OSDs). Benchmarking showed over 1.4 million 4K random read IOPS at an average latency of 1ms and 220K 4K random write IOPS at 5ms latency. For a 70/30% read/write mix, over 560K random IOPS were achieved at 3ms latency. Sysbench MySQL testing on the cluster showed linear scaling from 2 to 8 queries per second with an average I/O size of 16KB.
The document discusses accelerating Ceph storage performance using SPDK. SPDK introduces optimizations like asynchronous APIs, userspace I/O stacks, and polling mode drivers to reduce software overhead and better utilize fast storage devices. This allows Ceph to better support high performance networks and storage like NVMe SSDs. The document provides an example where SPDK helped XSKY's BlueStore object store achieve significant performance gains over the standard Ceph implementation.
Ceph Day KL - Ceph Tiering with High Performance ArchiectureCeph Community
Ceph can provide storage tiering with different performance levels. It allows combining SSDs, SAS, and SATA disks from multiple nodes into pools to provide tiered storage. Performance testing showed that for reads, Ceph provided good performance across all tiers, while for writes Nvme disks had the best performance compared to SSD, SAS, and SATA disks. FIO, IOmeter, and IOzone were some of the tools used to measure throughput and IOPS.
Ceph Day Beijing - Ceph All-Flash Array Design Based on NUMA ArchitectureDanielle Womboldt
This document discusses an all-flash Ceph array design from QCT based on NUMA architecture. It provides an agenda that covers all-flash Ceph and use cases, QCT's all-flash Ceph solution for IOPS, an overview of QCT's lab environment and detailed architecture, and the importance of NUMA. It also includes sections on why all-flash storage is used, different all-flash Ceph use cases, QCT's IOPS-optimized all-flash Ceph solution, benefits of using NVMe storage, QCT's lab test environment, Ceph tuning recommendations, and benefits of using multi-partitioned NVMe SSDs for Ceph OSDs.
Ceph Day Tokyo - Bit-Isle's 3 years footprint with Ceph Ceph Community
Bit-isle has been using Ceph storage with OpenStack for 3 years, starting with a proof of concept in 2013. They have three Ceph environments - a development environment using OpenStack Havana and Ceph Dumpling, a staging environment using OpenStack Juno and Ceph Giant, and a production customer environment using OpenStack Kilo and Ceph Hammer. They chose Ceph because it provides high performance scalable storage without the need for expensive dedicated storage appliances or many storage engineers. Their initial POC was successful and showed Ceph could provide fault tolerance and cooperate well with OpenStack.
Ceph Day Seoul - AFCeph: SKT Scale Out Storage Ceph Ceph Community
SK Telecom is optimizing Ceph for all-flash storage to improve performance and efficiency. Recent work includes enhancing BlueStore, implementing quality of service controls, and exploring data deduplication techniques. Looking ahead, SKT aims to further leverage NVRAM/SSD technologies and expand use of all-flash Ceph in its cloud infrastructure.
Ceph Day Taipei - Delivering cost-effective, high performance, Ceph cluster Ceph Community
An all-NVMe Ceph cluster was configured with 5 storage nodes, each containing 20 Intel P3700 SSDs providing a total of 80 object storage daemons (OSDs). Benchmarking showed over 1.4 million 4K random read IOPS at an average latency of 1ms and 220K 4K random write IOPS at 5ms latency. For a 70/30% read/write mix, over 560K random IOPS were achieved at 3ms latency. Sysbench MySQL testing on the cluster showed linear scaling from 2 to 8 queries per second with an average I/O size of 16KB.
The document discusses accelerating Ceph storage performance using SPDK. SPDK introduces optimizations like asynchronous APIs, userspace I/O stacks, and polling mode drivers to reduce software overhead and better utilize fast storage devices. This allows Ceph to better support high performance networks and storage like NVMe SSDs. The document provides an example where SPDK helped XSKY's BlueStore object store achieve significant performance gains over the standard Ceph implementation.
Ceph Day KL - Ceph Tiering with High Performance ArchiectureCeph Community
Ceph can provide storage tiering with different performance levels. It allows combining SSDs, SAS, and SATA disks from multiple nodes into pools to provide tiered storage. Performance testing showed that for reads, Ceph provided good performance across all tiers, while for writes Nvme disks had the best performance compared to SSD, SAS, and SATA disks. FIO, IOmeter, and IOzone were some of the tools used to measure throughput and IOPS.
Ceph Day Beijing - Ceph All-Flash Array Design Based on NUMA ArchitectureDanielle Womboldt
This document discusses an all-flash Ceph array design from QCT based on NUMA architecture. It provides an agenda that covers all-flash Ceph and use cases, QCT's all-flash Ceph solution for IOPS, an overview of QCT's lab environment and detailed architecture, and the importance of NUMA. It also includes sections on why all-flash storage is used, different all-flash Ceph use cases, QCT's IOPS-optimized all-flash Ceph solution, benefits of using NVMe storage, QCT's lab test environment, Ceph tuning recommendations, and benefits of using multi-partitioned NVMe SSDs for Ceph OSDs.
Ceph Day Tokyo - Delivering cost effective, high performance Ceph clusterCeph Community
This document discusses an all-NVMe Ceph cluster configuration for MySQL hosting. It describes a 5-node Ceph cluster with Intel Xeon processors, 128GB of RAM, and 20 Intel SSDs providing 80 object storage devices (OSDs) for a total effective capacity of 19TB. Benchmark results show the cluster achieving over 1.4 million IOPS for 4K random reads with an average latency of 1ms, and over 220K IOPS for 4K random writes with 5ms latency. Sysbench tests of MySQL databases on the cluster using 16KB IOs showed response times under 10ms for query depths from 2 to 8.
Ceph Day Taipei - How ARM Microserver Cluster Performs in CephCeph Community
This document discusses how an ARM-based microserver cluster performs for Ceph storage. It outlines the issues with using a single server node with multiple Ceph OSDs, and describes how using a single micro server with one OSD reduces failure domains and bottlenecks. Test results show increased performance and lower power consumption compared to traditional servers. The document also introduces Ambedded's Ceph storage appliance and management software.
This document analyzes the write amplification factor (WAF) of different Ceph storage configurations. It finds that FileStore has a very high WAF of up to 96x for small writes due to journal-on-journaling. KeyValueStore also has a high WAF from compaction. BlueStore has lower WAF than other options except for small overwrites that require journaling. XFS outperforms ext4 by more efficiently handling journaling. The best configuration depends on the workload and request sizes.
iSCSI provides a standard way to access Ceph block storage remotely over TCP/IP. SUSE Enterprise Storage 3 includes an iSCSI target driver that allows any iSCSI initiator to connect to Ceph storage. This provides multiple platforms with standardized access to Ceph without needing to join the cluster. Optimizations are made in iSCSI to efficiently handle SCER operations by offloading work to OSDs.
openATTIC provides a web-based interface for managing Ceph and other storage. It currently allows pool, OSD, and RBD management along with cluster monitoring. Future plans include extended pool and OSD management, CephFS and RGW integration, and deployment/configuration of Ceph nodes via Salt.
Ceph Day Seoul - Ceph: a decade in the making and still going strong Ceph Community
Ceph began as a research project in 2005 to create a scalable object storage system. It was incubated at various organizations until 2012 when Inktank was formed to support Ceph development and adoption. Inktank focused on releasing stable versions, building infrastructure to test and support Ceph, and growing the developer community. This helped Ceph see widespread adoption in production environments, with Inktank later being acquired by Red Hat in 2014 to further expand Ceph's use.
Intel - optimizing ceph performance by leveraging intel® optane™ and 3 d nand...inwin stack
Kenny Chang (張任伯) (Storage Solution Architect, Intel)
With the trend that Solid State Drive (SSD) becomes more affordable, more and more cloud providers are trying to provide high performance, highly reliable storage for their customers with SSDs. Ceph is becoming one of most open source scale-out storage solutions in worldwide market. More and more customers have strong demands that using SSD in Ceph to build high performance storage solutions for their Openstack clouds.
The disrupted Intel® Optane SSDs based on 3D Xpoint technology fills the performance gap between DRAM and NAND based SSD while the Intel® 3D NAND TLC is reducing cost gap between SSD and traditional spindle hard drive and makes it possible for all flash storage. In this session, we will
1) Discuss OpenStack storage Ceph reference design on the first Intel Optane (3D Xpoint) and P4500 TLC NAND based all-flash Ceph cluster, it delivers multi-million IOPS with extremely low latency as well as increase storage density with competitive dollar-per-gigabyte costs
2) Share Ceph bluestore tunings and optimizations, latency analysis, TCO model, IOPS/TB, IOPS/$ based on the reference architecture to demonstrate this high performance, cost effective solution.
Ceph Day San Jose - All-Flahs Ceph on NUMA-Balanced Server Ceph Community
The document discusses optimizing Ceph storage performance on QCT servers using NUMA-balanced hardware and tuning. It provides details on QCT hardware configurations for throughput, capacity and IOPS-optimized Ceph storage. It also describes testing done in QCT labs using a 5-node all-NVMe Ceph cluster that showed significant performance gains from software tuning and using multiple OSD partitions per SSD.
Ceph Day Seoul - Ceph on Arm Scaleable and Efficient Ceph Community
This document discusses how Ceph storage solutions can benefit from ARM-based platforms. It outlines how the ARM ecosystem provides increased efficiency and scale for Ceph through lower costs, higher energy efficiency, and simplified designs. Examples are given of various companies delivering Ceph clusters using ARM processors, including solutions optimized for microservers, converged infrastructure, and enterprise storage. The recent Jewel release of Ceph added support for the AARCH64 instruction set, opening up additional opportunities for Ceph on ARM platforms.
The document discusses using the Storage Performance Development Kit (SPDK) to optimize Ceph performance. SPDK provides userspace libraries and drivers to unlock the full potential of Intel storage technologies. It summarizes current SPDK support in Ceph's BlueStore backend and proposes leveraging SPDK further to accelerate Ceph's block services through optimized SPDK targets and caching. Collaboration is needed between the SPDK and Ceph communities to fully realize these optimizations.
Ceph Day San Jose - Red Hat Storage Acceleration Utlizing Flash TechnologyCeph Community
The document discusses three ways to accelerate application performance with flash storage using Ceph software defined storage: 1) utilizing all flash storage to maximize performance, 2) using a hybrid configuration with flash and HDDs to balance performance and capacity, and 3) using all HDD storage for maximum capacity but lowest performance. It also examines using NVMe SSDs versus SATA SSDs, and how to optimize Linux settings and Ceph configuration to improve flash performance for applications.
This document outlines an agenda for a presentation on running MySQL on Ceph storage. It includes a comparison of MySQL on Ceph versus AWS, results from a head-to-head performance lab test between the two platforms, and considerations for hardware architectures and configurations optimized for MySQL workloads on Ceph. The lab tests showed that Ceph could match or exceed AWS on both performance metrics like IOPS/GB and price/performance metrics like storage cost per IOP.
This presentation discusses how ARM-based systems can provide scalable and efficient CEPH storage solutions. It outlines how the ARM ecosystem is innovating in storage, including low-cost enterprise SMB storage and energy efficient enterprise storage racks. Examples are given of CEPH implementations on ARM, such as a 504 node CEPH cluster using converged microservers and CEPH performance on ThunderX platforms comparable to x86 servers but with lower total cost of ownership. Overall, the presentation argues ARM-based systems can deliver scalable, portable, and optimized intelligent flexible cloud storage using CEPH.
QCT Ceph Solution - Design Consideration and Reference ArchitecturePatrick McGarry
This document discusses QCT's Ceph storage solutions, including an overview of Ceph architecture, QCT hardware platforms, Red Hat Ceph software, workload considerations, reference architectures, test results and a QCT/Red Hat whitepaper. It provides technical details on QCT's throughput-optimized and capacity-optimized solutions and shows how they address different storage needs through workload-driven design. Hands-on testing and a test drive lab are offered to explore Ceph features and configurations.
The document discusses Ceph storage performance on all-flash storage systems. It describes how SanDisk optimized Ceph for all-flash environments by tuning the OSD to handle the high performance of flash drives. The optimizations allowed over 200,000 IOPS per OSD using 12 CPU cores. Testing on SanDisk's InfiniFlash storage system showed it achieving over 1.5 million random read IOPS and 200,000 random write IOPS at 64KB block size. Latency was also very low, with 99% of operations under 5ms for reads. The document outlines reference configurations for the InfiniFlash system optimized for small, medium and large workloads.
Ceph Day Beijing - Storage Modernization with Intel and CephDanielle Womboldt
The document discusses trends in data growth and storage technologies that are driving the need for storage modernization. It outlines Intel's role in advancing the storage industry through open source technologies and standards. A significant portion of the document focuses on Intel's work optimizing Ceph for Intel platforms, including profiling and benchmarking Ceph performance on Intel SSDs, 3D XPoint, and Optane drives.
Salesforce uses Ceph for various storage needs including block storage, replacing some SAN scenarios, and as a general purpose blob store. They are experimenting with multiple small Ceph clusters across different availability zones. Performance testing shows good random read and write speeds for SSD-only pools. Challenges include scaling to meet their needs, ensuring security and isolation across multiple tenants, and managing clusters across many data centers.
This document provides updates on Ceph community events and initiatives, including:
- An upcoming APAC roadshow for Ceph Days events in various cities from August 18-29
- Metrics on community growth tracked via the Bitergia platform and other means
- The User Committee's work on release cadence, contributor credits, and governance
- Monthly Ceph Tech Talks and Ceph Developer Monthly online meetings
- Plans to redesign the Ceph.com website and launch a new domain
- A traveling Ceph demo pod to demonstrate installations
- An upcoming EdX MOOC course on open source storage in cooperation with The Linux Foundation
- The release and continued testing of the CephFS distributed file
Ceph Day San Jose - From Zero to Ceph in One Minute Ceph Community
Croit is a new startup that aims to simplify Ceph management. Their solution involves live booting Ceph nodes without installing an operating system, managing the entire cluster from a web interface, and allowing any employee to perform basic tasks. Croit was founded by people experienced with Ceph who encountered common problems like complex management scripts and hardware issues. Their goal is to eliminate the need for specialists by automating tasks and enabling easy scaling through a diskless architecture and centralized management portal.
Ceph Day San Jose - Ceph in a Post-Cloud World Ceph Community
Ceph started as a scale-centric storage solution for large-scale applications like cloud storage but is now well-positioned for new use cases at the edge of networks and with IoT devices. With 50 billion IoT devices expected by 2020, data will need to be processed in real-time at the edge rather than in cloud data centers. Ceph is developing new capabilities like support for ARM processors, containerization using Kubernetes, and higher scales of 100PB+ to better serve edge and IoT workloads processing large amounts of data where it is generated. The future of Ceph lies in a post-cloud world where data gravity is located at the network edge rather than in centralized cloud data centers.
The document discusses Ceph performance on all-flash storage systems. It describes optimizations made to Ceph's OSD architecture and write path that have led to significant performance improvements when deployed on SanDisk's InfiniFlash all-flash storage. These include reducing CPU utilization and improving throughput and latency. Example performance metrics are provided showing random read IOPS over 1.5M and latency under 5ms for most operations. The document also outlines the InfiniFlash hardware architecture and roadmap for further Ceph optimizations including new storage backends like BlueStore.
Ceph Day Tokyo - Ceph on ARM: Scaleable and Efficient Ceph Community
This document discusses the opportunity for Ceph storage solutions using ARM processors. It outlines how the ARM ecosystem enables scalable and efficient storage options through increased performance, lower costs, and greater energy efficiency. The recent Jewel release of Ceph added support for AARCH64 processors. Several companies are developing Ceph clusters using ARM-based platforms that demonstrate benefits like reduced power consumption and total cost of ownership compared to x86 solutions.
Ceph Day Tokyo - Delivering cost effective, high performance Ceph clusterCeph Community
This document discusses an all-NVMe Ceph cluster configuration for MySQL hosting. It describes a 5-node Ceph cluster with Intel Xeon processors, 128GB of RAM, and 20 Intel SSDs providing 80 object storage devices (OSDs) for a total effective capacity of 19TB. Benchmark results show the cluster achieving over 1.4 million IOPS for 4K random reads with an average latency of 1ms, and over 220K IOPS for 4K random writes with 5ms latency. Sysbench tests of MySQL databases on the cluster using 16KB IOs showed response times under 10ms for query depths from 2 to 8.
Ceph Day Taipei - How ARM Microserver Cluster Performs in CephCeph Community
This document discusses how an ARM-based microserver cluster performs for Ceph storage. It outlines the issues with using a single server node with multiple Ceph OSDs, and describes how using a single micro server with one OSD reduces failure domains and bottlenecks. Test results show increased performance and lower power consumption compared to traditional servers. The document also introduces Ambedded's Ceph storage appliance and management software.
This document analyzes the write amplification factor (WAF) of different Ceph storage configurations. It finds that FileStore has a very high WAF of up to 96x for small writes due to journal-on-journaling. KeyValueStore also has a high WAF from compaction. BlueStore has lower WAF than other options except for small overwrites that require journaling. XFS outperforms ext4 by more efficiently handling journaling. The best configuration depends on the workload and request sizes.
iSCSI provides a standard way to access Ceph block storage remotely over TCP/IP. SUSE Enterprise Storage 3 includes an iSCSI target driver that allows any iSCSI initiator to connect to Ceph storage. This provides multiple platforms with standardized access to Ceph without needing to join the cluster. Optimizations are made in iSCSI to efficiently handle SCER operations by offloading work to OSDs.
openATTIC provides a web-based interface for managing Ceph and other storage. It currently allows pool, OSD, and RBD management along with cluster monitoring. Future plans include extended pool and OSD management, CephFS and RGW integration, and deployment/configuration of Ceph nodes via Salt.
Ceph Day Seoul - Ceph: a decade in the making and still going strong Ceph Community
Ceph began as a research project in 2005 to create a scalable object storage system. It was incubated at various organizations until 2012 when Inktank was formed to support Ceph development and adoption. Inktank focused on releasing stable versions, building infrastructure to test and support Ceph, and growing the developer community. This helped Ceph see widespread adoption in production environments, with Inktank later being acquired by Red Hat in 2014 to further expand Ceph's use.
Intel - optimizing ceph performance by leveraging intel® optane™ and 3 d nand...inwin stack
Kenny Chang (張任伯) (Storage Solution Architect, Intel)
With the trend that Solid State Drive (SSD) becomes more affordable, more and more cloud providers are trying to provide high performance, highly reliable storage for their customers with SSDs. Ceph is becoming one of most open source scale-out storage solutions in worldwide market. More and more customers have strong demands that using SSD in Ceph to build high performance storage solutions for their Openstack clouds.
The disrupted Intel® Optane SSDs based on 3D Xpoint technology fills the performance gap between DRAM and NAND based SSD while the Intel® 3D NAND TLC is reducing cost gap between SSD and traditional spindle hard drive and makes it possible for all flash storage. In this session, we will
1) Discuss OpenStack storage Ceph reference design on the first Intel Optane (3D Xpoint) and P4500 TLC NAND based all-flash Ceph cluster, it delivers multi-million IOPS with extremely low latency as well as increase storage density with competitive dollar-per-gigabyte costs
2) Share Ceph bluestore tunings and optimizations, latency analysis, TCO model, IOPS/TB, IOPS/$ based on the reference architecture to demonstrate this high performance, cost effective solution.
Ceph Day San Jose - All-Flahs Ceph on NUMA-Balanced Server Ceph Community
The document discusses optimizing Ceph storage performance on QCT servers using NUMA-balanced hardware and tuning. It provides details on QCT hardware configurations for throughput, capacity and IOPS-optimized Ceph storage. It also describes testing done in QCT labs using a 5-node all-NVMe Ceph cluster that showed significant performance gains from software tuning and using multiple OSD partitions per SSD.
Ceph Day Seoul - Ceph on Arm Scaleable and Efficient Ceph Community
This document discusses how Ceph storage solutions can benefit from ARM-based platforms. It outlines how the ARM ecosystem provides increased efficiency and scale for Ceph through lower costs, higher energy efficiency, and simplified designs. Examples are given of various companies delivering Ceph clusters using ARM processors, including solutions optimized for microservers, converged infrastructure, and enterprise storage. The recent Jewel release of Ceph added support for the AARCH64 instruction set, opening up additional opportunities for Ceph on ARM platforms.
The document discusses using the Storage Performance Development Kit (SPDK) to optimize Ceph performance. SPDK provides userspace libraries and drivers to unlock the full potential of Intel storage technologies. It summarizes current SPDK support in Ceph's BlueStore backend and proposes leveraging SPDK further to accelerate Ceph's block services through optimized SPDK targets and caching. Collaboration is needed between the SPDK and Ceph communities to fully realize these optimizations.
Ceph Day San Jose - Red Hat Storage Acceleration Utlizing Flash TechnologyCeph Community
The document discusses three ways to accelerate application performance with flash storage using Ceph software defined storage: 1) utilizing all flash storage to maximize performance, 2) using a hybrid configuration with flash and HDDs to balance performance and capacity, and 3) using all HDD storage for maximum capacity but lowest performance. It also examines using NVMe SSDs versus SATA SSDs, and how to optimize Linux settings and Ceph configuration to improve flash performance for applications.
This document outlines an agenda for a presentation on running MySQL on Ceph storage. It includes a comparison of MySQL on Ceph versus AWS, results from a head-to-head performance lab test between the two platforms, and considerations for hardware architectures and configurations optimized for MySQL workloads on Ceph. The lab tests showed that Ceph could match or exceed AWS on both performance metrics like IOPS/GB and price/performance metrics like storage cost per IOP.
This presentation discusses how ARM-based systems can provide scalable and efficient CEPH storage solutions. It outlines how the ARM ecosystem is innovating in storage, including low-cost enterprise SMB storage and energy efficient enterprise storage racks. Examples are given of CEPH implementations on ARM, such as a 504 node CEPH cluster using converged microservers and CEPH performance on ThunderX platforms comparable to x86 servers but with lower total cost of ownership. Overall, the presentation argues ARM-based systems can deliver scalable, portable, and optimized intelligent flexible cloud storage using CEPH.
QCT Ceph Solution - Design Consideration and Reference ArchitecturePatrick McGarry
This document discusses QCT's Ceph storage solutions, including an overview of Ceph architecture, QCT hardware platforms, Red Hat Ceph software, workload considerations, reference architectures, test results and a QCT/Red Hat whitepaper. It provides technical details on QCT's throughput-optimized and capacity-optimized solutions and shows how they address different storage needs through workload-driven design. Hands-on testing and a test drive lab are offered to explore Ceph features and configurations.
The document discusses Ceph storage performance on all-flash storage systems. It describes how SanDisk optimized Ceph for all-flash environments by tuning the OSD to handle the high performance of flash drives. The optimizations allowed over 200,000 IOPS per OSD using 12 CPU cores. Testing on SanDisk's InfiniFlash storage system showed it achieving over 1.5 million random read IOPS and 200,000 random write IOPS at 64KB block size. Latency was also very low, with 99% of operations under 5ms for reads. The document outlines reference configurations for the InfiniFlash system optimized for small, medium and large workloads.
Ceph Day Beijing - Storage Modernization with Intel and CephDanielle Womboldt
The document discusses trends in data growth and storage technologies that are driving the need for storage modernization. It outlines Intel's role in advancing the storage industry through open source technologies and standards. A significant portion of the document focuses on Intel's work optimizing Ceph for Intel platforms, including profiling and benchmarking Ceph performance on Intel SSDs, 3D XPoint, and Optane drives.
Salesforce uses Ceph for various storage needs including block storage, replacing some SAN scenarios, and as a general purpose blob store. They are experimenting with multiple small Ceph clusters across different availability zones. Performance testing shows good random read and write speeds for SSD-only pools. Challenges include scaling to meet their needs, ensuring security and isolation across multiple tenants, and managing clusters across many data centers.
This document provides updates on Ceph community events and initiatives, including:
- An upcoming APAC roadshow for Ceph Days events in various cities from August 18-29
- Metrics on community growth tracked via the Bitergia platform and other means
- The User Committee's work on release cadence, contributor credits, and governance
- Monthly Ceph Tech Talks and Ceph Developer Monthly online meetings
- Plans to redesign the Ceph.com website and launch a new domain
- A traveling Ceph demo pod to demonstrate installations
- An upcoming EdX MOOC course on open source storage in cooperation with The Linux Foundation
- The release and continued testing of the CephFS distributed file
Ceph Day San Jose - From Zero to Ceph in One Minute Ceph Community
Croit is a new startup that aims to simplify Ceph management. Their solution involves live booting Ceph nodes without installing an operating system, managing the entire cluster from a web interface, and allowing any employee to perform basic tasks. Croit was founded by people experienced with Ceph who encountered common problems like complex management scripts and hardware issues. Their goal is to eliminate the need for specialists by automating tasks and enabling easy scaling through a diskless architecture and centralized management portal.
Ceph Day San Jose - Ceph in a Post-Cloud World Ceph Community
Ceph started as a scale-centric storage solution for large-scale applications like cloud storage but is now well-positioned for new use cases at the edge of networks and with IoT devices. With 50 billion IoT devices expected by 2020, data will need to be processed in real-time at the edge rather than in cloud data centers. Ceph is developing new capabilities like support for ARM processors, containerization using Kubernetes, and higher scales of 100PB+ to better serve edge and IoT workloads processing large amounts of data where it is generated. The future of Ceph lies in a post-cloud world where data gravity is located at the network edge rather than in centralized cloud data centers.
The document discusses Ceph performance on all-flash storage systems. It describes optimizations made to Ceph's OSD architecture and write path that have led to significant performance improvements when deployed on SanDisk's InfiniFlash all-flash storage. These include reducing CPU utilization and improving throughput and latency. Example performance metrics are provided showing random read IOPS over 1.5M and latency under 5ms for most operations. The document also outlines the InfiniFlash hardware architecture and roadmap for further Ceph optimizations including new storage backends like BlueStore.
Ceph Day Tokyo - Ceph on ARM: Scaleable and Efficient Ceph Community
This document discusses the opportunity for Ceph storage solutions using ARM processors. It outlines how the ARM ecosystem enables scalable and efficient storage options through increased performance, lower costs, and greater energy efficiency. The recent Jewel release of Ceph added support for AARCH64 processors. Several companies are developing Ceph clusters using ARM-based platforms that demonstrate benefits like reduced power consumption and total cost of ownership compared to x86 solutions.
The document discusses implementing a high availability NAS gateway using CephFS. A 6 node Ceph cluster with 80TB of usable storage was configured with 2 gateway servers running Samba and Ganesha NFS to provide SMB and NFS shares of the CephFS filesystem. CTDB was used to enable failover between the gateways and floating IP addresses allow seamless failover. Some issues were encountered around snapshots, quotas and performance, but the overall solution provides high availability NAS capabilities using standard protocols and CephFS.
Ceph Day San Jose - Enable Fast Big Data Analytics on Ceph with Alluxio Ceph Community
Adit Madan from Alluxio presented how to enable fast big data analytics on Ceph object storage using Alluxio. Alluxio acts as a virtual distributed file system that caches data in memory to accelerate access to data stored in Ceph. This provides orders of magnitude faster performance for Spark queries on large datasets. A demo on EC2 showed Spark counting a 60GB dataset was 20x faster when using Alluxio to cache data from Ceph compared to directly accessing Ceph.
Ceph Day San Jose - Object Storage for Big Data Ceph Community
This document discusses using object storage for big data. It outlines key stakeholders in big data projects and what they want from object storage solutions. It then discusses using the Ceph object store to provide an elastic data lake that can disaggregate compute resources from storage. This allows analytics to be performed directly on the object store without expensive ETL processes. It also describes testing various analytics use cases and workloads with the Ceph object store.
This document provides updates on Ceph community events and initiatives, including:
- An upcoming APAC roadshow for Ceph Days events in various cities from August 18-29
- Metrics on community growth tracked via various platforms
- Details on the User Committee and steps towards broader governance
- Recurring technical talks and developer meetings held monthly
- Plans to redesign the Ceph.com website on a new hosting platform
- A traveling Ceph demo pod to demonstrate the technology
- An upcoming MOOC course on open source storage technologies including Ceph
- Notes on the new stable release of CephFS in Jewel and need for more testing
- Announcement of Cephalocon 2017 conference
London Ceph Day: Unified Cloud Storage with Synnefo + Ceph + GanetiCeph Community
Vangelis Koukis presented on the Greek Research and Technology Network's (GRNET) public cloud service called Okeanos, which uses Synnefo, Ganeti, and Ceph to provide a production-quality IaaS cloud. Okeanos has been in production since 2011, currently supports over 3,500 users and 5,500 active VMs after initially spawning over 160,000 VMs. The presentation discussed the architecture, challenges of operating a public cloud with persistent VMs, and experiences with rolling upgrades, live migrations, and scaling the cloud infrastructure.
- Librados is a C/C++ library and programming interface that provides applications access to the Ceph distributed object store (RADOS) and hides the complexity of networking, data distribution, replication and failure recovery.
- It has language bindings for C, C++, Python, Java and other languages and provides both low-level and high-level APIs.
- Librados is used by many Ceph components and tools like RADOS Gateway (RGW) and the rados command line interface. It also enables developers to build custom applications that use Ceph for storage including applications for mail systems, Hadoop, logging and more.
- Ctrip uses Ceph for storage in their private cloud and VDesktop environments, storing over 10 petabytes of data across various use cases including databases, file sharing, and big data.
- They implemented Ceph RGW with Swift API for object storage to enable continuous delivery of software packages across data centers with low latency access.
- For data sync between data centers, Ctrip deployed a federated Ceph gateway solution instead of direct sync due to issues with stability, flexibility, and scalability of other approaches.
i. SUSE Enterprise Storage 3 provides iSCSI access to connect remotely to ceph storage over TCP/IP, allowing any iSCSI initiator to access the storage over a network. The iSCSI target driver sits on top of RBD (RADOS block device) to enable this access.
ii. Configuring the lrbd package simplifies setting up an iSCSI gateway to Ceph. Multiple gateways can be configured for high availability using targetcli utility.
iii. Optimizations have been made to the iSCSI gateway to efficiently handle certain SCSI operations like atomic compare and write by offloading work to OSDs to avoid locking on gateway nodes.
Ceph Day Shanghai - CeTune - Benchmarking and tuning your Ceph cluster Ceph Community
CeTune is an open source toolkit for deploying, benchmarking, analyzing, and tuning Ceph clusters. It provides a web UI for configuring Ceph deployments and benchmarks, and generates reports with performance metrics and system analysis. CeTune aims to simplify tuning Ceph for beginners and engineers, and provide an easy way for developers to stress test and analyze performance. It includes modules for deployment, benchmarking with tools like FIO and Cosbench, system analysis with tools like iostat and sar, and tuning Ceph configurations.
Ceph Day Shanghai - SSD/NVM Technology Boosting Ceph Performance Ceph Community
This document discusses using SSDs and emerging non-volatile memory technologies like 3D XPoint to boost performance of Ceph storage clusters. It outlines how SSDs can be used as journals and caches to significantly increase throughput and reduce latency compared to HDD-only clusters. A case study from Yahoo showed that using Intel NVMe SSDs with caching software delivered over 2x throughput and half the latency with only 5% of data cached. Future technologies like 3D NAND and 3D XPoint will allow building higher performance, higher capacity SSDs that could extend the use of Ceph.
Ceph Day Shanghai - Recovery Erasure Coding and Cache TieringCeph Community
This document discusses recovery, erasure coding, and cache tiering in Ceph. It provides an overview of the RADOS components including OSDs, monitors, and CRUSH, which calculates data placement across the cluster. It describes how peering and recovery work to maintain data consistency. It also outlines how Ceph implements tiered storage with cache and backing pools, using erasure coding for durability and caching techniques to improve performance.
This document discusses tools for analyzing Ceph performance. It begins by describing common performance issues users encounter with Ceph and potential solutions like tuning configuration values or benchmarking. The rest of the document details various monitoring and benchmarking tools that can help identify bottlenecks like the dispatch layer, object store, or hardware. It provides examples of using tools like dstat, iostat, perf, systemtap, ceph perf dump, and benchmarking tools like Fio, rbd-replay and ceph_perf_local. It concludes with a case study where unaligned partitions and a driver bug were causing low IOPS that were resolved by fixing the partition alignment and downgrading the NVMe driver.
Accelerating Cassandra Workloads on Ceph with All-Flash PCIE SSDSCeph Community
This document summarizes the performance of an all-NVMe Ceph cluster using Intel P3700 NVMe SSDs. Key results include achieving over 1.35 million 4K random read IOPS and 171K 4K random write IOPS with sub-millisecond latency. Partitioning the NVMe drives into multiple OSDs improved performance and CPU utilization compared to a single OSD per drive. The cluster also demonstrated over 5GB/s of sequential bandwidth.
The document summarizes Intel's new Solid-State Drive Data Center Family for PCIe. It provides an overview of Intel's SSD product families for different market segments. It then focuses on the new Data Center Family for PCIe, highlighting its native PCIe interface, performance benefits over SAS/SATA, endurance, reliability features, and product lineup. Finally, it lists upcoming events where Intel will promote the new data center SSD family.
Ceph Day Beijing - Storage Modernization with Intel & Ceph Ceph Community
The document discusses trends in data growth and storage technologies that are driving the need for storage modernization. It outlines Intel's role in advancing the storage industry through open source technologies and standards. Specifically, it focuses on Intel's work optimizing Ceph for Intel platforms, including performance profiling, enabling Intel optimized solutions, and end customer proofs-of-concept using Ceph with Intel SSDs, Optane, and platforms.
The document discusses optimizing Ceph performance by leveraging Intel Optane and 3D NAND TLC SSDs. It provides an overview of a Ceph cluster configuration with Intel Optane SSDs for journal/metadata and Intel 3D NAND SSDs for data storage. The document analyzes the performance of this all-flash Ceph cluster, finding over 2.8 million IOPS with low latency. It also examines various optimizations that could further improve Ceph performance on Intel Optane-based storage.
Ceph Day Beijing - Optimizing Ceph Performance by Leveraging Intel Optane and...Danielle Womboldt
Optimizing Ceph performance by leveraging Intel Optane and 3D NAND TLC SSDs. The document discusses using Intel Optane SSDs as journal/metadata drives and Intel 3D NAND SSDs as data drives in Ceph clusters. It provides examples of configurations and analysis of a 2.8 million IOPS Ceph cluster using this approach. Tuning recommendations are also provided to optimize performance.
Ceph Day Beijing - Optimizing Ceph performance by leveraging Intel Optane and...Ceph Community
The document discusses optimizing Ceph performance by leveraging Intel Optane and 3D NAND TLC SSDs. It provides an overview of using Optane SSDs as journal/metadata drives and 3D NAND SSDs as data drives in Ceph configurations. It then describes achieving 2.8 million IOPS on an all-flash Ceph cluster with this configuration and analyzes the performance of Optane-based all-flash Ceph arrays. Various tunings are also discussed to improve Ceph performance on Optane drives.
Forwarding Plane Opportunities: How to Accelerate DeploymentCharo Sanchez
Intel® Select Solution for NFVI Forwarding Platform (NFVI FP) is an enhanced NFVI solution for 4G or 5G core User Plane Functions (UPF), broadband use cases, such as virtual Broadband Network Gateway (vBNG), network services such as virtual Evolved Packet Core (vEPC), IPsec Gateways (vSecGW), and cable use cases such as virtual Cable Modem Termination System (vCMTS) that demand high performance and packet processing throughput. The Advantech SKY-8101D server is a verified Intel Select Solution for NFVI FP plus, base and controller node with Red Hat Enterprise Linux and Red Hat OpenStack tuned to meet a performance threshold capable of serving large numbers of subscribers thanks to a more efficient use of the infrastructure for lower TCO.
Cерверы Depo storm 3400 на базе новейших процессоров intel xeon e5 2600v3 finDEPO Computers
The document discusses a webinar on DEPO Storm 3400 servers based on the latest Intel Xeon E5-2600v3 processors. The webinar will include presentations on the new processor capabilities and related technologies like DDR4, SAS 12Gbit, NVM-E. It will also present DEPO's Storm 3400 server models that implement these technologies and discuss example server infrastructure solutions for companies of different sizes. Attendees will be able to ask questions and download materials after the webinar.
The document discusses using the Storage Performance Development Kit (SPDK) to optimize Ceph storage performance. SPDK provides userspace libraries and drivers to unlock the full potential of Intel storage technologies. It summarizes current SPDK support in Ceph's BlueStore backend and proposes leveraging SPDK further to accelerate Ceph's block services using optimized SPDK targets and caching. Collaboration is needed between the SPDK and Ceph communities to fully realize these performance benefits.
This document discusses security features of the SPARC M7 CPU. It introduces Silicon Secured Memory, which provides hardware-based memory protection to stop malicious programs from accessing other application memory without performance impact. This results in improved security, reliability, and availability of applications. Benchmark results are also provided showing the SPARC M7's performance advantages over other chips.
Webinář "Konsolidace Oracle DB na systémech s procesory M7, včetně migrace z konkurenčních serverových platforem"
Prezentuje Josef Šlahůnek, Oracle
9.3.2016
Red Hat Storage Day Atlanta - Designing Ceph Clusters Using Intel-Based Hardw...Red_Hat_Storage
This document discusses the need for storage modernization driven by trends like mobile, social media, IoT and big data. It outlines how scale-out architectures using open source Ceph software can help meet this need more cost effectively than traditional scale-up storage. Specific optimizations for IOPS, throughput and capacity are described. Intel is presented as helping advance the industry through open source contributions and optimized platforms, software and SSD technologies. Real-world examples are given showing the wide performance range Ceph can provide.
Ceph on Intel: Intel Storage Components, Benchmarks, and ContributionsColleen Corrice
At Red Hat Storage Day Minneapolis on 4/12/16, Intel's Dan Ferber presented on Intel storage components, benchmarks, and contributions as they relate to Ceph.
Ceph on Intel: Intel Storage Components, Benchmarks, and ContributionsRed_Hat_Storage
At Red Hat Storage Day Minneapolis on 4/12/16, Intel's Dan Ferber presented on Intel storage components, benchmarks, and contributions as they relate to Ceph.
The document discusses Intel technologies for high performance computing. It provides an overview of Intel's product families targeted at HPC workloads, including the Xeon E5-2600 v3 and E7-8800 v3 processor families. It also reviews some basics of HPC, including factors that impact performance such as memory bandwidth and latency. The document emphasizes that data movement between the CPU and memory hierarchy can often be a bottleneck, and that optimizing for high floating point operations per memory access is important.
Accelerate Your Apache Spark with Intel Optane DC Persistent MemoryDatabricks
The capacity of data grows rapidly in big data area, more and more memory are consumed either in the computation or holding the intermediate data for analytic jobs. For those memory intensive workloads, end-point users have to scale out the computation cluster or extend memory with storage like HDD or SSD to meet the requirement of computing tasks. For scaling out the cluster, the extra cost from cluster management, operation and maintenance will increase the total cost if the extra CPU resources are not fully utilized. To address the shortcoming above, Intel Optane DC persistent memory (Optane DCPM) breaks the traditional memory/storage hierarchy and scale up the computing server with higher capacity persistent memory. Also it brings higher bandwidth & lower latency than storage like SSD or HDD. And Apache Spark is widely used in the analytics like SQL and Machine Learning on the cloud environment. For cloud environment, low performance of remote data access is typical a stop gap for users especially for some I/O intensive queries. For the ML workload, it's an iterative model which I/O bandwidth is the key to the end-2-end performance. In this talk, we will introduce how to accelerate Spark SQL with OAP (https://github.com/Intel-bigdata/OAP) to accelerate SQL performance on Cloud to archive 8X performance gain and RDD cache to improve K-means performance with 2.5X performance gain leveraging Intel Optane DCPM. Also we will have a deep dive how Optane DCPM for these performance gains.
Speakers: Cheng Xu, Piotr Balcer
The document discusses the HP ProLiant DL580 Gen8 server. It provides details on the server's breakthrough 4S performance and scalability as well as its leading x86 availability and reliability. The document also summarizes the server's compelling efficiencies and enhanced capabilities over previous generations such as increased processor and memory performance, bandwidth, and storage capacities.
The document discusses Ceph storage performance on all-flash storage systems. It notes that Ceph was originally optimized for HDDs and required tuning and algorithm changes to achieve flash-level performance. SanDisk worked with the Ceph community to optimize the object storage daemon (OSD) for flash, improving read and write throughput. Benchmark results show the SanDisk InfiniFlash system delivering over 1 million IOPS and 15GB/s throughput using Ceph software. Reference configurations provide guidance on hardware requirements for small, medium, and large workloads.
Accelerating Virtual Machine Access with the Storage Performance Development ...Michelle Holley
Abstract: Although new non-volatile media inherently offers very low latency, remote access
using protocols such as NVMe-oF and presenting the data to VMs via virtualized interfaces such as virtio
adds considerable software overhead. One way to reduce the overhead is to use the Storage
Performance Development Kit (SPDK), an open-source software project that provides building blocks for
scalable and efficient storage applications with breakthrough performance. Comparing the software
paths for virtualizing block storage I/O illustrates the advantages of the SPDK-based approach. Empirical
data shows that using SPDK can improve CPU efficiency by up to 10 x and reduce latency up to 50% over
existing methods. Future enhancements for SPDK will make its advantages even greater.
Speaker Bio: Anu Rao is Product line manager for storage software in Data center Group. She helps
customer ease into and adopt open source Storage software like Storage Performance Development Kit
(SPDK) and Intelligent Software Acceleration-Library (ISA-L).
Similar to Ceph Day Seoul - Delivering Cost Effective, High Performance Ceph cluster (20)
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slackshyamraj55
Discover the seamless integration of RPA (Robotic Process Automation), COMPOSER, and APM with AWS IDP enhanced with Slack notifications. Explore how these technologies converge to streamline workflows, optimize performance, and ensure secure access, all while leveraging the power of AWS IDP and real-time communication via Slack notifications.
A Comprehensive Guide to DeFi Development Services in 2024Intelisync
DeFi represents a paradigm shift in the financial industry. Instead of relying on traditional, centralized institutions like banks, DeFi leverages blockchain technology to create a decentralized network of financial services. This means that financial transactions can occur directly between parties, without intermediaries, using smart contracts on platforms like Ethereum.
In 2024, we are witnessing an explosion of new DeFi projects and protocols, each pushing the boundaries of what’s possible in finance.
In summary, DeFi in 2024 is not just a trend; it’s a revolution that democratizes finance, enhances security and transparency, and fosters continuous innovation. As we proceed through this presentation, we'll explore the various components and services of DeFi in detail, shedding light on how they are transforming the financial landscape.
At Intelisync, we specialize in providing comprehensive DeFi development services tailored to meet the unique needs of our clients. From smart contract development to dApp creation and security audits, we ensure that your DeFi project is built with innovation, security, and scalability in mind. Trust Intelisync to guide you through the intricate landscape of decentralized finance and unlock the full potential of blockchain technology.
Ready to take your DeFi project to the next level? Partner with Intelisync for expert DeFi development services today!
Introduction of Cybersecurity with OSS at Code Europe 2024Hiroshi SHIBATA
I develop the Ruby programming language, RubyGems, and Bundler, which are package managers for Ruby. Today, I will introduce how to enhance the security of your application using open-source software (OSS) examples from Ruby and RubyGems.
The first topic is CVE (Common Vulnerabilities and Exposures). I have published CVEs many times. But what exactly is a CVE? I'll provide a basic understanding of CVEs and explain how to detect and handle vulnerabilities in OSS.
Next, let's discuss package managers. Package managers play a critical role in the OSS ecosystem. I'll explain how to manage library dependencies in your application.
I'll share insights into how the Ruby and RubyGems core team works to keep our ecosystem safe. By the end of this talk, you'll have a better understanding of how to safeguard your code.
Digital Marketing Trends in 2024 | Guide for Staying AheadWask
https://www.wask.co/ebooks/digital-marketing-trends-in-2024
Feeling lost in the digital marketing whirlwind of 2024? Technology is changing, consumer habits are evolving, and staying ahead of the curve feels like a never-ending pursuit. This e-book is your compass. Dive into actionable insights to handle the complexities of modern marketing. From hyper-personalization to the power of user-generated content, learn how to build long-term relationships with your audience and unlock the secrets to success in the ever-shifting digital landscape.
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...Tatiana Kojar
Skybuffer AI, built on the robust SAP Business Technology Platform (SAP BTP), is the latest and most advanced version of our AI development, reaffirming our commitment to delivering top-tier AI solutions. Skybuffer AI harnesses all the innovative capabilities of the SAP BTP in the AI domain, from Conversational AI to cutting-edge Generative AI and Retrieval-Augmented Generation (RAG). It also helps SAP customers safeguard their investments into SAP Conversational AI and ensure a seamless, one-click transition to SAP Business AI.
With Skybuffer AI, various AI models can be integrated into a single communication channel such as Microsoft Teams. This integration empowers business users with insights drawn from SAP backend systems, enterprise documents, and the expansive knowledge of Generative AI. And the best part of it is that it is all managed through our intuitive no-code Action Server interface, requiring no extensive coding knowledge and making the advanced AI accessible to more users.
Programming Foundation Models with DSPy - Meetup SlidesZilliz
Prompting language models is hard, while programming language models is easy. In this talk, I will discuss the state-of-the-art framework DSPy for programming foundation models with its powerful optimizers and runtime constraint system.
5th LF Energy Power Grid Model Meet-up SlidesDanBrown980551
5th Power Grid Model Meet-up
It is with great pleasure that we extend to you an invitation to the 5th Power Grid Model Meet-up, scheduled for 6th June 2024. This event will adopt a hybrid format, allowing participants to join us either through an online Mircosoft Teams session or in person at TU/e located at Den Dolech 2, Eindhoven, Netherlands. The meet-up will be hosted by Eindhoven University of Technology (TU/e), a research university specializing in engineering science & technology.
Power Grid Model
The global energy transition is placing new and unprecedented demands on Distribution System Operators (DSOs). Alongside upgrades to grid capacity, processes such as digitization, capacity optimization, and congestion management are becoming vital for delivering reliable services.
Power Grid Model is an open source project from Linux Foundation Energy and provides a calculation engine that is increasingly essential for DSOs. It offers a standards-based foundation enabling real-time power systems analysis, simulations of electrical power grids, and sophisticated what-if analysis. In addition, it enables in-depth studies and analysis of the electrical power grid’s behavior and performance. This comprehensive model incorporates essential factors such as power generation capacity, electrical losses, voltage levels, power flows, and system stability.
Power Grid Model is currently being applied in a wide variety of use cases, including grid planning, expansion, reliability, and congestion studies. It can also help in analyzing the impact of renewable energy integration, assessing the effects of disturbances or faults, and developing strategies for grid control and optimization.
What to expect
For the upcoming meetup we are organizing, we have an exciting lineup of activities planned:
-Insightful presentations covering two practical applications of the Power Grid Model.
-An update on the latest advancements in Power Grid -Model technology during the first and second quarters of 2024.
-An interactive brainstorming session to discuss and propose new feature requests.
-An opportunity to connect with fellow Power Grid Model enthusiasts and users.
GraphRAG for Life Science to increase LLM accuracyTomaz Bratanic
GraphRAG for life science domain, where you retriever information from biomedical knowledge graphs using LLMs to increase the accuracy and performance of generated answers
Main news related to the CCS TSI 2023 (2023/1695)Jakub Marek
An English 🇬🇧 translation of a presentation to the speech I gave about the main changes brought by CCS TSI 2023 at the biggest Czech conference on Communications and signalling systems on Railways, which was held in Clarion Hotel Olomouc from 7th to 9th November 2023 (konferenceszt.cz). Attended by around 500 participants and 200 on-line followers.
The original Czech 🇨🇿 version of the presentation can be found here: https://www.slideshare.net/slideshow/hlavni-novinky-souvisejici-s-ccs-tsi-2023-2023-1695/269688092 .
The videorecording (in Czech) from the presentation is available here: https://youtu.be/WzjJWm4IyPk?si=SImb06tuXGb30BEH .
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/temporal-event-neural-networks-a-more-efficient-alternative-to-the-transformer-a-presentation-from-brainchip/
Chris Jones, Director of Product Management at BrainChip , presents the “Temporal Event Neural Networks: A More Efficient Alternative to the Transformer” tutorial at the May 2024 Embedded Vision Summit.
The expansion of AI services necessitates enhanced computational capabilities on edge devices. Temporal Event Neural Networks (TENNs), developed by BrainChip, represent a novel and highly efficient state-space network. TENNs demonstrate exceptional proficiency in handling multi-dimensional streaming data, facilitating advancements in object detection, action recognition, speech enhancement and language model/sequence generation. Through the utilization of polynomial-based continuous convolutions, TENNs streamline models, expedite training processes and significantly diminish memory requirements, achieving notable reductions of up to 50x in parameters and 5,000x in energy consumption compared to prevailing methodologies like transformers.
Integration with BrainChip’s Akida neuromorphic hardware IP further enhances TENNs’ capabilities, enabling the realization of highly capable, portable and passively cooled edge devices. This presentation delves into the technical innovations underlying TENNs, presents real-world benchmarks, and elucidates how this cutting-edge approach is positioned to revolutionize edge AI across diverse applications.
Trusted Execution Environment for Decentralized Process MiningLucaBarbaro3
Presentation of the paper "Trusted Execution Environment for Decentralized Process Mining" given during the CAiSE 2024 Conference in Cyprus on June 7, 2024.
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...alexjohnson7307
Predictive maintenance is a proactive approach that anticipates equipment failures before they happen. At the forefront of this innovative strategy is Artificial Intelligence (AI), which brings unprecedented precision and efficiency. AI in predictive maintenance is transforming industries by reducing downtime, minimizing costs, and enhancing productivity.
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyScyllaDB
Freshworks creates AI-boosted business software that helps employees work more efficiently and effectively. Managing data across multiple RDBMS and NoSQL databases was already a challenge at their current scale. To prepare for 10X growth, they knew it was time to rethink their database strategy. Learn how they architected a solution that would simplify scaling while keeping costs under control.
3. Agenda
• Introduction, Ceph at Intel
• All-flash Ceph configurations and benchmark data
• OEMs/ISVs/Intel Ceph Reference Architects/Recipes
• Future Ceph* with Intel NVM Technologies
3D XpointTM and 3D NAND SSD
• Summary
3*Other names and brands may be claimed as the property of others.
4. 4
Acknowledgements
This is team work.
Thanks for the contributions of Intel Team:
PRC team: Jian Zhang, Yuan Zhou, Haodong Tang, Jianpeng Ma, Ning Li
US team: Daniel Ferber, Tushar Gohad, Orlando Moreno, Anjaneya Chagam
6. 6
Ceph at Intel – A brief introduction
Optimize for Intel® platforms, flash and networking
• Compression, Encryption hardware offloads (QAT & SOCs)
• PMStore (for 3D XPoint DIMMs)
• RBD caching and Cache tiering with NVM
• IA optimized storage libraries to reduce latency (ISA-L, SPDK)
Performance profiling, analysis and community contributions
• All flash workload profiling and latency analysis, performance portal http://01.org/cephperf
• Streaming, Database and Analytics workload driven optimizations
Ceph enterprise usages and hardening
• Manageability (Virtual Storage Manager)
• Multi Data Center clustering (e.g., async mirroring)
End Customer POCs with focus on broad industry influence
• CDN, Cloud DVR, Video Surveillance, Ceph Cloud Services, Analytics
• Working with 50+ customers to help them enabling Ceph based storage solutions
POCs
Ready to use IA, Intel NVM optimized systems & solutions from OEMs & ISVs
• Ready to use IA, Intel NVM optimized systems & solutions from OEMs & ISVs
• Intel system configurations, white papers, case studies
• Industry events coverage
Go to
market
Intel® Storage
Acceleration Library
(Intel® ISA-L)
Intel® Storage Performance
Development Kit (Intel® SPDK)
Intel® Cache Acceleration
Software (Intel® CAS)
Virtual Storage Manager Ce-Tune Ceph Profiler
7. 7
Intel Ceph Contribution Timeline
2014 2015 2016
* Right Edge of box indicates approximate release date
New Key/Value Store
Backend (rocksdb)
Giant* Hammer Infernalis Jewel
CRUSH Placement
Algorithm improvements
(straw2 bucket type)
Bluestore Backend
Optimizations for NVM
Bluestore SPDK
Optimizations
RADOS I/O Hinting
(35% better EC Write erformance)
Cache-tiering with SSDs
(Write support)
PMStore
(NVM-optimized backend
based on libpmem)
RGW, Bluestore
Compression, Encryption
(w/ ISA-L, QAT backend)
Virtual Storage Manager
(VSM) Open Sourced
CeTune
Open Sourced
Erasure Coding
support with ISA-L
Cache-tiering with SSDs
(Read support)
Client-side Block Cache
(librbd)
11. Suggested Configurations for Ceph* Storage Node
Standard/good (baseline):
Use cases/Applications: that need high capacity storage with high
throughput performance
NVMe*/PCIe* SSD for Journal + Caching, HDDs as OSD data drive
Example: 1x 1.6TB Intel® SSD DC P3700 as Journal + Intel® Cache
Acceleration Software (Intel® CAS) + 12 HDDs
Better IOPS
Use cases/Applications: that need higher performance especially for
throughput, IOPS and SLAs with medium storage capacity requirements
NVMe/PCIe SSD as Journal, no caching, High capacity SATA SSD for
data drive
Example: 1x 800GB Intel® SSD DC P3700 + 4 to 6x 1.6TB DC S3510
Best Performance
Use cases/Applications: that need highest performance (throughput
and IOPS) and low latency.
All NVMe/PCIe SSDs
Example: 4 to 6 x 2TB Intel SSD DC P3700 Series
More Information: https://intelassetlibrary.tagcmd.com/#assets/gallery/11492083/details
*Other names and brands may be claimed as the property of others.
11
Ceph* storage node --Good
CPU Intel(R) Xeon(R) CPU E5-2650v3
Memory 64 GB
NIC 10GbE
Disks 1x 1.6TB P3700 + 12 x 4TB HDDs (1:12 ratio)
P3700 as Journal and caching
Caching software Intel(R) CAS 3.0, option: Intel(R) RSTe/MD4.3
Ceph* Storage node --Better
CPU Intel(R) Xeon(R) CPU E5-2690
Memory 128 GB
NIC Duel 10GbE
Disks 1x Intel(R) DC P3700(800G) + 4x Intel(R) DC S3510 1.6TB
Ceph* Storage node --Best
CPU Intel(R) Xeon(R) CPU E5-2699v3
Memory >= 128 GB
NIC 2x 40GbE, 4x dual 10GbE
Disks 4 to 6 x Intel® DC P3700 2TB
12. 12
All Flash (PCIe* SSD + SATA SSD) Ceph Configuration
2x10Gb NIC
Test Environment
CEPH1
MON
OSD1 OSD8…
FIO FIO
CLIENT 1
1x10Gb NIC
.
FIO FIO
CLIENT 2
FIO FIO
CLIENT 3
FIO FIO
CLIENT 4
FIO FIO
CLIENT 5
CEPH2
OSD1 OSD8…
CEPH3
OSD1 OSD8…
CEPH4
OSD1 OSD8…
CEPH5
OSD1 OSD8…
“Better IOPS Ceph Configuration”¹
More Information: https://intelassetlibrary.tagcmd.com/#assets/gallery/11492083/details
*Other names and brands may be claimed as the property of others.
¹ For configuration see Slide 5
5x Client Node
• Intel® Xeon® processor E5-
2699 v3 @ 2.3GHz, 64GB
mem
• 10Gb NIC
5x Storage Node
• Intel® Xeon® processor E5-
2699 v3 @ 2.3 GHz
• 128GB Memory
• 1x 1T HDD for OS
• 1x Intel® DC P3700 800G
SSD for Journal (U.2)
• 4x 1.6TB Intel® SSD DC
S3510 as data drive
• 2 OSD instances one each
Intel® DC S3510 SSD
13. 13
Ceph* on All Flash Array
--Tuning and optimization efforts
• Up to 16x performance improvement for 4K random read, peak throughput
1.08M IOPS
• Up to 7.6x performance improvement for 4K random write, 140K IOPS
4K Random Read Tunings 4K Random Write Tunings
Default Single OSD Single OSD
Tuning-1 2 OSD instances per SSD 2 OSD instances per SSD
Tuning-2 Tuning1 + debug=0 Tuning2+Debug 0
Tuning-3 Tuning2 + jemalloc
tuning3+ op_tracker off, tuning fd
cache
Tuning-4 Tuning3 + read_ahead_size=16 Tuning4+jemalloc
Tuning-5 Tuning4 + osd_op_thread=32 Tuning4 + Rocksdb to store omap
Tuning-6 Tuning5 + rbd_op_thread=4 N/A
-
2.00
4.00
6.00
8.00
10.00
12.00
14.00
16.00
18.00
Default Tuning-1 Tuning-2 Tuning-3 Tuning-4 Tuning-5 Tuning-6
Normalized
4K random Read/Write Tunings
4K Random Read 4K random write
Performance numbers are Intel Internal estimates
For more complete information about performance and benchmark results, visit www.intel.com/benchmarks
Intel and Intel logos are trademarks of Intel Corporation or its subsidiaries in the U.S. and/or other countries
14. 14
Ceph* on All Flash Array
--Tuning and optimization efforts
1.08M IOPS for 4K random read, 144K IOPS for 4K random write with tunings
and optimizations
1
2
4
8
16
32
64
128
0 200000 400000 600000 800000 1000000 1200000 1400000
LATENCY(MS)
IOPS
RANDOM READ PERFORMANCE
RBD # SCALE TEST
4K Rand.R 8K Rand.R 16K Rand.R 64K Rand.R
63K 64k Random Read
IOPS @ 40ms
300K 16k Random
Read IOPS @ 10 ms
1.08M 4k Random
Read IOPS @ 3.4ms500K 8k Random
Read IOPS @ 8.8ms
0
2
4
6
8
10
0 20000 40000 60000 80000 100000 120000 140000 160000
LATENCY(MS)
IOPS
RANDOM WRITE PERFORMANCE
RBD # SCALE TEST
4K Rand.W 8K Rand.w 16K Rand.W 64K Rand.W
23K 64k Random Write
IOPS @ 2.6ms
88K 16kRandom Write
IOPS @ 2.7ms
132K 8k Random Write
IOPS @ 4.1ms
144K 4kRandom
Write IOPS @ 4.3ms
Excellent random read performance and Acceptable random write performance
Performance numbers are Intel Internal estimates
For more complete information about performance and benchmark results, visit www.intel.com/benchmarks
Intel and Intel logos are trademarks of Intel Corporation or its subsidiaries in the U.S. and/or other countries
15. Ceph* on All Flash Array
--Ceph*: SSD Cluster vs. HDD Cluster
• Both journal on PCI Express*/NVM Express* SSD
• 4K random write, need ~ 58x HDD Cluster (~ 2320 HDDs) to
get same performance
• 4K random read, need ~ 175x HDD Cluster (~ 7024 HDDs)
to get the same performance
ALL SSD Ceph* helps provide excellent TCO (both Capx and Opex), not only performance but
also space, Power, Fail rate, etc.
Client Node
• 5 nodes with Intel® Xeon® processor E5-2699 v3 @ 2.30GHz,
64GB memory
• OS : Ubuntu* Trusty
Storage Node
• 5 nodes with Intel® Xeon® processor E5-2699 v3 @ 2.30GHz,
128GB memory
• Ceph* Version : 9.2.0, OS : Ubuntu* Trusty
• 1 x Intel(R) DC P3700 SSDs for Journal per node
Cluster difference:
SSD cluster : 4 x Intel(R) DC S3510 1.6TB for OSD per node
HDD cluster : 10 x SATA 7200RPM HDDs as OSD per node
15
0
50
100
150
200
4K Rand.W 4K Rand.R
Normalized
Performance Comparison
HDD SSD
~ 58.2
~175.6
Performance numbers are Intel Internal estimates
For more complete information about performance and benchmark results, visit www.intel.com/benchmarks
Intel and Intel logos are trademarks of Intel Corporation or its subsidiaries in the U.S. and/or other countries
16. 16
All-NVMe Ceph Cluster for MySQL Hosting
Supermicro 1028U-TN10RT+
NVMe2
NVMe3 NVMe4
CephOSD1
CephOSD2
CephOSD3
CephOSD4
CephOSD16
5-Node all-NVMe Ceph Cluster
Dual-Xeon E5 2699v4@2.2GHz, 44C HT, 128GB DDR4
RHEL7.2, 3.10-327, Ceph v10.2.0, bluestore async
ClusterNW2x10GbE
10x Client Systems
Dual-socket Xeon E5 2699v3@2.3GHz
36 Cores HT, 128GB DDR4
Public NW 2x 10GbE
Docker3
Sysbench Client
Docker4
Sysbench Client
DB containers
16 vCPUs, 32GB mem,
200GB RBD volume,
100GB MySQL dataset,
InnoDB buf cache 25GB (25%)
CephRBDClient
Docker1 (krbd)
MySQL DB Server
NVMe1
Client containers
16 vCPUs, 32GB RAM
FIO 2.8, Sysbench 0.5Docker2 (krbd)
MySQL DB Server
20x 1.6TB P3700 SSDs
80 OSDs
2x Replication
19TB Effective Capacity
Tests at cluster fill-level 82%
17. FIO 4K Random Read/Write Performance and Latency
Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Any difference in system hardware or
software design or configuration may affect actual performance. See configuration slides in backup for details on software configuration and test benchmark
parameters.
0
1
2
3
4
5
6
7
8
9
10
11
12
0 200000 400000 600000 800000 1000000 1200000 1400000 1600000 1800000
AverageLatency(ms)
IOPS
IODepth Scaling - Latency vs IOPS - Read, Write, and 70/30 4K Random Mix
5 nodes, 80 OSDs, Xeon E5 2699v4 Dual Socket / 128GB Ram / 2x10GbE
Ceph 10.2.1 w/ BlueStore. 6x RBD FIO Clients
100% Rand Read 100% Rand Write 70% Rand Read
~1.4M 4k Random Read IOPS
@~1 ms avg
~220k 4k Random Write IOPS
@~5 ms avg
~560k 70/30% (OLTP)
Random IOPS @~3 ms avg ~1.6M 4k Random Read IOPS
@~2.2 ms avg
First Ceph cluster to break ~1.4 Million 4K random IOPS, ~1ms response time in 5U
17
18. Sysbench MySQL OLTP Performance
(100% SELECT, 16KB Avg IO Size, QD=2-8 Avg)
InnoDB buf pool = 25%, SQL dataset = 100GB
0
5
10
15
20
25
30
35
0 200000 400000 600000 800000 1000000 1200000 1400000
AvgLatency(ms)
Aggregate Queries Per Second (QPS)
Sysbench Thread Scaling - Latency vs QPS – 100% read (Point SELECTs)
5 nodes, 80 OSDs, Xeon E5 2699v4 Dual Socket / 128GB Ram / 2x10GbE
Ceph 10.1.2 w/ BlueStore. 20 Docker-rbd Sysbench Clients (16vCPUs, 32GB)
100% Random Read
~55000 QPS with 1 client
1 million QPS with 20 clients @ ~11 ms avg
2 Sysbench threads/client
~1.3 million QPS with 20 Sysbench clients,
8 Sysbench threads/client
Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Any difference in system hardware or
software design or configuration may affect actual performance. See configuration slides in backup for details on software configuration and test benchmark
parameters.
18
Database page size = 16KB
19. Sysbench MySQL OLTP Performance
(100% UPDATE, 70/30% SELECT/UPDATE)
0
50
100
150
200
250
300
350
400
450
500
0 100000 200000 300000 400000 500000 600000
AvgLatency(ms)
Aggregate Queries Per Second (QPS)
Sysbench Thread Scaling - Latency vs QPS – 100% Write (Index UPDATEs), 70/30% OLTP
5 nodes, 80 OSDs, Xeon E5 2699v4 Dual Socket / 128GB Ram / 2x10GbE
Ceph 10.2.1 w/ BlueStore. 20 Docker-rbd Sysbench Clients (16vCPU, 32GB)
100% Random Write 70/30% Read/Write
~400k 70/30% OLTP QPS@~50 ms avg
~25000 QPS w/ 1 Sysbench client (4-8 threads)
~100k Write QPS@~200 ms avg (Aggregate, 20 clients)
~5500 QPS w/ 1 Sysbench client (2-4 threads)
InnoDB buf pool = 25%, SQL dataset = 100GB
Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Any difference in system hardware or
software design or configuration may affect actual performance. See configuration slides in backup for details on software configuration and test benchmark
parameters.
19
Database page size = 16KB
25. Moore’s Law Continues to Disrupt the Computing Industry
U.2 SSD
First Intel® SSD for
Commercial Usage
2017 >10TB
1,000,000x
the capacity while
shrinking the
form factor
1992 12MB
Source: Intel projections on SSD capacity
2019201820172014
>6TB >30TB 1xxTB>10TB
25
26. 3D XPoint™
Latency: ~100X
Size of Data: ~1,000X
NAND
Latency: ~100,000X
Size of Data: ~1,000X
Latency: 1X
Size of Data: 1X
SRAM
Latency: ~10 MillionX
Size of Data: ~10,000 X
HDD
Latency: ~10X
Size of Data: ~100X
DRAM
3DXpoint™TECHNOLOGY
STORAGE
Technology claims are based on comparisons of latency, density and write cycling metrics amongst memory technologies recorded on published specifications of
in-market memory products against internal Intel specifications.
Performance numbers are Intel Internal estimates
For more complete information about performance and benchmark results, visit www.intel.com/benchmarks
Intel and Intel logos are trademarks of Intel Corporation or its subsidiaries in the U.S. and/or other countries
27. 27
Tests document performance of components on a particular test, in specific systems. Differences in hardware, software, or configuration will affect actual performance.
Consult other sources of information to evaluate performance as you consider your purchase. For more complete information about performance and benchmark
results, visit http://www.intel.com/performance. Server Configuration: 2x Intel® Xeon® E5 2690 v3 NVM Express* (NVMe) NAND based SSD: Intel P3700 800 GB, 3D
Xpoint based SSD: Optane NVMe OS: Red Hat* 7.1
Intel® Optane™ storage (prototype) vs Intel® SSD DC
P3700 Series at QD=1
28. 28
5X lower 99th%
Higher is better
*Benchmarked on early prototype samples, 2S Haswell/Broadwell Xeon platform single server.
Data produced without any tuning. We expect performance to improve with tuning.
PCIe SSD Intel Optane
Lower is better
PCIe SSD Intel Optane
2X the
Throughput
Performance numbers are Intel Internal estimates
For more complete information about performance and benchmark results, visit www.intel.com/benchmarks
Intel and Intel logos are trademarks of Intel Corporation or its subsidiaries in the U.S. and/or other countries
29. Storage Hierarchy Tomorrow
Hot
3D XPoint™ DIMMs
NVM Express* (NVMe)
3D XPoint™ SSDs
Warm
NVMe 3D NAND SSDs
Cold
NVMe 3D NAND SSDs
SATA or SAS HDDs
~6GB/s per channel
~250 nanosecond latency
PCI Express* (PCIe*) 3.0 x4 link, ~3.2 GB/s
<10 microsecond latency
SATA* 6Gbps
Minutes offline
DRAM: 10GB/s per channel, ~100 nanosecond latency
PCIe 3.0 x4, x2 link
<100 microsecond latency
Comparisons between memory technologies based on in-market product specifications and internal Intel specifications.
Server side and/or AFA
Business Processing
High Performance/In-Memory Analytics
Scientific
Cloud Web/Search/Graph
Big Data Analytics (Hadoop*)
Object Store / Active-Archive
Swift, lambert, HDFS, Ceph*
Low cost archive
29
30. 30
3D XPoint™ & 3D NAND Enable
High performance & cost effective solutions
Enterprise class, highly reliable, feature rich, and
cost effective AFA solution:
‒ NVMe as Journal, 3D NAND TLC SSD as data store
Enhance value through special software
optimization on filestore and bluestore backend
Ceph Node
S3510
1.6TB
S3510
1.6TB
S3510
1.6TB
S3510
1.6TB
P3700
U.2 800GB
Ceph Node
P4500
4TB
P4500
4TB
P4500
4TB
P4500
4TB
P3700 & 3D Xpoint™ SSDs
3D NAND
P4500
4TB
3D XPoint™
(performance) (capacity)
31. 31
3D Xpoint™ opportunities: Bluestore backend
• Three usages for PMEM device
• Backend of bluestore: raw PMEM block device or
file of dax-enabled FS
• Backend of rocksdb: raw PMEM block device or
file of dax-enabled FS
• Backend of rocksdb’s WAL: raw PMEM block
device or file of DAX-enabled FS
• Two methods for accessing PMEM devices
• libpmemblk
• mmap + libpmemlib
• https://github.com/Ceph*/Ceph*/pull/8761
BlueStore
Rocksdb
BlueFS
PMEMDevice PMEMDevice PMEMDevice
Metadata
Libpmemlib
Libpmemblk
DAX Enabled File System
mmap
Load/store
mmap
Load/store
File
File
File
API
API
Data
32. Summary
• Strong demands and trends to all-flash array Ceph* solutions
• IOPS/SLA based applications such as SQL Database can be backend
with all flash Ceph
• NVM technologies such as 3D Xpoint and 3D NANDs enable new
performance capabilities and expedite all flash adoptions
• Bluestore shows significant performance increase compared with
filestore, but still needs to be improved
• Let’s work together to make Ceph* more efficient with all-flash array!
32
37. Storage interface
Use FIORBD as storage interface
Tool
• Use “dd” to prepare data for R/W tests
• Use fio (ioengine=libaio, direct=1) to generate 4 IO patterns: sequential write/read, random write/read
• Access Span: 60GB
Run rules
• Drop osds page caches ( “1” > /proc/sys/vm/drop_caches)
• 100 secs for warm up, 600 secs for data collection
• Run 4KB/64KB tests under different # of rbds (1 to 120)
Testing Methodology