Scylla Summit 2018: Rebuilding the Ceph Distributed Storage Solution with Seastar
•Download as PPTX, PDF•
3 likes•455 views
RedHat built a distributed object storage solution named Ceph which first debuted ten years ago. Now we are seeing rapid developments in the industry and we want to take advantage of them. In this talk, we will briefly introduce Ceph, revisit the problems we are seeing when profiling its I/O performance with flash device, and explain why we want to embrace the future by switching to Seastar. We’ll share our experiences with the audience of how and when we are porting our software to this framework.
Report
Share
Report
Share
Recommended
Unikraft: Fast, Specialized Unikernels the Easy Way
P99 CONF
Unikernels are famous for providing excellent performance in terms of boot times, throughput and memory consumption, to name a few metrics. However, they are infamous for making it hard and extremely time consuming to extract such performance, and for needing significant engineering effort in order to port applications to them. We introduce Unikraft, a novel micro-library OS that (1) fully modularizes OS primitives so that it is easy to customize the unikernel and include only relevant components and (2) exposes a set of composable, performance-oriented APIs in order to make it easy for developers to obtain high performance.
Our evaluation using off-the-shelf applications such as nginx, SQLite, and Redis shows that running them on Unikraft results in a 1.7x-2.7x performance improvement compared to Linux guests. In addition, Unikraft images for these apps are around 1MB, require less than 10MB of RAM to run, and boot in around 1ms on top of the VMM time (total boot time 3ms-40ms). Unikraft is a Linux Foundation open source project and can be found at www.unikraft.org.
Scylla Summit 2018: Rebuilding the Ceph Distributed Storage Solution with Sea...
This document summarizes plans to rebuild the Ceph distributed storage system using Seastar, a framework for high-performance event-driven applications. Ceph is an open-source distributed storage platform that provides object, block, and file storage at scale. It uses a thread pool model that has limitations around lock contention and context switching. Seastar uses an asynchronous message passing model without locks that could improve Ceph's performance. The plan is to backfill Ceph components starting with critical I/O paths to prioritize basic functionality, then add supporting features later to fully rebuild Ceph on Seastar.
Whoops! I Rewrote It in Rust
Three engineers, at various points, each take their own approach adding Rust to a C codebase, each being more and more ambitious. I initially just wanted to replace the server’s networking and event loop with an equally fast Rust implementation. We’d reuse many core components that were in C and just call into them from Rust. Surely it wouldn’t be that much code…
Pelikan is Twitter’s open source and modular framework for in-memory caching, allowing us to replace Memcached and Redis forks with a single codebase and achieve better performance. At Twitter, we operate hundreds of cache clusters storing hundreds of terabytes of small objects in memory. In-memory caching is critical, and demands performance, reliability, and efficiency.
In this talk, I’ll share my adventures in working on Pelikan and how rewriting it in Rust can be more than just a meme.
Object Compaction in Cloud for High Yield
In file systems, large sequential writes are more beneficial than small random writes, and hence many storage systems implement a log structured file system. In the same way, the cloud favors large objects more than small objects. Cloud providers place throttling limits on PUTs and GETs, and so it takes significantly longer time to upload a bunch of small objects than a large object of the aggregate size. Moreover, there are per-PUT calls associated with uploading smaller objects.
In Netflix, a lot of media assets and their relevant metadata is generated and pushed to cloud.
We would like to propose a strategy to compact these small objects into larger blobs before uploading them to Cloud. We will discuss how to select relevant smaller objects, and manage the indexing of these objects within the blob along with modification in reads, overwrites and deletes.
Finally, we would showcase the potential impact of such a strategy on Netflix assets in terms of cost and performance.
Rust Is Safe. But Is It Fast?
Rust promises developers the execution speed of non-managed languages like C++, with the safety guarantees of managed languages like Go. Its fast rise in popularity shows this promise has been largely upheld.
However, the situation is a bit muddier for the newer asynchronous extensions. This talk will explore some of the pitfalls that users may face while developing asynchronous Rust applications that have direct consequences in their ability to hit that sweet low p99. We will see how the Glommio asynchronous executor tries to deal with some of those problems, and what the future holds.
G1: To Infinity and Beyond
G1 has been around for quite some time now and since JDK 9 it is the default garbage collector in OpenJDK. The community working on G1 is big and the contributions over the last few years have made a significant impact on the overall performance. This talk will focus on some of these features and how they have improved G1 in various ways, including smaller memory footprint and shorter P99 pause times. We will also take a brief look at what features we have lined up for the future.
Vanquishing Latency Outliers in the Lightbits LightOS Software Defined Storag...
Lightbits LightOS software defined storage system aims to provide low latency storage access over existing Ethernet networks by pooling NVMe flash across servers. It addresses latency challenges from flash media, the network, and management operations through an intelligent flash management backend, NVMe/TCP frontend, write buffering, and isolation of management processes. Performance measurements show average read latencies of 150-170 microseconds and average write latencies of 90-125 microseconds even under high load, demonstrating its ability to keep latency low at scale.
[POSS 2019] OVirt and Ceph: Perfect Combination.?
This document discusses using the open-source virtualization platform Ovirt together with the distributed storage platform Ceph. It evaluates three options for combining Ovirt and Ceph that provide high availability and scalability without single points of failure. The recommended option uses CephFS to provide POSIX-compliant storage for virtual machines hosted on an Ovirt cluster, along with GlusterFS for the Ovirt hosted engine and Ceph monitor nodes. This allows up to seven hosts without any single points of failure.
Recommended
Unikraft: Fast, Specialized Unikernels the Easy Way
P99 CONF
Unikernels are famous for providing excellent performance in terms of boot times, throughput and memory consumption, to name a few metrics. However, they are infamous for making it hard and extremely time consuming to extract such performance, and for needing significant engineering effort in order to port applications to them. We introduce Unikraft, a novel micro-library OS that (1) fully modularizes OS primitives so that it is easy to customize the unikernel and include only relevant components and (2) exposes a set of composable, performance-oriented APIs in order to make it easy for developers to obtain high performance.
Our evaluation using off-the-shelf applications such as nginx, SQLite, and Redis shows that running them on Unikraft results in a 1.7x-2.7x performance improvement compared to Linux guests. In addition, Unikraft images for these apps are around 1MB, require less than 10MB of RAM to run, and boot in around 1ms on top of the VMM time (total boot time 3ms-40ms). Unikraft is a Linux Foundation open source project and can be found at www.unikraft.org.
Scylla Summit 2018: Rebuilding the Ceph Distributed Storage Solution with Sea...
This document summarizes plans to rebuild the Ceph distributed storage system using Seastar, a framework for high-performance event-driven applications. Ceph is an open-source distributed storage platform that provides object, block, and file storage at scale. It uses a thread pool model that has limitations around lock contention and context switching. Seastar uses an asynchronous message passing model without locks that could improve Ceph's performance. The plan is to backfill Ceph components starting with critical I/O paths to prioritize basic functionality, then add supporting features later to fully rebuild Ceph on Seastar.
Whoops! I Rewrote It in Rust
Three engineers, at various points, each take their own approach adding Rust to a C codebase, each being more and more ambitious. I initially just wanted to replace the server’s networking and event loop with an equally fast Rust implementation. We’d reuse many core components that were in C and just call into them from Rust. Surely it wouldn’t be that much code…
Pelikan is Twitter’s open source and modular framework for in-memory caching, allowing us to replace Memcached and Redis forks with a single codebase and achieve better performance. At Twitter, we operate hundreds of cache clusters storing hundreds of terabytes of small objects in memory. In-memory caching is critical, and demands performance, reliability, and efficiency.
In this talk, I’ll share my adventures in working on Pelikan and how rewriting it in Rust can be more than just a meme.
Object Compaction in Cloud for High Yield
In file systems, large sequential writes are more beneficial than small random writes, and hence many storage systems implement a log structured file system. In the same way, the cloud favors large objects more than small objects. Cloud providers place throttling limits on PUTs and GETs, and so it takes significantly longer time to upload a bunch of small objects than a large object of the aggregate size. Moreover, there are per-PUT calls associated with uploading smaller objects.
In Netflix, a lot of media assets and their relevant metadata is generated and pushed to cloud.
We would like to propose a strategy to compact these small objects into larger blobs before uploading them to Cloud. We will discuss how to select relevant smaller objects, and manage the indexing of these objects within the blob along with modification in reads, overwrites and deletes.
Finally, we would showcase the potential impact of such a strategy on Netflix assets in terms of cost and performance.
Rust Is Safe. But Is It Fast?
Rust promises developers the execution speed of non-managed languages like C++, with the safety guarantees of managed languages like Go. Its fast rise in popularity shows this promise has been largely upheld.
However, the situation is a bit muddier for the newer asynchronous extensions. This talk will explore some of the pitfalls that users may face while developing asynchronous Rust applications that have direct consequences in their ability to hit that sweet low p99. We will see how the Glommio asynchronous executor tries to deal with some of those problems, and what the future holds.
G1: To Infinity and Beyond
G1 has been around for quite some time now and since JDK 9 it is the default garbage collector in OpenJDK. The community working on G1 is big and the contributions over the last few years have made a significant impact on the overall performance. This talk will focus on some of these features and how they have improved G1 in various ways, including smaller memory footprint and shorter P99 pause times. We will also take a brief look at what features we have lined up for the future.
Vanquishing Latency Outliers in the Lightbits LightOS Software Defined Storag...
Lightbits LightOS software defined storage system aims to provide low latency storage access over existing Ethernet networks by pooling NVMe flash across servers. It addresses latency challenges from flash media, the network, and management operations through an intelligent flash management backend, NVMe/TCP frontend, write buffering, and isolation of management processes. Performance measurements show average read latencies of 150-170 microseconds and average write latencies of 90-125 microseconds even under high load, demonstrating its ability to keep latency low at scale.
[POSS 2019] OVirt and Ceph: Perfect Combination.?
This document discusses using the open-source virtualization platform Ovirt together with the distributed storage platform Ceph. It evaluates three options for combining Ovirt and Ceph that provide high availability and scalability without single points of failure. The recommended option uses CephFS to provide POSIX-compliant storage for virtual machines hosted on an Ovirt cluster, along with GlusterFS for the Ovirt hosted engine and Ceph monitor nodes. This allows up to seven hosts without any single points of failure.
P99CONF — What We Need to Unlearn About Persistent Storage
System software engineers have long been taught that disks are slow and sequential I/O is key to performance. With SSD drives I/O really got much faster but not simpler. In this brave new world of rocket-speed throughputs an engineer has to distinguish sustained workload from bursts, (still) take care about I/O buffer sizes, account for disks’ internal parallelism and study mixed I/O characteristics in advance. In this talk we will share some key performance measurements of the modern hardware we’re taking at ScyllaDB and our opinion about the implications for the database and system software design.
Understanding Apache Kafka P99 Latency at Scale
Apache Kafka is a highly popular distributed system used by many organizations to connect systems, build microservices, create data mesh, etc. However, as a distributed system, understanding its performance could be a challenge, so many moving parts exist.
In this talk, we are going to review the key moving parts (producers, consumers, replication, network, etc), a strategy to measure and interpret the performance results for consumers and producers and a general guideline for deciding about performance in Apache Kafka.
An attendee will take home after the talk a proven method to measure, evaluate and optimise the performance of an Apache Kafka based infrastructure. A key skill for low throughput users, but especially for the biggest scale deployments.
Extreme HTTP Performance Tuning: 1.2M API req/s on a 4 vCPU EC2 Instance
In this talk I will walk you through the performance tuning steps that I took to serve 1.2M JSON requests per second from a 4 vCPU c5 instance, using a simple API server written in C.
At the start of the journey the server is capable of a very respectable 224k req/s with the default configuration. Along the way I made extensive use of tools like FlameGraph and bpftrace to measure, analyze, and optimize the entire stack, from the application framework, to the network driver, all the way down to the kernel.
I began this wild adventure without any prior low-level performance optimization experience; but once I started going down the performance tuning rabbit-hole, there was no turning back. Fueled by my curiosity, willingness to learn, and relentless persistence, I was able to boost performance by over 400% and reduce p99 latency by almost 80%.
Build Low-Latency Applications in Rust on ScyllaDB
Join us for a developer workshop where we’ll go hands-on to explore the affinities between Rust, the Tokio framework, and ScyllaDB. You’ll go live with our sample Rust application, built on our new, high performance native Rust client driver.
Avoiding Data Hotspots at Scale
There are two key choices when scaling a NoSQL data store:
choosing between a hash or a range based sharding and choosing the right sharding key. Any choice is a trade-off between scalability of read, append, and update workloads.
In this talk I will present the standard scaling techniques,
some non-universal sharding tricks, less obvious reasons for
hotspots, as well as techniques to avoid them.
Rust's Journey to Async/await
Rust's Journey to Async/Await
- Rust was initially built for synchronous I/O but the community wanted asynchronous capabilities for building network services.
- This led to the development of futures in Rust to support asynchronous programming. However, futures had design issues that motivated futures 2.0.
- Async/await was developed to make asynchronous code look synchronous while teaching the borrow checker about asynchronous constraints. This involved debate around syntax.
- After years of work, async/await landed in Rust 1.38, completing Rust's journey to fully supporting asynchronous programming. The implementation involved resolving language-specific challenges.
Crimson: Ceph for the Age of NVMe and Persistent Memory
Ceph is a mature open source software-defined storage solution that was created over a decade ago.
During that time new faster storage technologies have emerged including NVMe and Persistent memory.
The crimson project aim is to create a better Ceph OSD that is more well suited to those faster devices. The crimson OSD is built on the Seastar C++ framework and can leverage these devices by minimizing latency, cpu overhead, and cross-core communication. This talk will discuss the project design, our current status, and our future plans.
Seastore: Next Generation Backing Store for Ceph
Ceph is an open source distributed file system addressing file, block, and object storage use cases. Next generation storage devices require a change in strategy, so the community has been developing crimson-osd, an eventual replacement for ceph-osd intended to minimize cpu overhead and improve throughput and latency. Seastore is a new backing store for crimson-osd targeted at emerging storage technologies including persistent memory and ZNS devices.
RISC-V on Edge: Porting EVE and Alpine Linux to RISC-V
This document discusses porting the Alpine Linux distribution and EVE operating system to RISC-V processors. It provides an overview of RISC-V and describes how Alpine Linux and EVE-OS are well-suited for edge computing workloads due to their small footprint and use of containers. The document then outlines the steps taken to port Alpine Linux and its packages to RISC-V, including getting key packages like libunwind to build and working with the Alpine community. It concludes by encouraging more development of software for the emerging RISC-V server platforms.
Rust Primer
Rust has something unique to offer that languages in that space have never had before, and that is a degree of safety that languages like C and C++ have never had. Rust promises to deliver equivalent or better performance and greater productivity with guaranteed memory safety and data race freedom while allowing complete and direct control over memory.
This video will cover:
What is Rust?
Benefits of Rust
Rust Ecosystem
Popular Applications in Rust
Cassandra To Infinity And Beyond
How Teads scale with Apache Cassandra.
Internet scale means tons of data, read heavy workload, massive data ingestion and low latency.
The French AdTech company Teads uses Cassandra massively, a reliable and performant Open Source database.
Spawning Cassandra nodes in AWS is a piece of cake with Terraform and Chef.
IPv4aaS tutorial and hands-on
This document discusses various techniques for IPv6 transition and coexistence with IPv4, including:
- Dual-stack which allows simultaneous support of both IPv4 and IPv6.
- Tunnels which encapsulate IPv6 packets in IPv4 packets to provide IPv6 connectivity through IPv4 networks.
- Translation techniques like NAT64 which allow communication between IPv4-only and IPv6-only nodes.
RADOS improvements and roadmap - Greg Farnum, Josh Durgin, Kefu Chai
Cephalocon APAC 2018
March 22-23, 2018 - Beijing, China
Greg Farnum，Red Hat RADOS Core Developer
Josh Durgin, Red Hat RADOS Lead
Kefu Chai, Red Hat Senior Software Engineer
Scaling Apache Pulsar to 10 Petabytes/Day
This document discusses optimizing an Apache Pulsar cluster to handle 10 PB of data per day for a financial customer. Initial estimates showed the cluster would need over 1000 VMs using HDD storage. Various optimizations were implemented, including eliminating the journal, using direct I/O, compression, and C++ client optimizations. This reduced the estimated number of needed VMs to 200 using L-SSD storage per VM. The optimized cluster can now meet the customer's requirements of processing 10 PB of data per day with 3 hours of retention and zone failure protection.
EVCache & Moneta (GoSF)
Presentation given at the GoSF meetup on July 20, 2016. It was also recorded on BigMarker here: https://www.bigmarker.com/remote-meetup-go/GoSF-EVCache-Peripheral-I-O-Building-Origin-Cache-for-Images
Application Caching: The Hidden Microservice (SAConf)
Presentation given at the O'Reilly Software Architecture Conference in San Francisco on November 15, 2016.
Where Did All These Cycles Go?
This document discusses various places where CPU cycles are wasted in production systems based on insights from continuous profiling of large compute clusters. Some examples of wasted cycles include misconfiguration of the underlying OS, suboptimal choices of dependencies, and excessive serialization/deserialization. Specific issues highlighted include gettimeofday calls on older AWS instances, kubelet directory walking, exception throwing in xgboost, and high CPU usage from zlib. Continuous fleet-wide profiling is becoming an important tool for identifying such performance issues.
OSCON 2017: To contain or not to contain
I propose a method of choosing the optimal platform based on workload characterization. Learn the differences between containers and virtualization, where they complement each other, and how the two should (and shouldn't) be used together.
Practical SystemTAP basics: Perl memory profiling
This document provides an overview of the SystemTap dynamic tracing tool. It can trace both kernel and userspace functions and variables. Scripts are written in a powerful scripting language that can inline C code and are compiled into kernel modules. Practical examples are provided, including a Perl memory profiler that traces memory allocations to different Perl packages. Safety features like timeouts help prevent scripts from overloading systems.
Rust, Wright's Law, and the Future of Low-Latency Systems
The coming decade will see two important changes with profound ramifications for low-latency systems: the rise of Rust-based systems, and the ceding of Moore's Law to Wright's Law. In this talk, we will discuss these two trends, and (especially) their confluence -- and explain why we believe that the future of low-latency systems will include Rust programs in some surprising places.
Ceph
This document summarizes a presentation about Ceph, an open-source distributed storage system. It discusses Ceph's introduction and components, benchmarks Ceph's block and object storage performance on Intel architecture, and describes optimizations like cache tiering and erasure coding. It also outlines Intel's product portfolio in supporting Ceph through optimized CPUs, flash storage, networking, server boards, software libraries, and contributions to the open source Ceph community.
Ceph: Open Source Storage Software Optimizations on Intel® Architecture for C...
Après la petite intro sur le stockage distribué et la description de Ceph, Jian Zhang réalise dans cette présentation quelques benchmarks intéressants : tests séquentiels, tests random et surtout comparaison des résultats avant et après optimisations. Les paramètres de configuration touchés et optimisations (Large page numbers, Omap data sur un disque séparé, ...) apportent au minimum 2x de perf en plus.
More Related Content
What's hot
P99CONF — What We Need to Unlearn About Persistent Storage
System software engineers have long been taught that disks are slow and sequential I/O is key to performance. With SSD drives I/O really got much faster but not simpler. In this brave new world of rocket-speed throughputs an engineer has to distinguish sustained workload from bursts, (still) take care about I/O buffer sizes, account for disks’ internal parallelism and study mixed I/O characteristics in advance. In this talk we will share some key performance measurements of the modern hardware we’re taking at ScyllaDB and our opinion about the implications for the database and system software design.
Understanding Apache Kafka P99 Latency at Scale
Apache Kafka is a highly popular distributed system used by many organizations to connect systems, build microservices, create data mesh, etc. However, as a distributed system, understanding its performance could be a challenge, so many moving parts exist.
In this talk, we are going to review the key moving parts (producers, consumers, replication, network, etc), a strategy to measure and interpret the performance results for consumers and producers and a general guideline for deciding about performance in Apache Kafka.
An attendee will take home after the talk a proven method to measure, evaluate and optimise the performance of an Apache Kafka based infrastructure. A key skill for low throughput users, but especially for the biggest scale deployments.
Extreme HTTP Performance Tuning: 1.2M API req/s on a 4 vCPU EC2 Instance
In this talk I will walk you through the performance tuning steps that I took to serve 1.2M JSON requests per second from a 4 vCPU c5 instance, using a simple API server written in C.
At the start of the journey the server is capable of a very respectable 224k req/s with the default configuration. Along the way I made extensive use of tools like FlameGraph and bpftrace to measure, analyze, and optimize the entire stack, from the application framework, to the network driver, all the way down to the kernel.
I began this wild adventure without any prior low-level performance optimization experience; but once I started going down the performance tuning rabbit-hole, there was no turning back. Fueled by my curiosity, willingness to learn, and relentless persistence, I was able to boost performance by over 400% and reduce p99 latency by almost 80%.
Build Low-Latency Applications in Rust on ScyllaDB
Join us for a developer workshop where we’ll go hands-on to explore the affinities between Rust, the Tokio framework, and ScyllaDB. You’ll go live with our sample Rust application, built on our new, high performance native Rust client driver.
Avoiding Data Hotspots at Scale
There are two key choices when scaling a NoSQL data store:
choosing between a hash or a range based sharding and choosing the right sharding key. Any choice is a trade-off between scalability of read, append, and update workloads.
In this talk I will present the standard scaling techniques,
some non-universal sharding tricks, less obvious reasons for
hotspots, as well as techniques to avoid them.
Rust's Journey to Async/await
Rust's Journey to Async/Await
- Rust was initially built for synchronous I/O but the community wanted asynchronous capabilities for building network services.
- This led to the development of futures in Rust to support asynchronous programming. However, futures had design issues that motivated futures 2.0.
- Async/await was developed to make asynchronous code look synchronous while teaching the borrow checker about asynchronous constraints. This involved debate around syntax.
- After years of work, async/await landed in Rust 1.38, completing Rust's journey to fully supporting asynchronous programming. The implementation involved resolving language-specific challenges.
Crimson: Ceph for the Age of NVMe and Persistent Memory
Ceph is a mature open source software-defined storage solution that was created over a decade ago.
During that time new faster storage technologies have emerged including NVMe and Persistent memory.
The crimson project aim is to create a better Ceph OSD that is more well suited to those faster devices. The crimson OSD is built on the Seastar C++ framework and can leverage these devices by minimizing latency, cpu overhead, and cross-core communication. This talk will discuss the project design, our current status, and our future plans.
Seastore: Next Generation Backing Store for Ceph
Ceph is an open source distributed file system addressing file, block, and object storage use cases. Next generation storage devices require a change in strategy, so the community has been developing crimson-osd, an eventual replacement for ceph-osd intended to minimize cpu overhead and improve throughput and latency. Seastore is a new backing store for crimson-osd targeted at emerging storage technologies including persistent memory and ZNS devices.
RISC-V on Edge: Porting EVE and Alpine Linux to RISC-V
This document discusses porting the Alpine Linux distribution and EVE operating system to RISC-V processors. It provides an overview of RISC-V and describes how Alpine Linux and EVE-OS are well-suited for edge computing workloads due to their small footprint and use of containers. The document then outlines the steps taken to port Alpine Linux and its packages to RISC-V, including getting key packages like libunwind to build and working with the Alpine community. It concludes by encouraging more development of software for the emerging RISC-V server platforms.
Rust Primer
Rust has something unique to offer that languages in that space have never had before, and that is a degree of safety that languages like C and C++ have never had. Rust promises to deliver equivalent or better performance and greater productivity with guaranteed memory safety and data race freedom while allowing complete and direct control over memory.
This video will cover:
What is Rust?
Benefits of Rust
Rust Ecosystem
Popular Applications in Rust
Cassandra To Infinity And Beyond
How Teads scale with Apache Cassandra.
Internet scale means tons of data, read heavy workload, massive data ingestion and low latency.
The French AdTech company Teads uses Cassandra massively, a reliable and performant Open Source database.
Spawning Cassandra nodes in AWS is a piece of cake with Terraform and Chef.
IPv4aaS tutorial and hands-on
This document discusses various techniques for IPv6 transition and coexistence with IPv4, including:
- Dual-stack which allows simultaneous support of both IPv4 and IPv6.
- Tunnels which encapsulate IPv6 packets in IPv4 packets to provide IPv6 connectivity through IPv4 networks.
- Translation techniques like NAT64 which allow communication between IPv4-only and IPv6-only nodes.
RADOS improvements and roadmap - Greg Farnum, Josh Durgin, Kefu Chai
Cephalocon APAC 2018
March 22-23, 2018 - Beijing, China
Greg Farnum，Red Hat RADOS Core Developer
Josh Durgin, Red Hat RADOS Lead
Kefu Chai, Red Hat Senior Software Engineer
Scaling Apache Pulsar to 10 Petabytes/Day
This document discusses optimizing an Apache Pulsar cluster to handle 10 PB of data per day for a financial customer. Initial estimates showed the cluster would need over 1000 VMs using HDD storage. Various optimizations were implemented, including eliminating the journal, using direct I/O, compression, and C++ client optimizations. This reduced the estimated number of needed VMs to 200 using L-SSD storage per VM. The optimized cluster can now meet the customer's requirements of processing 10 PB of data per day with 3 hours of retention and zone failure protection.
EVCache & Moneta (GoSF)
Presentation given at the GoSF meetup on July 20, 2016. It was also recorded on BigMarker here: https://www.bigmarker.com/remote-meetup-go/GoSF-EVCache-Peripheral-I-O-Building-Origin-Cache-for-Images
Application Caching: The Hidden Microservice (SAConf)
Presentation given at the O'Reilly Software Architecture Conference in San Francisco on November 15, 2016.
Where Did All These Cycles Go?
This document discusses various places where CPU cycles are wasted in production systems based on insights from continuous profiling of large compute clusters. Some examples of wasted cycles include misconfiguration of the underlying OS, suboptimal choices of dependencies, and excessive serialization/deserialization. Specific issues highlighted include gettimeofday calls on older AWS instances, kubelet directory walking, exception throwing in xgboost, and high CPU usage from zlib. Continuous fleet-wide profiling is becoming an important tool for identifying such performance issues.
OSCON 2017: To contain or not to contain
I propose a method of choosing the optimal platform based on workload characterization. Learn the differences between containers and virtualization, where they complement each other, and how the two should (and shouldn't) be used together.
Practical SystemTAP basics: Perl memory profiling
This document provides an overview of the SystemTap dynamic tracing tool. It can trace both kernel and userspace functions and variables. Scripts are written in a powerful scripting language that can inline C code and are compiled into kernel modules. Practical examples are provided, including a Perl memory profiler that traces memory allocations to different Perl packages. Safety features like timeouts help prevent scripts from overloading systems.
Rust, Wright's Law, and the Future of Low-Latency Systems
The coming decade will see two important changes with profound ramifications for low-latency systems: the rise of Rust-based systems, and the ceding of Moore's Law to Wright's Law. In this talk, we will discuss these two trends, and (especially) their confluence -- and explain why we believe that the future of low-latency systems will include Rust programs in some surprising places.
What's hot (20)
P99CONF — What We Need to Unlearn About Persistent Storage
P99CONF — What We Need to Unlearn About Persistent Storage
Extreme HTTP Performance Tuning: 1.2M API req/s on a 4 vCPU EC2 Instance
Extreme HTTP Performance Tuning: 1.2M API req/s on a 4 vCPU EC2 Instance
Build Low-Latency Applications in Rust on ScyllaDB
Build Low-Latency Applications in Rust on ScyllaDB
Crimson: Ceph for the Age of NVMe and Persistent Memory
Crimson: Ceph for the Age of NVMe and Persistent Memory
RISC-V on Edge: Porting EVE and Alpine Linux to RISC-V
RISC-V on Edge: Porting EVE and Alpine Linux to RISC-V
RADOS improvements and roadmap - Greg Farnum, Josh Durgin, Kefu Chai
RADOS improvements and roadmap - Greg Farnum, Josh Durgin, Kefu Chai
Application Caching: The Hidden Microservice (SAConf)
Application Caching: The Hidden Microservice (SAConf)
Rust, Wright's Law, and the Future of Low-Latency Systems
Rust, Wright's Law, and the Future of Low-Latency Systems
Similar to Scylla Summit 2018: Rebuilding the Ceph Distributed Storage Solution with Seastar
Ceph
This document summarizes a presentation about Ceph, an open-source distributed storage system. It discusses Ceph's introduction and components, benchmarks Ceph's block and object storage performance on Intel architecture, and describes optimizations like cache tiering and erasure coding. It also outlines Intel's product portfolio in supporting Ceph through optimized CPUs, flash storage, networking, server boards, software libraries, and contributions to the open source Ceph community.
Ceph: Open Source Storage Software Optimizations on Intel® Architecture for C...
Après la petite intro sur le stockage distribué et la description de Ceph, Jian Zhang réalise dans cette présentation quelques benchmarks intéressants : tests séquentiels, tests random et surtout comparaison des résultats avant et après optimisations. Les paramètres de configuration touchés et optimisations (Large page numbers, Omap data sur un disque séparé, ...) apportent au minimum 2x de perf en plus.
Ceph Community Talk on High-Performance Solid Sate Ceph
The document summarizes a presentation given by representatives from various companies on optimizing Ceph for high-performance solid state drives. It discusses testing a real workload on a Ceph cluster with 50 SSD nodes that achieved over 280,000 read and write IOPS. Areas for further optimization were identified, such as reducing latency spikes and improving single-threaded performance. Various companies then described their contributions to Ceph performance, such as Intel providing hardware for testing and Samsung discussing SSD interface improvements.
Ceph in the GRNET cloud stack
Presentation on how GRNET uses Ceph as a storage backend on its Cloud Computing services. Technical specs, lessons learned, future plans.
Presentation held at the 1st GEANT SIG-CISS Meeting in Amsterdam, 2017-09-25.
GRNET - Greek Research and Technology network is the state-owned Greek NREN.
Reference Architecture: Architecting Ceph Storage Solutions
This document discusses reference architectures for Ceph storage solutions. It provides guidance on key design considerations for Ceph clusters, including workload profiling, storage access methods, capacity planning, fault tolerance, and data protection schemes. Example hardware configurations are also presented for different performance and cost optimization targets.
Ceph Day SF 2015 - Keynote
Ceph has been in development for over a decade since its beginnings as a research project at UC Santa Cruz in the 2000s. It was incubated at DreamHost and later spun out to form Inktank to build Ceph into an enterprise-grade open source storage platform. Key developments included the RADOS distributed object store, librados client library, RBD block device, and S3-compatible radosgw object store. Ceph now has a large community and is used in many production deployments, with continued work to improve performance, add new features like erasure coding, and expand its capabilities for big data and the enterprise.
Sanger OpenStack presentation March 2017
A description of the Sanger Institute's journey with OpenStack to date, covering RHOSP, Ceph, S3, user applications, and future plans. Given at the Sanger Institute's OpenStack Day.
Ceph in 2023 and Beyond.pdf
Ceph’s recent developments, community news, and best practices by Dan van der Ster at the HEPix Autumn 2023 Workshop
TUT18972: Unleash the power of Ceph across the Data Center
From SUSECon 2015: Smooth integration of emerging Software Defined Storage technologies into traditional Data Center using Fiber Channel and iSCSI as key values for success.
London Ceph Day Keynote: Building Tomorrow's Ceph
This document provides an overview of the history and development of Ceph, an open-source distributed storage system. It discusses Sage Weil's initial research that led to Ceph's creation, the incubation of Ceph at DreamHost, the launch of Inktank to support Ceph's development and adoption, and the current state of Ceph including its growing community and usage in production deployments. It also outlines Weil's vision for Ceph's future, including improving governance, adding new technologies like tiering and erasure coding, and expanding its role in areas like big data and the enterprise storage market.
In-memory Data Management Trends & Techniques
- Hardware trends like increasing cores/CPU and RAM sizes enable in-memory data management techniques. Commodity servers can now support terabytes of memory.
- Different levels of data storage have vastly different access times, from registers (<1ns) to disk (4-7ms). Caching data in faster levels of storage improves performance.
- Techniques to exploit data locality, cache hierarchies, tiered storage, parallelism and in-situ processing can help overcome hardware limitations and achieve fast, real-time processing. Emerging in-memory databases use these techniques to enable new types of operational analytics.
Openstack Summit HK - Ceph defacto - eNovance
Sébastien Han presented on Ceph and its integration with OpenStack. Ceph is an open source distributed storage system that is well-suited for OpenStack deployments due to its self-managing capabilities and ability to scale storage resources easily. The integration between Ceph and OpenStack has improved significantly in recent OpenStack releases like Havana, with features like Cinder backup to Ceph and the ability to boot Nova instances using RBD images. Further integration work is planned for upcoming releases to fully leverage Ceph's capabilities.
Replication Distilled: Hazelcast Deep Dive @ In-Memory Computing Summit San F...
This document summarizes Hazelcast's replication protocol. It uses primary copy replication where operations are sent to primary replicas for strong consistency when reachable. It partitions data to scale primaries, updates replicas asynchronously, and uses anti-entropy to sync backups. During network partitions, consistency can be lost and divergence must be resolved upon reconnection using merge policies. The protocol is classified as PA/EC under the PACELC principle, favoring availability over consistency, though failure detectors and CP data structures allow it to behave more consistently in some cases.
Disaggregating Ceph using NVMeoF
This document discusses disaggregating Ceph storage using NVMe over Fabrics (NVMeoF). It motivates using NVMeoF by showing the performance limitations of directly attaching multiple NVMe drives to individual compute nodes. It then proposes a design to leverage the full resources of a cluster by distributing NVMe drives across dedicated storage nodes and connecting them to compute nodes over a high performance fabric using NVMeoF and RDMA. Some initial Ceph performance measurements using this model show improved IOPS and latency compared to the direct attached approach. Future work could explore using SPDK and Linux kernel improvements to further optimize performance.
Your 1st Ceph cluster
This document provides an overview and planning guidelines for a first Ceph cluster. It discusses Ceph's object, block, and file storage capabilities and how it integrates with OpenStack. Hardware sizing examples are given for a 1 petabyte storage cluster with 500 VMs requiring 100 IOPS each. Specific lessons learned are also outlined, such as realistic IOPS expectations from HDD and SSD backends, recommended CPU and RAM per OSD, and best practices around networking and deployment.
Ceph Day Santa Clara: Keynote: Building Tomorrow's Ceph
Sage Weil, Creator of Ceph, Founder & CTO, Inktank
Keynote presentation at Ceph Day Santa Clara.
Ceph Day NYC: Building Tomorrow's Ceph
This talk was given by Sage Weil at Ceph Day NYC. In it, he covers the history of the Ceph project and discusses what the future has in store.
Scaling Ceph at CERN - Ceph Day Frankfurt
This document summarizes Dan van der Ster's experience scaling Ceph at CERN. CERN uses Ceph as the backend storage for OpenStack volumes and images, with plans to also use it for physics data archival and analysis. The 3PB Ceph cluster consists of 47 disk servers and 1,128 OSDs. Some lessons learned include managing latency, handling many objects, tuning CRUSH, trusting clients, and avoiding human errors when managing such a large cluster.
Introduction into Ceph storage for OpenStack
Ceph began as a research project in 2005 to create a scalable object storage system. It was incubated at DreamHost from 2007-2012 and spun out as an independent company called Inktank in 2012. Key developments included the RADOS distributed storage cluster, erasure coding, and the Ceph filesystem. The project has grown a large community and is used in many production deployments, focusing on areas like tiering, erasure coding, replication, and integrating with the Linux kernel. Future plans include improving CephFS, expanding the ecosystem through different storage backends, strengthening governance, and targeting new use cases in big data and the enterprise.
Erasure Code at Scale - Thomas William Byrne
Cephalocon APAC 2018
March 22-23, 2018 - Beijing, China
Thomas William Byrne, Science and Technology Facilities Council Linux System Admin
Similar to Scylla Summit 2018: Rebuilding the Ceph Distributed Storage Solution with Seastar (20)
Ceph: Open Source Storage Software Optimizations on Intel® Architecture for C...
Ceph: Open Source Storage Software Optimizations on Intel® Architecture for C...
Ceph Community Talk on High-Performance Solid Sate Ceph
Ceph Community Talk on High-Performance Solid Sate Ceph
Reference Architecture: Architecting Ceph Storage Solutions
Reference Architecture: Architecting Ceph Storage Solutions
TUT18972: Unleash the power of Ceph across the Data Center
TUT18972: Unleash the power of Ceph across the Data Center
Replication Distilled: Hazelcast Deep Dive @ In-Memory Computing Summit San F...
Replication Distilled: Hazelcast Deep Dive @ In-Memory Computing Summit San F...
Ceph Day Santa Clara: Keynote: Building Tomorrow's Ceph
Ceph Day Santa Clara: Keynote: Building Tomorrow's Ceph
More from ScyllaDB
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks creates AI-boosted business software that helps employees work more efficiently and effectively. Managing data across multiple RDBMS and NoSQL databases was already a challenge at their current scale. To prepare for 10X growth, they knew it was time to rethink their database strategy. Learn how they architected a solution that would simplify scaling while keeping costs under control.
Optimizing NoSQL Performance Through Observability
ScyllaDB has the potential to deliver impressive performance and scalability. The better you understand how it works, the more you can squeeze out of it. But before you squeeze, make sure you know what to monitor!
Watch our experienced Postgres developer work through monitoring and performance strategies that help him understand what mistakes he’s made moving to NoSQL. And learn with him as our database performance expert offers friendly guidance on how to use monitoring and performance tuning to get his sample Rust application on the right track.
This webinar focuses on using monitoring and performance tuning to discover and correct mistakes that commonly occur when developers move from SQL to NoSQL. For example:
- Common issues getting up and running with the monitoring stack
- Using the CQL optimizations dashboard
- Common issues causing high latency in a node
- Common issues causing replica imbalance
- What a healthy system looks like in terms of memory
- Key metrics to keep an eye on
This isn’t “Death-by-Powerpoint.” We’ll walk through problems encountered while migrating a real application from Postgres to ScyllaDB – and try to fix them live as well.
Event-Driven Architecture Masterclass: Challenges in Stream Processing
Discuss the core tradeoffs and considerations involved in order-free and ordered stream processing. Brian Taylor walks through the pros and cons of three different approaches: no data dependency, deferred inter-event data dependency, and streaming inter-event data dependency.
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
We start by setting up a common ground introducing why relational databases fall short, addressing common EDA characteristics such as the need for real-time response times and schemaless approaches to address recurring changes to adapt and on-board new use cases. Next, interact with a sample Rust-based application: a social network app demonstrating an integration of both ScyllaDB and Redpanda.
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Discover how to avoid common pitfalls when shifting to an event-driven architecture (EDA) in order to boost system recovery and scalability. We cover Kafka Schema Registry, in-broker transformations, event sourcing, and more.
Developer Data Modeling Mistakes: From Postgres to NoSQL
See where an RDBMS-pro’s intuition leads him astray – and learn practical tips for the data modeling transition
ScyllaDB has the potential to deliver impressive performance and scalability. The better you understand how it works, the more you can squeeze out of it. However, developers new to high-performance NoSQL intuitively shoot themselves in the foot with respect to things like table design, query design, indexing, and partitioning.
Watch where our experienced Postgres developer intuitively falls into traps that hurt performance and scalability. And learn with him as our database performance expert offers friendly guidance on navigating all the unexpected behaviors that tend to trip up RDBMS experts.
This webinar focuses on common data modeling and querying mistakes that occur when developers move from SQL to NoSQL. For example:
- Understanding query first design principles
- Planning for schema evolution
- Steering clear of common pitfalls and anti-patterns
- Assessing data access patterns
This isn’t “Death-by-Powerpoint.” We’ll walk through problems encountered while migrating a real application from Postgres to ScyllaDB – and try to fix them live as well.
What Developers Need to Unlearn for High Performance NoSQL
See where an RDBMS-pro’s intuition leads him astray – and learn practical tips for the transition
ScyllaDB has the potential to deliver impressive performance and scalability. The better you understand how it works, the more you can squeeze out of it. However, developers new to high-performance NoSQL intuitively shoot themselves in the foot with respect to things like table design, query design, indexing, and partitioning.
Watch where our experienced Postgres developer intuitively falls into traps that hurt performance and scalability. And learn with him as our database performance expert offers friendly guidance on navigating all the unexpected behaviors that tend to trip up RDBMS experts.
Our first webinar of this series will cover common mistakes with practices such as:
- Translating the data model to NoSQL
- Optimizing table design
- Optimizing query performance
- Planning for partitioning
This isn’t “Death-by-Powerpoint.” We’ll walk through problems encountered while migrating a real application from Postgres to ScyllaDB – and try to fix them live as well.
Low Latency at Extreme Scale: Proven Practices & Pitfalls
Expert tips on how to maximize your database performance at scale
Untangle the complexity of achieving database performance at scale. Join this webinar to discover commonly overlooked ways to get predictable low latency, even at extreme scale. Our Solution Architects will walk you through the strategies and pitfalls learned by working on thousands of real-world distributed database projects, many reaching 1M OPS with single-digit MS latencies.
In addition to offering clear recommendations, we’ll also explain the process behind how we arrived at them – so you can benefit from the lessons learned by other teams.
We’ll cover how to:
- Design and deploy a large-scale distributed database cluster
- Optimize your clients’ interactions with it
- Expand the cluster horizontally and globally
- Ensure it survives whatever disasters the world throws at it
Dissecting Real-World Database Performance Dilemmas
Tackling your own database performance challenges is serious business. For a change of pace, let’s have some fun learning from other teams’ performance predicaments.
Join us for an interactive session where we dissect four specific database performance challenges faced by teams considering or using ScyllaDB. For each dilemma, we'll:
- Examine the context and technical requirements
- Talk about potential solutions and cover the pros and cons of each
- Disclose what approach the team took, and how it worked out
About the speaker:
Felipe is an IT specialist with years of experience on distributed systems and open-source technologies. He is one of the co-authors of "Database Performance at Scale", an Open Access, freely available publication for individuals interested on improving database performance. At ScyllaDB, he works as a Solution Architect.
Beyond Linear Scaling: A New Path for Performance with ScyllaDB
Linear scaling (sometimes near linear scaling) is often mentioned in several benchmarks, articles and product comparisons as proof that a given technology and algorithmic optimizations perform better than another. But is that really what performance is all about, and should you even care?
This webinar discusses performance beyond linear scalability, including what typically matters more when running high throughput and low latency workloads at scale. We'll cover how ScyllaDB offers unparalleled performance and share our insights on:
- The hidden aspects of linear scaling
- When linear scaling matters most and when it’s simply irrelevant
- Often overlooked considerations for optimizing and measuring distributed systems performance
Watch now to learn from our experience (and lessons learned) in building the fastest NoSQL database in the world.
Dissecting Real-World Database Performance Dilemmas
Navigating Complex Database Performance Hurdles
Tackling your own database performance challenges is serious business. For a change of pace, let’s have some fun learning from other teams’ performance predicaments.
Join us for an interactive session where we dissect 4 specific database performance challenges faced by teams considering or using ScyllaDB. For each dilemma:
- The presenters will describe the context and technical requirements
- Together, we’ll talk about potential solutions and cover the pros and cons of each
- Finally, we’ll disclose what approach the team took, and how it worked out
Throughout the event, we’ll have opportunities to win ScyllaDB swag and prizes! Come prepared to engage in lively discussions and gain valuable insight into database performance strategies.
Database Performance at Scale Masterclass: Workload Characteristics by Felipe...
Felipe Cardeneti Mendes, Solutions Architect at ScyllaDB
Navigating workload-specific performance challenges and tradeoffs.
Felipe Mendes covers how to navigate the top performance challenges and tradeoffs that you’re likely to face with your project’s specific workload characteristics and technical/business requirements.
Database Performance at Scale Masterclass: Database Internals by Pavel Emelya...
Pavel Emelyanov, Principal Engineer at ScyllaDB
Botond Denes, C++ Developer at ScyllaDB
What performance-minded engineers need to know.
Hear from Pavel Emelyanov and Botond Dénes on the impact of database internals – specifically, what to look for if you need latency and/or throughput improvements.
Database Performance at Scale Masterclass: Driver Strategies by Piotr Sarna
Piotr Sarna, Software Engineer at Turso
Understanding and tapping your driver’s performance potential.
Piotr Sarna discusses how to get the most out of a driver, particularly from the performance perspective, and select a driver that’s a good fit for your needs.
Replacing Your Cache with ScyllaDB
This document discusses replacing external caching solutions with using the internal caching capabilities of ScyllaDB. It provides examples of companies that improved performance, reduced costs and complexity by moving from Redis or Elasticsearch with an external cache to using ScyllaDB's embedded cache instead. The document also outlines some of the advantages of ScyllaDB's cache like improved latency, coherency with the database and observability compared to external caching layers.
Powering Real-Time Apps with ScyllaDB_ Low Latency & Linear Scalability
Discover how your team can achieve low latency at the extreme scale that your data-intensive applications require. We’ll walk you through an example of how ScyllaDB scales linearly to achieve 1M and then 2M OPS – with <1ms P99 latency. We’ll cover how this works on a sample realtime app (an ML feature store), share best practices for performance, and talk about the most important tradeoffs you’ll need to negotiate.
Join us to learn:
- Why and how to ensure your database takes full advantage of your cloud infrastructure
- What architectural considerations matter most for high throughput and low latency
- Key factors to consider when selecting a high-performance database
7 Reasons Not to Put an External Cache in Front of Your Database.pptx
This document discusses the pros and cons of placing an external cache in front of a database. It introduces Tomasz Grabiec and Tzach Livyatan from ScyllaDB and describes ScyllaDB's optimized internal caching design. External caches can increase latency and costs while ignoring the database's context and workload knowledge. ScyllaDB embeds its cache to minimize overhead and ensure data and query awareness. The document shares customer examples that improved performance and reduced costs by moving from cached databases to ScyllaDB.
Getting the most out of ScyllaDB
Expert tips on how to maximize your database potential
If you’re considering or getting started with ScyllaDB, you’re probably intrigued by its potential to achieve high throughput and predictable low latency at a reasonable cost. So how do you ensure that you’re maximizing that potential for your team’s specific workloads and use case?
This webinar offers practical advice for navigating the various decision points you’ll face as you assess whether ScyllaDB is a good fit for your team and later roll it out into production. We’ll cover the most critical considerations, tradeoffs, and recommendations related to:
- Infrastructure selection
- ScyllaDB configuration
- Client-side setup
- Data modeling
NoSQL Database Migration Masterclass - Session 2: The Anatomy of a Migration
In this talk, Felipe Mendes, Solutions Architect at ScyllaDB, shares how 4 companies managed their migration. He covers:
Disney+ – No migration needed!
Discord – Shadow cluster
OpenWeb – TTL expiration, cover Load and Stream
MyHeritage – Counters
ShareChat – Bonus: A bit of everything
NoSQL Database Migration Masterclass - Session 3: Migration Logistics
In this talk, Lubos discusses tools and methods for a successful migration. He covers:
Methods
Data (re)modeling
APIs
Spark Migrator
DS bulk
Tuning
Testing/monitoring
More from ScyllaDB (20)
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Optimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through Observability
Event-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQL
What Developers Need to Unlearn for High Performance NoSQL
What Developers Need to Unlearn for High Performance NoSQL
Low Latency at Extreme Scale: Proven Practices & Pitfalls
Low Latency at Extreme Scale: Proven Practices & Pitfalls
Dissecting Real-World Database Performance Dilemmas
Dissecting Real-World Database Performance Dilemmas
Beyond Linear Scaling: A New Path for Performance with ScyllaDB
Beyond Linear Scaling: A New Path for Performance with ScyllaDB
Dissecting Real-World Database Performance Dilemmas
Dissecting Real-World Database Performance Dilemmas
Database Performance at Scale Masterclass: Workload Characteristics by Felipe...
Database Performance at Scale Masterclass: Workload Characteristics by Felipe...
Database Performance at Scale Masterclass: Database Internals by Pavel Emelya...
Database Performance at Scale Masterclass: Database Internals by Pavel Emelya...
Database Performance at Scale Masterclass: Driver Strategies by Piotr Sarna
Database Performance at Scale Masterclass: Driver Strategies by Piotr Sarna
Powering Real-Time Apps with ScyllaDB_ Low Latency & Linear Scalability
Powering Real-Time Apps with ScyllaDB_ Low Latency & Linear Scalability
7 Reasons Not to Put an External Cache in Front of Your Database.pptx
7 Reasons Not to Put an External Cache in Front of Your Database.pptx
NoSQL Database Migration Masterclass - Session 2: The Anatomy of a Migration
NoSQL Database Migration Masterclass - Session 2: The Anatomy of a Migration
NoSQL Database Migration Masterclass - Session 3: Migration Logistics
NoSQL Database Migration Masterclass - Session 3: Migration Logistics
Recently uploaded
zOS Mainframe JES2-JES3 JCL-JECL Differences
Document which explains the difference between Jes2 and Jes3
Hand Rolled Applicative User Validation Code Kata
Could you use a simple piece of Scala validation code (granted, a very simplistic one too!) that you can rewrite, now and again, to refresh your basic understanding of Applicative operators <*>, <*, *>?
The goal is not to write perfect code showcasing validation, but rather, to provide a small, rough-and ready exercise to reinforce your muscle-memory.
Despite its grandiose-sounding title, this deck consists of just three slides showing the Scala 3 code to be rewritten whenever the details of the operators begin to fade away.
The code is my rough and ready translation of a Haskell user-validation program found in a book called Finding Success (and Failure) in Haskell - Fall in love with applicative functors.
What next after learning python programming basics
What next after learning python programming basics
Fundamentals of Programming and Language Processors
Fundamentals of Programming and Language Processors
UI5con 2024 - Bring Your Own Design System
How do you combine the OpenUI5/SAPUI5 programming model with a design system that makes its controls available as Web Components? Since OpenUI5/SAPUI5 1.120, the framework supports the integration of any Web Components. This makes it possible, for example, to natively embed own Web Components of your design system which are created with Stencil. The integration embeds the Web Components in a way that they can be used naturally in XMLViews, like with standard UI5 controls, and can be bound with data binding. Learn how you can also make use of the Web Components base class in OpenUI5/SAPUI5 to also integrate your Web Components and get inspired by the solution to generate a custom UI5 library providing the Web Components control wrappers for the native ones.
Odoo ERP Vs. Traditional ERP Systems – A Comparative Analysis
When it is all about ERP solutions, companies typically meet their needs with common ERP solutions like SAP, Oracle, and Microsoft Dynamics. These big players have demonstrated that ERP systems can be either simple or highly comprehensive. This remains true today, but there are new factors to consider, including a promising new contender in the market that’s Odoo. This blog compares Odoo ERP with traditional ERP systems and explains why many companies now see Odoo ERP as the best choice.
What are ERP Systems?
An ERP, or Enterprise Resource Planning, system provides your company with valuable information to help you make better decisions and boost your ROI. You should choose an ERP system based on your company’s specific needs. For instance, if you run a manufacturing or retail business, you will need an ERP system that efficiently manages inventory. A consulting firm, on the other hand, would benefit from an ERP system that enhances daily operations. Similarly, eCommerce stores would select an ERP system tailored to their needs.
Because different businesses have different requirements, ERP system functionalities can vary. Among the various ERP systems available, Odoo ERP is considered one of the best in the ERp market with more than 12 million global users today.
Odoo is an open-source ERP system initially designed for small to medium-sized businesses but now suitable for a wide range of companies. Odoo offers a scalable and configurable point-of-sale management solution and allows you to create customised modules for specific industries. Odoo is gaining more popularity because it is built in a way that allows easy customisation, has a user-friendly interface, and is affordable. Here, you will cover the main differences and get to know why Odoo is gaining attention despite the many other ERP systems available in the market.
Using Xen Hypervisor for Functional Safety
An update on making Xen hypervisor functionally safe and enhancing its usage in automotive and industrial use cases
316895207-SAP-Oil-and-Gas-Downstream-Training.pptx
316895207-SAP-Oil-and-Gas-Downstream-Training.pptx
Measures in SQL (SIGMOD 2024, Santiago, Chile)
SQL has attained widespread adoption, but Business Intelligence tools still use their own higher level languages based upon a multidimensional paradigm. Composable calculations are what is missing from SQL, and we propose a new kind of column, called a measure, that attaches a calculation to a table. Like regular tables, tables with measures are composable and closed when used in queries.
SQL-with-measures has the power, conciseness and reusability of multidimensional languages but retains SQL semantics. Measure invocations can be expanded in place to simple, clear SQL.
To define the evaluation semantics for measures, we introduce context-sensitive expressions (a way to evaluate multidimensional expressions that is consistent with existing SQL semantics), a concept called evaluation context, and several operations for setting and modifying the evaluation context.
A talk at SIGMOD, June 9–15, 2024, Santiago, Chile
Authors: Julian Hyde (Google) and John Fremlin (Google)
https://doi.org/10.1145/3626246.3653374
Malibou Pitch Deck For Its €3M Seed Round
French start-up Malibou raised a €3 million Seed Round to develop its payroll and human resources
management platform for VSEs and SMEs. The financing round was led by investors Breega, Y Combinator, and FCVC.
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
Charla impartida en el evento de "KuberTENes Birthday Bash Guadalajara" para celebrar el 10mo. aniversario de Kubernetes #kuberTENes #celebr8k8s #k8s
Energy consumption of Database Management - Florina Jonuzi
Presentation from Florina Jonuzi at the GSD Community Stage Meetup on June 06, 2024
socradar-q1-2024-aviation-industry-report.pdf
SOCRadar's Aviation Industry Q1 Incident Report is out now!
The aviation industry has always been a prime target for cybercriminals due to its critical infrastructure and high stakes. In the first quarter of 2024, the sector faced an alarming surge in cybersecurity threats, revealing its vulnerabilities and the relentless sophistication of cyber attackers.
SOCRadar’s Aviation Industry, Quarterly Incident Report, provides an in-depth analysis of these threats, detected and examined through our extensive monitoring of hacker forums, Telegram channels, and dark web platforms.
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
Wondering how X-Sign gained popularity in a quick time span? This eSign functionality of XfilesPro DocuPrime has many advancements to offer for Salesforce users. Explore them now!
How to write a program in any programming language
How to write a program in any programming language
Modelling Up - DDDEurope 2024 - Amsterdam
What to do when you have a perfect model for your software but you are constrained by an imperfect business model?
This talk explores the challenges of bringing modelling rigour to the business and strategy levels, and talking to your non-technical counterparts in the process.
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
Regarding mobile operating systems, two major players dominate our thoughts: Android and iPhone. With Android leading the market, software development companies are focused on delivering apps compatible with this OS. Ensuring an app's functionality across various Android devices, OS versions, and hardware specifications is critical, making Android app testing essential.
Using Query Store in Azure PostgreSQL to Understand Query Performance
Microsoft has added an excellent new extension in PostgreSQL on their Azure Platform. This session, presented at Posette 2024, covers what Query Store is and the types of information you can get out of it.
Artificia Intellicence and XPath Extension Functions
The purpose of this presentation is to provide an overview of how you can use AI from XSLT, XQuery, Schematron, or XML Refactoring operations, the potential benefits of using AI, and some of the challenges we face.
Recently uploaded (20)
What next after learning python programming basics
What next after learning python programming basics
Fundamentals of Programming and Language Processors
Fundamentals of Programming and Language Processors
Odoo ERP Vs. Traditional ERP Systems – A Comparative Analysis
Odoo ERP Vs. Traditional ERP Systems – A Comparative Analysis
316895207-SAP-Oil-and-Gas-Downstream-Training.pptx
316895207-SAP-Oil-and-Gas-Downstream-Training.pptx
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
Energy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina Jonuzi
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
How to write a program in any programming language
How to write a program in any programming language
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
Using Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query Performance
Artificia Intellicence and XPath Extension Functions
Artificia Intellicence and XPath Extension Functions
Scylla Summit 2018: Rebuilding the Ceph Distributed Storage Solution with Seastar
- 1. Rebuild the Ceph Distributed Storage with Seastar Kefu Chai Software Engineer, Red Hat
- 2. Kefu Chai He is a software engineer who has been focusing on storage systems for over a decade. Currently, he is working on Ceph, an open source, distributed storage solution.
- 3. What is Ceph anyway?
- 4. About Ceph ▪ A platform • Object, block and filesystem ▪ Scalable • from 10 to 10,000 nodes (OSDs) ▪ No single point failure ▪ Free and Open Source (LGPL)
- 6. CRUSH
- 8. good not not good enough
- 9. The latency analysis for write path courtesy of Myongwon Oh, et al., ALCA: Locality-Aware Lock Contention Avoidance for NVMe-Based Scale-out Storage System
- 10. Blank Slide for full size diagram Storage Backend Network Messenger Thread Pool serving requests
- 11. A closer look at the thread pool
- 12. Random R/W performance The performance of random I/O is throttled by CPU
- 14. Latencies in I/O path ▪ TCP/IP stack in kernel space ▪ Coarse grained locks ▪ Internal fragmentation ▪ Memory copy for direct-I/O
- 15. Seastar to the Rescue
- 16. Non-seastar ▪ Thread pool ▪ Lock ▪ Share state ▪ TCP/IP in kernel ▪ Callbacks ▪ Multiple-threaded ▪ Fiber ▪ Lockless ▪ Message passing ▪ TCP/IP in userspace (0-copy) ▪ future/promise ▪ single-threaded Seastar
- 18. A message from the alien
- 19. A message from the alien ▪ std::future versus seastar::future // in rocksdb Status SequentialFile::read(size_t n, char* buf) { return alien::submit_to(io_shard, [n,buf,this] { return do_dma_read(n, buf); }).wait(); } ▪ Dedicated cores for the alien territory
- 20. Backfill with priority ▪ Critical path first • Basic I/O paths ▪ Supporting features later • recovery • Metric reporting • …
- 21. Thank You Any Questions ? Please stay in touch kchai@redhat.com @tchaik0v