The document discusses the Ceph distributed storage system. It provides an overview of what Ceph is, how it works, and its key features. Ceph is an open-source unified storage system that provides object storage, block storage, and a file system. It uses a distributed system of monitors, storage nodes, and metadata servers to store and retrieve data reliably across multiple machines.
CRUSH is the powerful, highly configurable algorithm Red Hat Ceph Storage uses to determine how data is stored across the many servers in a cluster. A healthy Red Hat Ceph Storage deployment depends on a properly configured CRUSH map. In this session, we will review the Red Hat Ceph Storage architecture and explain the purpose of CRUSH. Using example CRUSH maps, we will show you what works and what does not, and explain why.
Presented at Red Hat Summit 2016-06-29.
Ceph, Now and Later: Our Plan for Open Unified Cloud StorageSage Weil
Ceph is a highly scalable open source distributed storage system that provides object, block, and file interfaces on a single platform. Although Ceph RBD block storage has dominated OpenStack deployments for several years, maturing object (S3, Swift, and librados) interfaces and stable CephFS (file) interfaces now make Ceph the only fully open source unified storage platform.
This talk will cover Ceph's architectural vision and project mission and how our approach differs from alternative approaches to storage in the OpenStack ecosystem. In particular, we will look at how our open development model dovetails well with OpenStack, how major contributors are advancing Ceph capabilities and performance at a rapid pace to adapt to new hardware types and deployment models, and what major features we are priotizing for the next few years to meet the needs of expanding cloud workloads.
Distributed Storage and Compute With Ceph's librados (Vault 2015)Sage Weil
The Ceph distributed storage system sports object, block, and file interfaces to a single storage cluster. These interface are built on a distributed object storage and compute platform called RADOS, which exports a conceptually simple yet powerful interface for storing and processing large amounts of data and is well-suited for backing web-scale applications and data analytics. In features a rich object model, efficient key/value storage, atomic transactions (including efficient compare-and-swap semantics), object cloning and other primitives for supporting snapshots, simple inter-client communication and coordination (ala Zookeeper), and the ability to extend the object interface using arbitrary code executed on the storage node. This talk will focus on librados API, how it is used, the security model, and some examples of RADOS classes implementing interesting functionality.
BlueStore, A New Storage Backend for Ceph, One Year InSage Weil
BlueStore is a new storage backend for Ceph OSDs that consumes block devices directly, bypassing the local XFS file system that is currently used today. It's design is motivated by everything we've learned about OSD workloads and interface requirements over the last decade, and everything that has worked well and not so well when storing objects as files in local files systems like XFS, btrfs, or ext4. BlueStore has been under development for a bit more than a year now, and has reached a state where it is becoming usable in production. This talk will cover the BlueStore design, how it has evolved over the last year, and what challenges remain before it can become the new default storage backend.
CRUSH is the powerful, highly configurable algorithm Red Hat Ceph Storage uses to determine how data is stored across the many servers in a cluster. A healthy Red Hat Ceph Storage deployment depends on a properly configured CRUSH map. In this session, we will review the Red Hat Ceph Storage architecture and explain the purpose of CRUSH. Using example CRUSH maps, we will show you what works and what does not, and explain why.
Presented at Red Hat Summit 2016-06-29.
Ceph, Now and Later: Our Plan for Open Unified Cloud StorageSage Weil
Ceph is a highly scalable open source distributed storage system that provides object, block, and file interfaces on a single platform. Although Ceph RBD block storage has dominated OpenStack deployments for several years, maturing object (S3, Swift, and librados) interfaces and stable CephFS (file) interfaces now make Ceph the only fully open source unified storage platform.
This talk will cover Ceph's architectural vision and project mission and how our approach differs from alternative approaches to storage in the OpenStack ecosystem. In particular, we will look at how our open development model dovetails well with OpenStack, how major contributors are advancing Ceph capabilities and performance at a rapid pace to adapt to new hardware types and deployment models, and what major features we are priotizing for the next few years to meet the needs of expanding cloud workloads.
Distributed Storage and Compute With Ceph's librados (Vault 2015)Sage Weil
The Ceph distributed storage system sports object, block, and file interfaces to a single storage cluster. These interface are built on a distributed object storage and compute platform called RADOS, which exports a conceptually simple yet powerful interface for storing and processing large amounts of data and is well-suited for backing web-scale applications and data analytics. In features a rich object model, efficient key/value storage, atomic transactions (including efficient compare-and-swap semantics), object cloning and other primitives for supporting snapshots, simple inter-client communication and coordination (ala Zookeeper), and the ability to extend the object interface using arbitrary code executed on the storage node. This talk will focus on librados API, how it is used, the security model, and some examples of RADOS classes implementing interesting functionality.
BlueStore, A New Storage Backend for Ceph, One Year InSage Weil
BlueStore is a new storage backend for Ceph OSDs that consumes block devices directly, bypassing the local XFS file system that is currently used today. It's design is motivated by everything we've learned about OSD workloads and interface requirements over the last decade, and everything that has worked well and not so well when storing objects as files in local files systems like XFS, btrfs, or ext4. BlueStore has been under development for a bit more than a year now, and has reached a state where it is becoming usable in production. This talk will cover the BlueStore design, how it has evolved over the last year, and what challenges remain before it can become the new default storage backend.
Ceph is an open source distributed storage system that provides scalable object, block, and file interfaces on a commodity hardware. Luminous, the latest stable release of Ceph, was just released in August. This talk will cover all that is new in Luminous (there is a lot!) and provide a sneak peak at the roadmap for Mimic, which is due out in the Spring.
Keeping OpenStack storage trendy with Ceph and containersSage Weil
The conventional approach to deploying applications on OpenStack uses virtual machines (usually KVM) backed by block devices (usually Ceph RBD). As interest increases in container-based application deployment models like Docker, it is worth looking at what alternatives exist for combining compute and storage (both shared and non-shared). Mapping RBD block devices directly to host kernels trades isolation for performance and may be appropriate for many private clouds without significant changes to the infrastructure. More importantly, moving away from a virtualization allows for non-block interfaces and a range of alternative models based on file or object.
Attendees will leave this talk with a basic understanding of the storage components and services available to both virtual machines and Linux containers, a view of a several ways they can be combined and the performance, reliability, and security trade-offs associated with those possibilities, and several proposals for how the relevant OpenStack projects (Nova, Cinder, Manila) can work together to make it easy.
The State of Ceph, Manila, and Containers in OpenStackSage Weil
OpenStack users deploying Ceph for block (Cinder) and object (S3/Swift) are unsurprisingly looking at Manila and CephFS to round out a unified storage solution. Ceph is based on a low-level object storage layer call RADOS that serves as the foundation for its object, block, and file services. Manila's file as a service in OpenStack enables a range of container-based use-cases with Docker and Kubernetes, but a variety of deployment architectures are possible.
This talk will cover the current state of CephFS support in Manila, including upstream Manila support, Manila works in progress, a progress update on CephFS itself, including new multi-tenancy support to facilitate cloud deployments, and a discussion of how this impact container deployment scenarios in an OpenStack cloud.
Making distributed storage easy: usability in Ceph Luminous and beyondSage Weil
Distributed storage is complicated, and historically Ceph hasn't spent a lot of time trying to hide that complexity, instead focusing on correctness, features, and flexibility. There has been a recent shift in focus to simplifying and streamlining the user/operator experience so that the information that is actually important is available without the noise of irrelevant details. Recent feature work has also focused on simplifying configurations that were previously possible but required tedious configuration steps to manage.
This talk will cover the key new efforts in Ceph Luminous that aim to simplify and automate cluster management, as well as the plans for upcoming releases to address longstanding Cephisms that make it "hard" (e.g., choosing PG counts).
Understanding blue store, Ceph's new storage backend - Tim Serong, SUSEOpenStack
Audience Level
Intermediate
Synopsis
Ceph – the most popular storage solution for OpenStack – stores all data as a collection of objects. This object store was originally implemented on top of a POSIX filesystem, an approach that turned out to have a number of problems, notably with performance and complexity.
BlueStore, a new storage backend for Ceph, was created to solve these issues; the Ceph Jewel release included an early prototype. The code and on-disk format were declared stable (but experimental) for Ceph Kraken, and now in the upcoming Ceph Luminous release, BlueStore will be the recommended default storage backend.
With a 2-3x performance boost, you’ll want to look at migrating your Ceph clusters to BlueStore. This talk goes into detail about what BlueStore does, the problems it solves, and what you need to do to use it.
Speaker Bio:
Tim works for SUSE, hacking on Ceph and related technologies. He has spoken often about distributed storage and high availability at conferences such as linux.conf.au. In his spare time he wrangles pigs, chickens, sheep and ducks, and was declared by one colleague “teammate most likely to survive the zombie apocalypse”.
Actually any people and employers get high available in your applications, maintain ours environment always available not is easy task. In Open Source World exist tools that maje it possible. This appresentation is a module course UTAH NETWORXS about Hight Available and Performance Course. Utah Networxs is business school in Sao Paulo Brazil Work a Linux System for more than 17 years. Maked to Fabio Pires Directory Utah Networxs and Linux Specialist focusing Clusters and HA services.
Ceph is an open source distributed storage system that provides scalable object, block, and file interfaces on a commodity hardware. Luminous, the latest stable release of Ceph, was just released in August. This talk will cover all that is new in Luminous (there is a lot!) and provide a sneak peak at the roadmap for Mimic, which is due out in the Spring.
Keeping OpenStack storage trendy with Ceph and containersSage Weil
The conventional approach to deploying applications on OpenStack uses virtual machines (usually KVM) backed by block devices (usually Ceph RBD). As interest increases in container-based application deployment models like Docker, it is worth looking at what alternatives exist for combining compute and storage (both shared and non-shared). Mapping RBD block devices directly to host kernels trades isolation for performance and may be appropriate for many private clouds without significant changes to the infrastructure. More importantly, moving away from a virtualization allows for non-block interfaces and a range of alternative models based on file or object.
Attendees will leave this talk with a basic understanding of the storage components and services available to both virtual machines and Linux containers, a view of a several ways they can be combined and the performance, reliability, and security trade-offs associated with those possibilities, and several proposals for how the relevant OpenStack projects (Nova, Cinder, Manila) can work together to make it easy.
The State of Ceph, Manila, and Containers in OpenStackSage Weil
OpenStack users deploying Ceph for block (Cinder) and object (S3/Swift) are unsurprisingly looking at Manila and CephFS to round out a unified storage solution. Ceph is based on a low-level object storage layer call RADOS that serves as the foundation for its object, block, and file services. Manila's file as a service in OpenStack enables a range of container-based use-cases with Docker and Kubernetes, but a variety of deployment architectures are possible.
This talk will cover the current state of CephFS support in Manila, including upstream Manila support, Manila works in progress, a progress update on CephFS itself, including new multi-tenancy support to facilitate cloud deployments, and a discussion of how this impact container deployment scenarios in an OpenStack cloud.
Making distributed storage easy: usability in Ceph Luminous and beyondSage Weil
Distributed storage is complicated, and historically Ceph hasn't spent a lot of time trying to hide that complexity, instead focusing on correctness, features, and flexibility. There has been a recent shift in focus to simplifying and streamlining the user/operator experience so that the information that is actually important is available without the noise of irrelevant details. Recent feature work has also focused on simplifying configurations that were previously possible but required tedious configuration steps to manage.
This talk will cover the key new efforts in Ceph Luminous that aim to simplify and automate cluster management, as well as the plans for upcoming releases to address longstanding Cephisms that make it "hard" (e.g., choosing PG counts).
Understanding blue store, Ceph's new storage backend - Tim Serong, SUSEOpenStack
Audience Level
Intermediate
Synopsis
Ceph – the most popular storage solution for OpenStack – stores all data as a collection of objects. This object store was originally implemented on top of a POSIX filesystem, an approach that turned out to have a number of problems, notably with performance and complexity.
BlueStore, a new storage backend for Ceph, was created to solve these issues; the Ceph Jewel release included an early prototype. The code and on-disk format were declared stable (but experimental) for Ceph Kraken, and now in the upcoming Ceph Luminous release, BlueStore will be the recommended default storage backend.
With a 2-3x performance boost, you’ll want to look at migrating your Ceph clusters to BlueStore. This talk goes into detail about what BlueStore does, the problems it solves, and what you need to do to use it.
Speaker Bio:
Tim works for SUSE, hacking on Ceph and related technologies. He has spoken often about distributed storage and high availability at conferences such as linux.conf.au. In his spare time he wrangles pigs, chickens, sheep and ducks, and was declared by one colleague “teammate most likely to survive the zombie apocalypse”.
Actually any people and employers get high available in your applications, maintain ours environment always available not is easy task. In Open Source World exist tools that maje it possible. This appresentation is a module course UTAH NETWORXS about Hight Available and Performance Course. Utah Networxs is business school in Sao Paulo Brazil Work a Linux System for more than 17 years. Maked to Fabio Pires Directory Utah Networxs and Linux Specialist focusing Clusters and HA services.
A Storage Orchestrator for Kubernetes
Rook turns distributed storage systems into self-managing, self-scaling, self-healing storage services. It automates the tasks of a storage administrator: deployment, bootstrapping, configuration, provisioning, scaling, upgrading, migration, disaster recovery, monitoring, and resource management.
Rook uses the power of the Kubernetes platform to deliver its services: cloud-native container management, scheduling, and orchestration.
a comprehensive good introduction to the the Big data world in AWS cloud, hadoop, Streaming, batch, Kinesis, DynamoDB, Hbase, EMR, Athena, Hive, Spark, Piq, Impala, Oozie, Data pipeline, Security , Cost, Best practices
Running OpenStack in Production - Barcamp Saigon 2016Thang Man
My talk at http://www.barcampsaigon.com (2016) about how did we architect and configure the OpenStack-based private cloud for running in production at FimPlus.vn
Presentation held at GRNET Digital Technology Symposium on November 5-6, 2018 at the Stavros Niarchos Foundation Cultural Center, Athens, Greece.
• Introduction to Ceph and its internals
• Presentation of GRNET's Ceph deployments (technical specs, operations)
• Usecases: ESA Copernicus, ~okeanos, ViMa
In this session, you'll learn how RBD works, including how it:
Uses RADOS classes to make access easier from user space and within the Linux kernel.
Implements thin provisioning.
Builds on RADOS self-managed snapshots for cloning and differential backups.
Increases performance with caching of various kinds.
Uses watch/notify RADOS primitives to handle online management operations.
Integrates with QEMU, libvirt, and OpenStack.
Ceph is an open source distributed storage system designed for scalability and reliability. Ceph's block device, RADOS block device (RBD), is widely used to store virtual machines, and is the most popular block storage used with OpenStack.
In this session, you'll learn how RBD works, including how it:
* Uses RADOS classes to make access easier from user space and within the Linux kernel.
* Implements thin provisioning.
* Builds on RADOS self-managed snapshots for cloning and differential backups.
* Increases performance with caching of various kinds.
* Uses watch/notify RADOS primitives to handle online management operations.
* Integrates with QEMU, libvirt, and OpenStack.
Ceph Day Santa Clara: The Future of CephFS + Developing with LibradosCeph Community
Sage Weil, Creator of Ceph, Founder & CTO, Inktank
CephFS is a distributed filesystem built on RADOS, offering POSIX-semantics and a true scale-out architecture. While production deployments of CephFS do exist, it still needs lots of testing and hardening before it can be used in the most challenging (and interesting) scenarios. In this session, Sage will discuss the future of CephFS, includ- ing the areas where it still needs work and ways the community can help.
RADOS is a surprisingly flexible object store. To take advantage of its rich feature set, developers can build with its programmable library, librados. Librados is avail- able in many languages, and offers access to key/value stores, object classes, cluster health and status, and other useful RADOS internals. This session will cover how to use librados, discuss situations where librados is the right choice, and share a list of lesser-known RADOS features that developers can tap into.
Presentation from 2016 Austin OpenStack Summit.
The Ceph upstream community is declaring CephFS stable for the first time in the recent Jewel release, but that declaration comes with caveats: while we have filesystem repair tools and a horizontally scalable POSIX filesystem, we have default-disabled exciting features like horizontally-scalable metadata servers and snapshots. This talk will present exactly what features you can expect to see, what's blocking the inclusion of other features, and what you as a user can expect and can contribute by deploying or testing CephFS.
Introduction to HBase. HBase is a NoSQL databases which experienced a tremendous increase in popularity during the last years. Large companies like Facebook, LinkedIn, Foursquare are using HBase. In this presentation we will address questions like: what is HBase?, and compared to relational databases?, what is the architecture?, how does HBase work?, what about the schema design?, what about the IT ressources?. Questions that should help you consider whether this solution might be suitable in your case.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™UiPathCommunity
In questo evento online gratuito, organizzato dalla Community Italiana di UiPath, potrai esplorare le nuove funzionalità di Autopilot, il tool che integra l'Intelligenza Artificiale nei processi di sviluppo e utilizzo delle Automazioni.
📕 Vedremo insieme alcuni esempi dell'utilizzo di Autopilot in diversi tool della Suite UiPath:
Autopilot per Studio Web
Autopilot per Studio
Autopilot per Apps
Clipboard AI
GenAI applicata alla Document Understanding
👨🏫👨💻 Speakers:
Stefano Negro, UiPath MVPx3, RPA Tech Lead @ BSP Consultant
Flavio Martinelli, UiPath MVP 2023, Technical Account Manager @UiPath
Andrei Tasca, RPA Solutions Team Lead @NTT Data
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionAggregage
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
2. hello
● why you should care
● what is it, what it does
● how it works, how you can use it
● architecture
● objects
● recovery
● file system
● hadoop integration
● distributed computation
● object classes
● who we are, why we do this
3. why should you care about another
storage system?
requirements, time, money
4. requirements
● diverse storage needs
● object storage
● block devices (for VMs) with snapshots, cloning
● shared file system with POSIX, coherent caches
● structured data... files, block devices, or objects?
● scale
● terabytes, petabytes, exabytes
● heterogeneous hardware
● reliability and fault tolerance
5. time
● ease of administration
● no manual data migration, load balancing
● painless scaling
● expansion and contraction
● seamless migration
6. money
● low cost per gigabyte
● no vendor lock-in
● software solution
● run on commodity hardware
● open source
8. unified storage system
● objects
● small or large
● multi-protocol Netflix VM Hadoop
● block devices
radosgw RBD Ceph DFS
● snapshots, cloning RADOS
● files
● cache coherent
● snapshots
● usage accounting
9. open source
● LGPLv2
● copyleft
● free to link to proprietary code
● no copyright assignment
● no dual licensing
● no “enterprise-only” feature set
● active community
● commercial support
10. distributed storage system
● data center (not geo) scale
● 10s to 10,000s of machines
● terabytes to exabytes
● fault tolerant
● no SPoF
● commodity hardware
– ethernet, SATA/SAS, HDD/SSD
– RAID, SAN probably a waste of time, power, and money
11. architecture
● monitors (ceph-mon)
● 1s-10s, paxos
● lightweight process
● authentication, cluster membership,
critical cluster state
● object storage daemons (ceph-osd)
● 1s-10,000s
● smart, coordinate with peers
● clients (librados, librbd)
● zillions
● authenticate with monitors, talk directly
to ceph-osds
● metadata servers (ceph-mds)
● 1s-10s
● build POSIX file system on top of objects
12. rados object storage model
● pools
● 1s to 100s
● independent namespaces or object collections
● replication level, placement policy
● objects
● trillions
● blob of data (bytes to gigabytes)
● attributes (e.g., “version=12”; bytes to kilobytes)
● key/value bundle (bytes to gigabytes)
13. rados object API
● librados.so
● C, C++, Python, Java. shell.
● read/write (extent), truncate, remove; get/set/remove xattr or key
● like a file or .db file
●
efficient copy-on-write clone
●
atomic compound operations/transactions
●
read + getxattr, write + setxattr
● compare xattr value, if match write + setxattr
●
classes
● load new code into cluster to implement new methods
●
calc sha1, grep/filter, generate thumbnail
● encrypt, increment, rotate image
14. object storage
● client/server, host/device paradigm doesn't scale
● idle servers are wasted servers
● if storage devices don't coordinate, clients must
● ceph-osds are intelligent storage daemons
● coordinate with peers
● sensible, cluster-aware protocols
● flexible deployment
● one per disk
● one per host
● one per RAID volume
● sit on local file system
● btrfs, xfs, ext4, etc.
15. data distribution
● all objects are replicated N times
● objects are automatically placed, balanced, migrated
in a dynamic cluster
● must consider physical infrastructure
● ceph-osds on hosts in racks in rows in data centers
● three approaches
● pick a spot; remember where you put it
● pick a spot; write down where you put it
● calculate where to put it, where to find it
16. CRUSH
● pseudo-random placement algorithm
● uniform, weighted distribution
● fast calculation, no lookup
● placement rules
● in terms of physical infrastructure
– “3 replicas, same row, different racks”
● predictable, bounded migration on changes
● N → N + 1 ceph-osds means a bit over 1/Nth of
data moves
17. object placement
pool
hash(object name) % num_pg = pg
placement group (PG)
CRUSH(pg, cluster state, rule) = [A, B]
X
18. replication
● all data replicated N times
● ceph-osd cluster handles replication
● client writes to first replica
● reduce client bandwidth
● “only once” semantics
● cluster maintains strict consistently
19. recovery
● dynamic cluster
● nodes are added, removed
● nodes reboot, fail, recover
● “recovery” is the norm
● “map” records cluster state at point in time
– ceph-osd node status (up/down, weight, IP)
– CRUSH function specifying desired data distribution
● ceph-osds cooperatively migrate data to achieve that
● any map update potentially triggers data migration
● ceph-osds monitor peers for failure
● new nodes register with monitor
● administrator adjusts weights, mark out old hardware, etc.
20. rbd – rados block device
● replicated, reliable, high-performance virtual disk
● striped over objects across entire cluster
● thinly provisioned, snapshots
● image cloning (real soon now)
KVM
● well integrated librbd
librados
● Linux kernel driver (/dev/rbd0) KVM/Xen
● qemu/KVM + librbd ext4 rbd
rbd kernel
● libvirt, OpenStack
● sever link between virtual machine and host
● fail-over, live migration
21. ceph distributed file system
● shared cluster-coherent file system
● separate metadata and data paths
● avoid “server” bottleneck inherent in NFS etc
● ceph-mds cluster
● manages file system hierarchy
● redistributes load based on workload
● ultimately stores everything in objects
● highly stateful client sessions
● lots of caching, prefetching, locks and leases
22. an example
● mount -t ceph 1.2.3.4:/ /mnt
● 3 ceph-mon RT
● 2 ceph-mds RT (1 ceph-mds to -osd RT)
ceph-mon ceph-osd
● cd /mnt/foo/bar
● 2 ceph-mds RT (2 ceph-mds to -osd RT)
● ls -al
● open
● readdir
– 1 ceph-mds RT (1 ceph-mds to -osd RT)
● stat each file
● close ceph-mds
● cp * /tmp
● N ceph-osd RT
23. dynamic subtree partitioning
Root
ceph-mds
● efficient ● scalable
● hierarchical partition preserve ● arbitrarily partition metadata
locality ● adaptive
● dynamic ● move work from busy to idle
● daemons can join/leave servers
● take over for failed nodes ● replicate hot metadata
27. hadoop
● seamless integration ● can interact “normally” with
● Java libcephfs wrapper Hadoop data
● Hadoop CephFileSystem ● kernel mount
● drop-in replacement for HDFS ● ceph-fuse
● locality ● NFS/CIFS
● exposes data layout ● can colocate Hadoop with
● reads from local replica “normal” storage
● first write does not go to local ● avoid staging/destaging
node
28. distributed computation models
● object classes ● map/reduce
● tightly couple ● colocation of
computation with data computation and data is
optimization only
● carefully sandboxed
● more loosely sandboxed
● part of I/O pipeline
● orchestrated data flow
● atomic transactions between files, nodes
● rich data abstraction ● job scheduling
– blob of bytes (file) ● limited storage
– xattrs abstraction
– key/value bundle
31. best tool for the job
● key/value stores ● hbase/bigtable
● cassandra, riak, redis ● tablets, logs
● object store ● map/reduce
● RADOS (ceph) ● data flows
● map/reduce ● percolator
“filesystems” ● triggers, transactions
● GFS, HDFS
● POSIX filesystems
● ceph, lustre, gluster
32. can I deploy it already?
● rados object store is stable
● librados
● radosgw (RESTful APIs)
● rbd rados block device
● commercial support
● file system is almost ready
● feature complete
● suitable for testing, PoC, benchmarking
● needs testing, deliberate qa effort for production
33. why we do this
● limited options for scalable open source storage
● orangefs, lustre
● glusterfs
● HDFS
● proprietary solutions
● marry hardware and software
● expensive
● don't scale (well or out)
● industry needs to change
34. who we are
● created at UC Santa Cruz (2007)
● supported by DreamHost (2008-2011)
● spun off as new company (2012)
● downtown Los Angeles, downtown San Francisco
● growing user and developer community
● Silicon Valley, Asia, Europe
● Debian, SuSE, Canonical, RedHat
● cloud computing stacks
● we are hiring
● C/C++/Python developers
● sysadmins, testing engineers
http://ceph.com/
35.
36. librados, radosgw
● librados
HTTP
● direct parallel access to
cluster haproxy
● rich API HTTP
● radosgw your app radosgw radosgw
librados librados librados
● RESTful object storage
– S3, Swift APIs
● proxy HTTP to rados
● ACL-based security for
the big bad internet
37. why we like btrfs
● pervasive checksumming
● snapshots, copy-on-write
● efficient metadata (xattrs)
● inline data for small files
● transparent compression
● integrated volume management
● software RAID, mirroring, error recovery
● SSD-aware
● online fsck
● active development community