Ceph Day Santa Clara: Ceph Fundamentals
•Download as PPTX, PDF•
7 likes•1,851 views
Ross Turk, VP, Marketing & Community, Inktank Ceph is an open source distributed object store, network block device, and file system designed for reliability, performance, and scalability. It runs on standard hardware, has no single point of failure, and is supported by the Linux kernel. It also works great with OpenStack and CloudStack. If you’ve heard of Ceph but aren’t sure where it fits into your plans, this is the talk for you. Designed for those who are new to Ceph, this talk will cover Ceph’s design principles, overall architecture, and integration with other operational systems.
Recommended
Recommended
More Related Content
Similar to Ceph Day Santa Clara: Ceph Fundamentals
Similar to Ceph Day Santa Clara: Ceph Fundamentals (20)
Recently uploaded
Recently uploaded (20)
Ceph Day Santa Clara: Ceph Fundamentals
- 1. Ceph Fundamentals Ross Turk VP Community, Inktank
- 2. ME ME ME ME ME ME. 2 Ross Turk VP Community, Inktank ross@inktank.com @rossturk inktank.com | ceph.com
- 3. Ceph Architectural Overview Ah! Finally, 32 slides in and he gets to the nerdy stuff. 3
- 4. 4 RADOS A reliable, autonomous, distributed object store comprised of self-healing, self-managing, intelligent storage nodes LIBRADOS A library allowing apps to directly access RADOS, with support for C, C++, Java, Python, Ruby, and PHP RBD A reliable and fully- distributed block device, with a Linux kernel client and a QEMU/KVM driver CEPH FS A POSIX-compliant distributed file system, with a Linux kernel client and support for FUSE RADOSGW A bucket-based REST gateway, compatible with S3 and Swift APP APP HOST/VM CLIENT
- 5. 5 RADOS A reliable, autonomous, distributed object store comprised of self-healing, self-managing, intelligent storage nodes LIBRADOS A library allowing apps to directly access RADOS, with support for C, C++, Java, Python, Ruby, and PHP RBD A reliable and fully- distributed block device, with a Linux kernel client and a QEMU/KVM driver CEPH FS A POSIX-compliant distributed file system, with a Linux kernel client and support for FUSE RADOSGW A bucket-based REST gateway, compatible with S3 and Swift APP APP HOST/VM CLIENT
- 6. 6 DISK FS DISK DISK OSD DISK DISK OSD OSD OSD OSD FS FS FSFS btrfs xfs ext4 MMM
- 8. 8 Monitors: • Maintain cluster membership and state • Provide consensus for distributed decision-making • Small, odd number • These do not serve stored objects to clients M OSDs: • 10s to 10000s in a cluster • One per disk • (or one per SSD, RAID group…) • Serve stored objects to clients • Intelligently peer to perform replication and recovery tasks
- 9. 9 RADOS A reliable, autonomous, distributed object store comprised of self-healing, self-managing, intelligent storage nodes LIBRADOS A library allowing apps to directly access RADOS, with support for C, C++, Java, Python, Ruby, and PHP RBD A reliable and fully- distributed block device, with a Linux kernel client and a QEMU/KVM driver CEPH FS A POSIX-compliant distributed file system, with a Linux kernel client and support for FUSE RADOSGW A bucket-based REST gateway, compatible with S3 and Swift APP APP HOST/VM CLIENT
- 11. L LIBRADOS • Provides direct access to RADOS for applications • C, C++, Python, PHP, Java, Erlang • Direct access to storage nodes • No HTTP overhead
- 12. 12 RADOS A reliable, autonomous, distributed object store comprised of self-healing, self-managing, intelligent storage nodes LIBRADOS A library allowing apps to directly access RADOS, with support for C, C++, Java, Python, Ruby, and PHP RBD A reliable and fully- distributed block device, with a Linux kernel client and a QEMU/KVM driver CEPH FS A POSIX-compliant distributed file system, with a Linux kernel client and support for FUSE RADOSGW A bucket-based REST gateway, compatible with S3 and Swift APP APP HOST/VM CLIENT
- 14. 14 RADOS Gateway: • REST-based object storage proxy • Uses RADOS to store objects • API supports buckets, accounts • Usage accounting for billing • Compatible with S3 and Swift applications
- 16. 16 RADOS A reliable, autonomous, distributed object store comprised of self-healing, self-managing, intelligent storage nodes LIBRADOS A library allowing apps to directly access RADOS, with support for C, C++, Java, Python, Ruby, and PHP CEPH FS A POSIX-compliant distributed file system, with a Linux kernel client and support for FUSE RADOSGW A bucket-based REST gateway, compatible with S3 and Swift APP APP HOST/VM CLIENT RBD A reliable and fully- distributed block device, with a Linux kernel client and a QEMU/KVM driver
- 20. 20 RADOS Block Device: • Storage of disk images in RADOS • Decouples VMs from host • Images are striped across the cluster (pool) • Snapshots • Copy-on-write clones • Support in: • Mainline Linux Kernel (2.6.39+) • Qemu/KVM, native Xen coming soon • OpenStack, CloudStack, Nebula, Proxmox
- 21. 21 RADOS A reliable, autonomous, distributed object store comprised of self-healing, self-managing, intelligent storage nodes LIBRADOS A library allowing apps to directly access RADOS, with support for C, C++, Java, Python, Ruby, and PHP RBD A reliable and fully- distributed block device, with a Linux kernel client and a QEMU/KVM driver CEPH FS A POSIX-compliant distributed file system, with a Linux kernel client and support for FUSE RADOSGW A bucket-based REST gateway, compatible with S3 and Swift APP APP HOST/VM CLIENT
- 23. 23 Metadata Server • Manages metadata for a POSIX-compliant shared filesystem • Directory hierarchy • File metadata (owner, timestamps, mode, etc.) • Stores metadata in RADOS • Does not serve file data to clients • Only required for shared filesystem
- 24. What Makes Ceph Unique? Part one: it never, ever remembers where it puts stuff. 24
- 26. How Long Did It Take You To Find Your Keys This Morning? azmeen, Flickr / CC BY 2.0 26
- 28. Dear Diary: Today I Put My Keys on the Kitchen Counter Barnaby, Flickr / CC BY 2.0 28
- 30. I Always Put My Keys on the Hook By the Door vitamindave, Flickr / CC BY 2.0 30
- 31. HOW DO YOU FIND YOUR KEYS WHEN YOUR HOUSE IS INFINITELY BIG AND ALWAYS CHANGING? 31
- 32. The Answer: CRUSH!!!!! pasukaru76, Flickr / CC SA 2.0 32
- 33. 33 OBJECT 10 10 01 01 10 10 01 11 01 10 hash(object name) % num pg CRUSH(pg, cluster state, rule set)
- 34. 34 OBJECT 10 10 01 01 10 10 01 11 01 10
- 35. 35 CRUSH • Pseudo-random placement algorithm • Fast calculation, no lookup • Repeatable, deterministic • Statistically uniform distribution • Stable mapping • Limited data migration on change • Rule-based configuration • Infrastructure topology aware • Adjustable replication • Weighting
- 36. 36 CLIENT ??
- 37. 37 NAME:"foo" POOL:"bar" 0101 1111 1001 0011 1010 1101 0011 1011 "bar" = 3 hash("foo") % 256 = 0x23 OBJECT PLACEMENT GROUP 24 3 12 CRUSH TARGET OSDsPLACEMENT GROUP 3.23 3.23
- 38. 38
- 39. 39
- 40. 40 CLIENT ??
- 41. What Makes Ceph Unique Part two: it has smart block devices for all those impatient, selfish VMs. 41
- 43. HOW DO YOU SPIN UP THOUSANDS OF VMs INSTANTLY AND EFFICIENTLY? 43
- 44. 144 44 0 0 0 0 instant copy = 144
- 47. What Makes Ceph Unique? Part three: it has powerful friends, ones you probably already know. 47
- 48. 48 M M M APACHE CLOUDSTACK HYPER- VISOR PRIMARY STORAGE POOL SECONDARY STORAGE POOL snapshots templates images
- 49. 49 M M M OPENSTACK KEYSTONE API SWIFT API CINDER API GLANCE API NOVA API HYPER- VISOR RADOSGW
- 50. What Makes Ceph Unique? Part three: clustered metadata 50
- 52. 52 M M M
- 53. 53 one tree three metadata servers ??
- 54. 54
- 55. 55
- 56. 56
- 57. 57
- 59. Questions? 59 Ross Turk VP Community, Inktank ross@inktank.com @rossturk inktank.com | ceph.com
Editor's Notes
- RADOS is a distributed object store, and it’s the foundation for Ceph. On top of RADOS, the Ceph team has built three applications that allow you to store data and do fantastic things. But before we get into all of that, let’s start at the beginning of the story.
- But that’s a lot to digest all at once. Let’s start with RADOS.
- MDSs store all of their data within RADOS itself, but there’s still a problem…
- There are multiple MDSs!
- So how do you have one tree and multiple servers?
- If there’s just one MDS (which is a terrible idea), it manages metadata for the entire tree.
- When the second one comes along, it will intelligently partition the work by taking a subtree.
- When the third MDS arrives, it will attempt to split the tree again.
- Same with the fourth.
- A MDS can actually even just take a single directory or file, if the load is high enough. This all happens dynamically based on load and the structure of the data, and it’s called “dynamic subtree partitioning”.