Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The Future of Cloud Software Defined Storage with Ceph: Andrew Hatfield, Red Hat

2,867 views

Published on

Audience: Intermediate

About: Learn how cloud storage differs to traditional storage systems and how that delivers revolutionary benefits.

Starting with an overview of how Ceph integrates tightly into OpenStack, you’ll see why 62% of OpenStack users choose Ceph, we’ll then take a peek into the very near future to see how rapidly Ceph is advancing and how you’ll be able to achieve all your childhood hopes and dreams in ways you never thought possible.

Speaker Bio: Andrew Hatfield – Practice Lead–Cloud Storage and Big Data, Red Hat

Andrew has over 20 years experience in the IT industry across APAC, specialising in Databases, Directory Systems, Groupware, Virtualisation and Storage for Enterprise and Government organisations. When not helping customers slash costs and increase agility by moving to the software-defined storage future, he’s enjoying the subtle tones of Islay Whisky and shredding pow pow on the world’s best snowboard resorts.

OpenStack Australia Day - Sydney 2016
http://australiaday.openstack.org.au/sydney-2016/

Published in: Technology
  • Be the first to comment

The Future of Cloud Software Defined Storage with Ceph: Andrew Hatfield, Red Hat

  1. 1. The Future of Cloud Software Defined Storage with Ceph
  2. 2. 2 Andrew Hatfield Practice Lead – Cloud Storage and Big Data (ANZ) Red Hat andrew.hatfield@redhat.com @andrewhatfield /usr/bin/whoami
  3. 3. 3 Overview ● What is Ceph ● Integration with OpenStack – Stuff you already know is cool ● New stuff coming – Even more cool stuff!
  4. 4. 4 What is Ceph
  5. 5. 5 PROPRIETARY HARDWARE PROPRIETARY SOFTWARE SUPPORT & MAINTENANCE COMPUTER DISK COMPUTER DISK COMPUTER DISK STANDARD HARDWARE OPEN SOURCE SOFTWARE ENTERPRISE PRODUCTS & SERVICES COMPUTER DISK COMPUTER DISK COMPUTER DISK A Storage Revolution
  6. 6. 6 CEPH COMPONENTS RGW web services gateway for object storage, compatible with S3 and Swift LIBRADOS client library allowing apps to access RADOS (C, C++, Java, Python, Ruby, PHP) RADOS software-based, reliable, autonomous, distributed object store comprised of self-healing, self-managing, intelligent storage nodes and lightweight monitors RBD reliable, fully-distributed block device with cloud platform integration CEPHFS distributed file system with POSIX semantics and scale-out metadata management APP HOST/VM CLIENT
  7. 7. RADOS CLUSTER 7 APPLICATION M M M M M RADOS CLUSTER
  8. 8. OBJECT STORAGE DAEMONS 8 FS DISK OSD DISK OSD FS DISK OSD FS DISK OSD FS M M M
  9. 9. RADOS COMPONENTS 9 OSDs:  10s to 10,000s in a cluster  One per disk (or one per SSD, RAID group…)  Serve stored objects to clients  Intelligently peer for replication & recovery Monitors:  Maintain cluster membership and state  Provide consensus for distributed decision- making  Small, odd number  These do not serve stored objects to clients M
  10. 10. CRUSH: DYNAMIC DATA PLACEMENT 10 CRUSH:  Pseudo-random placement algorithm  Fast calculation, no lookup  Repeatable, deterministic  Statistically uniform distribution  Stable mapping  Limited data migration on change  Rule-based configuration  Infrastructure topology aware  Adjustable replication  Weighting
  11. 11. DATA IS ORGANIZED INTO POOLS 11 CLUSTER OBJECTS 10 01 01 10 10 01 11 01 1001 0110 10 01 11 01 POOLS (CONTAINING PGs) 10 01 11 01 10 01 01 10 01 10 10 01 11 01 10 01 10 01 10 11 01 11 01 10 10 01 01 01 10 10 01 01 POOL A POOL B POOL C POOL D OBJECTS OBJECTS OBJECTS
  12. 12. 12 Integration with OpenStack
  13. 13. THE RADOS GATEWAY 13 M M M RADOS CLUSTER RADOSGW LIBRADOS socket RADOSGW LIBRADOS APPLICATION APPLICATION REST
  14. 14. RADOSGW MAKES RADOS WEBBY 14 RADOSGW:  REST-based object storage proxy  Uses RADOS to store objects  API supports buckets, accounts  Usage accounting for billing  Compatible with S3 and Swift applications
  15. 15. RBD STORES VIRTUAL DISKS 15 RADOS BLOCK DEVICE:  Storage of disk images in RADOS  Decouples VMs from host  Images are striped across the cluster (pool)  Snapshots  Copy-on-write clones  Support in:  Mainline Linux Kernel (2.6.39+)  Qemu/KVM, native Xen coming soon  OpenStack, CloudStack, Nebula, Proxmox
  16. 16. KERNEL MODULE FOR MAX FLEXIBLE! 16 M M RADOS CLUSTER LINUX HOST KRBD
  17. 17. STORING VIRTUAL DISKS 17 M M RADOS CLUSTER HYPERVISOR LIBRBD VM
  18. 18. CEPH AND OPENSTACK 18 RADOSGW LIBRADOS M M RADOS CLUSTER OPENSTACK KEYSTONE CINDER GLANCE NOVASWIFT LIBRBD LIBRBD HYPER- VISOR LIBRBD
  19. 19. 19 What is New()?
  20. 20. 20 Recent news ● Ceph continues to be the #1 storage choice for Open Stack (April User Survey) ● Ceph is now an openstack.org Reference Architecture (joining LVM & NFS) ● Ceph at Percona Live for MySQL Database-as-a-Service ● CephFS is now stable and ready for testing (Jewel release) ● ceph-ansible deployment in 2 commands
  21. 21. 21 ● OpenStack Manila + CephFS for File-as-a-Service ● BlueStore – Direct to block device, no filesystem ● Active / Active Multi-master RGW for global object stores ● NFS Gateway for RGW (v4 only) for legacy access ● Active Directory support for RGW ● AWS4 Support ● Swift – Keystone v3, Object expiration, tenant namespaces, static large objects ● RBD mirroring ● Client IO prioritised over scrubbing ● Containerised Ceph Work in Progress
  22. 22. 22 THANK YOU! Andrew Hatfield Practice Lead - Cloud Storage and Big Data andrew.hatfield@ redhat.com @andrewhatfield

×