RBD, the RADOS Block Device in Ceph, gives you virtually unlimited scalability (without downtime), high performance, intelligent balancing and self-healing capabilities that traditional SANs can't provide. Ceph achieves this higher throughput through a unique system of placing objects across multiple nodes, and adaptive load balancing that replicates frequently accessed objects over more nodes. This talk will give a brief overview of the Ceph architecture, current integration with Apache CloudStack, and recent advancements with Xen and blktap2.

1. Ceph, Xen, and CloudStack:
Semper Melior
Xen User Summit| New Orleans, LA | 18 SEP 2013

2. •Patrick McGarry
•Community monkey
•Inktank / Ceph
•/. > ALU > P4
•@scuttlemonkey
•patrick@inktankcom
Accept no substitutes
C’est Moi
2

3. •Ceph in <30s
•Ceph, a little bit more
•Ceph in the wild
•Orchestration
•Community status
•What’s Next?
•Questions
The plan, Stan
Welcome!
3

4. On commodity hardware
Ceph can run on any
infrastructure, metal
or virtualized to
provide a cheap and
powerful storage
cluster.
Object, block, and file
Low overhead doesn’t
mean just hardware,
it means people too!
Awesomesauce
Infrastructure-aware
placement algorithm
allows you to do really
cool stuff.
Huge and beyond
Designed for exabyte,
current
implementations in
the multi-petabyte.
HPC, Big Data, Cloud,
raw storage.
…besides wicked-awesome?
What is Ceph?
Software All-in-1 CRUSH Scale
4

5. Find out more!
Ceph.com
…but you can find out more
Use it today
Dreamhost.com/cloud/DreamObjects
Get Support
Inktank.com
That WAS fast
5

6. OBJECTS VIRTUAL DISKS FILES & DIRECTORIES
CEPH
FILE SYSTEM
A distributed, scale-out
filesystem with POSIX
semantics that provides
storage for a legacy and
modern applications
CEPH
GATEWAY
A powerful S3- and Swift-
compatible gateway that
brings the power of the
Ceph Object Store to
modern applications
CEPH
BLOCK DEVICE
A distributed virtual block
device that delivers high-
performance, cost-effective
storage for virtual machines
and legacy applications
CEPH OBJECT STORE
A reliable, easy to manage, next-generation distributed object
store that provides storage of unstructured data for applications
6

7. 7

8. 8

9. • CRUSH
– Pseudo-random placement
algorithm
– Ensures even distribution
– Repeatable, deterministic
– Rule-based configuration
• Replica count
• Infrastructure topology
• Weighting
9

10. 10 10 01 01 10 10 01 11 01 10
10 10 01 01 10 10 01 11 01 10
hash(object name) % num pg
CRUSH(pg, clusterstate,ruleset)
10

11. 10 10 01 01 10 10 01 11 01 10
10 10 01 01 10 10 01 11 01 10
11

12. CLIENT
12

13. 13

14. 14

15. 15

16. CLIENT
??
16

17. …with Marty Stouffer
Ceph in the Wild
17

18. No incendiary devices please…
Linux Distros
18

19. Object&& Block
Via RBD and RGW (Swift API)
Our BFF
Identity
Via Keystone
More coming!
Work continues with updates in
Havana and Icehouse.
OpenStack
19

20. Block
Alternate primary, and secondary
Community maintained
Community
Wido from 42on.com
More coming in 4.2!
Snapshot & backup support
Cloning (layering) support
No NFS for system VMs
Secondary/Backup storage (s3)
CloudStack
20

21. A blatent ripoff!
Primary Storage Flow
•The mgmtservernevertalksto theCeph cluster
•One mgmtservercan manage 1000s of
hypervisors
•Mgmtservercan be clustered
•Multiple Ceph clusters/poolscan be added to
CloudStack cluster
21

22. A pretty package
A commercially
packaged OpenStack
solution back by
Ceph.
RADOS for Archipelago
Virtual server
management
software tool on top
of Xen or KVM.
RBD backed
Complete
virtualization
management with
KVM and containers.
BBC territory
Talk next week in
Berlin
So many delicious flavors
Other Cloud
SUSE Cloud Ganeti Proxmox OpenNebula
22

23. Since 2.6.35
Kernel clients for RBD
and CephFS. Active
development as a
Linux file system.
iSCSI ahoy!
One of the Linux iSCSI
target frameworks.
Emulates: SBC (disk),
SMC (jukebox), MMC
(CD/DVD), SSC (tape),
OSD.
Getting creative
Creative community
member used Ceph to
back their VMWare
infrastructure via
fibre channel.
You can always use more friends
Project Intersection
Kernel STGT VMWare
Love me!
Slightly out-of-date.
Some work has been
done, but could use
some love.
Wireshark
23

24. CephFS
CephFS can serve as a
drop-in replacement
for HDFS.
Upstream
Ceph vfs module
upstream samba.
CephFS or RBD
Reexporting CephFS
or RBD for NFS/CIFS.
MOAR projects
Project Intersection
Hadoop Samba Ganesha
Recently Open Source
Commercially
supported product
from Citrix. Recently
Open Sourced. Still a
bit of a tech preview.
XenServer
24

25. Support for libvirt
XenServer can manipulate Ceph!
Don’t let the naming fool you, it’s easy
Blktap{2,3,asplode}
Qemu; new boss, same as the old boss
(but not really)
What’s in a name?
Ceph :: XenServer :: Libvirt
Block device :: VDI :: storage vol
Pool :: Storage Repo :: storage pool
Doing it with Xen*
25

26. Thanks David Scott!
XenServer host arch
Xapi, XenAPI
xenopsd S M adapters
libvirt
libxl ceph ocfs2
libxenguest libxc qemu
xen
Client
(CloudStack, OpenStack, XenDesktop)
26

27. Come for the block
Stay for the object and file
No matter what you use!
Reduced Overhead
Easier to manage one cluster
“Other Stuff”
CephFS prototypes
fast development profile
ceph-devel
lots of partner action
Gateway Drug
27

28. Squash Hotspots
Multiple hosts = parallel workload
But what does that mean?
Instant Clones
No time to boot for many images
Live migration
Shared storage allows you to
move instances between compute
nodes transparently.
Blocks are delicious
28

29. Flexible APIs
Native support for swift and s3
And less filling!
Secondary Storage
Coming with 4.2
Horizontal Scaling
Easy with HAProxy or others
Objects can juggle
29

30. Neat prototypes
Image distribution to hypervisors
You can dress them up, but you can’t take them anywhere
Still early
You can fix that!
Outside uses
Great way to combine resources.
Files are tricksy
30

31. Where the metal meets the…software
Deploying this stuff
31

32. Procedural, Ruby
Written in Ruby, this
is more of the dev-
side of DevOps. Once
you get past the
learning curve it’s
powerful though.
Model-driven
Aimed more at the
sysadmin, this
procedural tool has a
very wide penetration
(even on Windows!).
Agentless, whole stack
Using the built-in
OpenSSH in your OS,
this super easy tool
goes further up the
stack than most.
Fast, 0MQ
Using ZeroMQ this tool
is designed for massive
scale and fast, fast, fast.
Unfortunately 0MQ has
no built in encryption.
The new hotness
Orchestration
Chef Puppet Ansible Salt
32

33. Canonical Unleashed
Being language
agnostic, this tool can
completely encapsulate
a service. Can also
handle provisioning all
the way down to
hardware.
Dell has skin in the game
Complete operations
platform that can dive
all the way down to
BIOS/RAID level.
Others are joining in
Custom provisioning
and orchestration,
just one example of
how busy this corner
of the market is.
Doing it w/o a tool
If you prefer not to
use a tool, Ceph gives
you an easy way to
deploy your cluster by
hand.
MOAR HOTNESS
Orchestration Cont’d
Juju Crowbar ComodIT Ceph-deploy
33

34. All your space are belong to us
Ceph Community
34

35. 35

36. Up and to the right!
Code Contributions
36

37. Up and to the right!
Commits
37

38. Up and to the right!
List Participation
38

39. This Ceph thing sounds hot.
What’s Next?
39

40. An ongoing process
While the first pass
for disaster recovery
is done, we want to
get to built-in, world-
wide replication.
Reception efficiency
Currently underway
in the community!
Headed to dynamic
Can already do this in
a static pool-based
setup. Looking to get
to a use-based
migration.
Making it open-er
Been talking about it
forever. The time is
coming!
Hop on board!
The Ceph Train
Geo-Replication ErasureCoding Tiering Governance
40

41. Quarterly Online Summit
Online summit puts
the core devs together
with the Ceph
community.
Not just for NYC
More planned,
including Santa Clara
and London. Keep an
eye out:
http://inktank.com/cephdays/
Geek-on-duty
During the week
there are times when
Ceph experts are
available to help. Stop
by oftc.net/ceph
Email makes the world go
Our mailing lists are
very active, check out
ceph.com for details
on how to join in!
Open Source is Open!
Get Involved!
CDS Ceph Day IRC Lists
41

42. http://wiki.ceph.com/04De
velopment/Project_Ideas
Lists, blueprints,
sideboard, paper cuts,
etc.
http://tracker.ceph.com/
All the things!
New #ceph-devel
Splitting off developer
chatter to make it
easier to filter
discussions.
http://ceph.com/resources
/mailing-list-irc/
Our mailing lists are
very active, check out
ceph.com for details
on how to join in!
Patches welcome
Projects
Wiki Redmine IRC Lists
42

43. Comments? Anything for the good of the cause?
Questions?
E-MAIL
patrick@inktank.com
WEBSITE
Ceph.com
SOCIAL
@scuttlemonkey
@ceph
Facebook.com/cephstorage
43