This document discusses deploying a Ceph cluster on DreamCompute, an OpenStack-powered cloud computing service from DreamHost. It begins with an overview of Ceph's scalability and uses for object, block, and file storage. The document then discusses DreamCompute's open source infrastructure and deploying Ceph using tools like Juju. It provides details on configuring the Ceph cluster by deploying MONs, OSDs, the RGW gateway, and MDS. It concludes by discussing next steps like geo-replication and erasure coding, and opportunities to get involved with the Ceph community.

1. Deploying a Ceph Cluster on DreamCompute
In-Ceph-tion

2. #DREAMCON2013
2
•Ceph in <30s
•DreamCompute
•OpenStack && Ceph
•Deploying Ceph
•Questions
Wat?

3. #DREAMCON2013
3
On commodity hardware
Ceph can run on any
infrastructure, metal
or virtualized to
provide a cheap and
powerful storage
cluster.
Object, block, and file
Low overhead doesn’t
mean just
hardware, it means
people too!
Awesomesauce
Infrastructure-aware
placement algorithm
allows you to do really
cool stuff.
Huge and beyond
Designed for
exabyte, current
implementations in
the multi-petabyte.
HPC, Big
Data, Cloud, raw
storage.
…besides wicked-awesome?
What is Ceph?
Software All-in-1 CRUSH Scale

4. #DREAMCON2013
4
Find out more!
Ceph.com
…but you can find out more
Use it today
Dreamhost.com/cloud/DreamObjects
Get Support
Inktank.com
That WAS fast

5. #DREAMCON2013
“DreamCompute is a highly scalable and cost-effective
cloud computing service that is built to power everything
from Web and mobile applications, digital media and
ecommerce Web sites, big data analytics, and test and
development environments. Using DreamCompute’s
open source infrastructure-as-a-service platform that is
powered by OpenStack®, customers can create and
prosper in the cloud.”
5
Sexy OpenStack Goodness
DreamCompute

6. #DREAMCON2013
6
NOM NOM NOM
That’s a mouthful

7. #DREAMCON2013
7
Leave Amazon behind
Spin up infrastructure
in the cloud. Want
100 Ubuntu
machines? No
problem!
No black boxes here
Everything is Open
Source tech, so you
can see the guts or
even help build them.
Can’t stop me now!
Tons of adoption and
support from
everyone from the
huge-mongous to the
boutique.
GUI or API, you pick
Whether you’re
plugging in
orchestration
frameworks or
spinning up a single
machine.
Distilled
DreamCompute
Your EC2 Open Momentum Easy

8. #DREAMCON2013
Storage / Compute
Dell Power Edge 515s
• Six Core AMD Processor
• 32G RAM
• Two on-board 300GB SAS drives (RAID-
1) contain the OS
• H710 (LSI9260) controller
• Storage is 12x 3TB drives, JBODs (RAID-
0, one drive per set)
• Two 10G NICs for data (or a single dual-
port 10G NIC)
• One NIC dedicated to IPMI
(10/100/1000Mbps)
• One 1G NIC dedicated to management
(isolated from IPMI connections)
8
Geek us out
Non-Storage (mon/gateway/mgmt)
Dell Power Edge R415s
•Two on-board 300GB SAS drives
(RAID-1) contain the OS
• Two 1TB SAS drives on same
controller (RAID-1) for logs
• Single 10G NIC
• One NIC dedicated to IPMI
(10/100/1000Mbps)
• One 1G NIC dedicated to management
What about the guts?

9. #DREAMCON2013
9
Resilient
Everything was designed for failure.
Not for the feint of heart
Capacity
Optimized for throughput (NIC speed)
Replication
This means that replication within
a pod can happen at very nearly
the same rate as replication
within a single rack.
Networking

10. #DREAMCON2013
1
0
I have no clue what you just said
Huh?

11. #DREAMCON2013
Just rest assured that the DREAMHOST
guys are really taking their time to get it
RIGHT. The system is designed to have
VAST RESOURCES at the ready, be
RESILIENT by design, and provide
ENTERPRISE-CLASS data security.
1
1
Because the DreamHost guys are awesome!
It’s Awesome

12. #DREAMCON2013
1
2
Good together
OpenStack && Ceph

13. #DREAMCON2013
Cinder
“Cinder provides an infrastructure for
managing volumes in OpenStack. It was
originally a Nova component called
nova-volume, but has become an
independent project since the Folsom
release.”
Cinder is able to boot a VM using a copy-
on-write clone of an image stored in
Glance.
1
3
Ceph’s best pals
Glance
“The Glance project provides services
for discovering, registering, and
retrieving virtual machine images.
Glance has a RESTful API that allows
querying of VM image metadata as well
as retrieval of the actual image.”
Ceph is able to store machine images for
Glance in a block device.
Cinder && Glance

14. #DREAMCON2013
1
4
Squash Hotspots
Multiple hosts = parallel workload
But what does that mean?
Instant Clones
No time to boot for many images
Live migration
Shared storage allows you to
move instances between compute
nodes transparently.
Looks delicious

15. #DREAMCON2013
Swift
“Swift is a highly available, distributed,
eventually consistent object/blob store.
Organizations can use Swift to store lots
of data efficiently, safely, and cheaply.
Swift is currently a core project.
1
5
The artist previously known as Swift
Why Ceph?
Ceph allows you to do both object and
block for OpenStack in a single cluster.
Additionally you can use the Swift API
natively via Ceph’s RESTful gateway.
Performance, fault tolerance, and self-
management advantages.
Object Storage

16. #DREAMCON2013
1
6
Come for the block
Stay for the object and file
Ceph, it only takes once!
Reduced Overhead
Easier to manage one cluster
File applications
While not production ready, many
are using CephFS for things like
image distribution.
Gateway Drug

17. #DREAMCON2013
1
7
Where the metal meets the…software
Deploying Ceph

18. #DREAMCON2013
1
8
Procedural, Ruby
Written in Ruby, this
is more of the dev-
side of DevOps. Once
you get past the
learning curve it’s
powerful though.
Model-driven
Aimed more at the
sysadmin, this
procedural tool has a
very wide penetration
(even on Windows!).
Agentless, whole stack
Using the built-in
OpenSSH in your
OS, this super easy
tool goes further up
the stack than most.
Fast, 0MQ
Using ZeroMQ this tool
is designed for massive
scale and fast, fast, fast.
Unfortunately 0MQ has
no built in encryption.
The new hotness
Orchestration
Chef Puppet Ansible Salt

19. #DREAMCON2013
1
9
Canonical Unleashed
Being language
agnostic, this tool can
completely encapsulate
a service. Can also
handle provisioning all
the way down to
hardware.
Dell has skin in the game
Complete operations
platform that can dive
all the way down to
BIOS/RAID level.
Others are joining in
Custom provisioning
and
orchestration, just
one example of how
busy this corner of
the market is.
Doing it w/o a tool
If you prefer not to
use a tool, Ceph gives
you an easy way to
deploy your cluster by
hand.
MOAR HOTNESS
Orchestration Cont’d
Juju Crowbar ComodIT Ceph-deploy

20. #DREAMCON2013
2
0
GUI or CLI
Made to be drag-and-drop easy.
My favorite flavor
Step-by-step
ceph.com/dev-notes/deploying-ceph-with-juju/
Don’t trust me
There are lots of tools and recipes
available. I use Juju because it
makes sense to my brain. Use
what works for you!
Ceph && Juju

21. #DREAMCON2013
2
1
Get credentials
OpenStack or EC2
Get to the action already
Config
Most are YAML
Bootstrap
Most need a machine to act as the
air traffic controller. Drop this in
to start spinning up
infrastructure.
Deploy!

22. #DREAMCON2013
2
2
Spin up MONs
Quorum and auth
MOAR
Spin up OSDs
Storage and replication
Voila! Storage!
You now have a functional Ceph
cluster that you can use for
OpenStack, raw storage, or your
own nefarious plans for world-
domination!
Deploy MOAR

23. #DREAMCON2013
2
3
Sping up RGW
RESTful gateway for object storage
For the over achiever
Spin up MDS
Metadata servers for CephFS (beta)
Second Cluster
With incremental snapshots (RBD),
gateway replication (RGW), and
directory-level snapshots (CephFS)
a second cluster can be set up for
disaster recovery.
Extra Credit

24. #DREAMCON2013
2
4
This Ceph thing sounds hot.
What’s Next?

25. #DREAMCON2013
2
5
An ongoing process
While the first pass
for disaster recovery
is done, we want to
get to built-in, world-
wide replication.
Reception efficiency
Currently underway
in the community!
Headed to dynamic
Can already do this in
a static pool-based
setup. Looking to get
to a use-based
migration.
The evolution of open
This project is moving
fast and we plan to
continue moving with
it!
Hop on board!
The Ceph Train
Geo-Replication Erasure Coding Tiering More OpenStack

26. #DREAMCON2013
2
6
Quarterly Online Summit
Coming up next
week, this online
summit puts the core
devs together with
the Ceph community.
Coming to a city near you!
Already planned for
Santa Clara and
London. Keep an eye
out:
http://inktank.com/cephdays/
Geek-on-duty
During the week
there are times when
Ceph experts are
available to help. Stop
by oftc.net/ceph
Email makes the world go
Our mailing lists are
very active, check out
ceph.com for details
on how to join in!
Open Source is Open!
Get Involved!
CDS Ceph Day IRC Lists

27. #DREAMCON2013
2
7
Comments? Anything for the good of the cause?
Questions?
E-MAIL
patrick@inktank.com
WEBSITE
Ceph.com
SOCIAL
@scuttlemonkey
@ceph
Facebook.com/cephstorage

28. SEE YOU NEXT YEAR!
THANKS FOR COMING