0
Ceph, Open Source,
and the Path to Ubiquity in Storage
AACS – 2014 Apr 02
Copyright © 2014 by Inktank | Private and Confidential
WHO?
2
Copyright © 2014 by Inktank | Private and Confidential
AGENDA
3
THE FORECAST
By 2020
over 15 ZB
of data will
be stored.
1 .5 ZB are
stored today.
4
Copyright © 2014 by Inktank | Private ...
THE PROBLEM
 Existing systems don’t
scale
 Increasing cost and
complexity
 Need to invest in new
platforms ahead of tim...
THE SOLUTION
PAST: SCALE UP
FUTURE: SCALE OUT
6
Copyright © 2014 by Inktank | Private and Confidential
INTRO TO CEPH
HISTORICAL TIMELINE
Copyright © 2013 by Inktank | Private and Confidential
8
RHEL-OSP &
RHEV Support
FEB 2014
MAY 2012
Lau...
ARCHITECTURAL COMPONENTS
9
Copyright © 2014 by Inktank | Private and Confidential
APP HOST/VM CLIENT
CEPH UNIFIED STORAGE
FILE
SYSTEM
BLOCK
STORAGE
OBJECT
STORAGE
Keystone
Geo-Replication
Native API
10
Multi-tenant
S3 & Swi...
TRADITIONAL STORAGE VS.
CEPH
TRADITIONAL
ENTERPRISE STORAGE
11
Copyright © 2014 by Inktank | Private and Confidential
CEPH WITH OPENSTACK
12
Copyright © 2014 by Inktank | Private and Confidential
WEB APPLICATION STORAGE
13
Copyright © 2014 by Inktank | Private and Confidential
S3/Swift S3/Swift S3/Swift S3/Swift
MULTI-SITE OBJECT STORAGE
14
Copyright © 2014 by Inktank | Private and Confidential
Copyright © 2014 by Inktank | Private and
Confidential
ARCHITECTURE
ARCHITECTURAL COMPONENTS
16
Copyright © 2014 by Inktank | Private and Confidential
APP HOST/VM CLIENT
1
DISK
FS
DISK DISK
OSD
DISK DISK
OSD OSD OSD OSD
FS FS FSFS
btrfs
xfs
ext4
MMM
17
1
M
M
M
HUMAN
18
1
Monitors:
Maintain cluster map
Provide consensus for distributed
decision-making
Must have an odd number
These do no...
2
M
M
M
CLIENT
01
10
data
metadata
20
2
Metadata Server
Manages metadata for a POSIX-
compliant shared filesystem
Directory hierarchy
File metadata (owner,
t...
CRUSH
 Pseudo-random placement algorithm
 Ensures even distribution
 Repeatable, deterministic
 Rule-based configurati...
2
10 10 01 01 10 10 01 11 01 10
10 10 01 01 10 10 01 11 01 10
hash(object name) % num pg
CRUSH(pg, cluster state, rule set...
2
CLIENT
??
24
2
25
2
26
2
CLIENT
??
27
Ceph Developer Summit
29
• Recent: “Giant”
• March 04-05
• wiki.ceph.com
• Virtual
(irc, hangout,
pad, blueprint,
youtube)...
New Contribute Page
30
• http://ceph.com/
community/
Contribute
• Source tree
• Issues
• Share
experiences
• Standups
• On...
New Ceph Wiki
31
 Accepted as a mentoring organization
 8 mentors from Inktank & Community
 http://ceph.com/gsoc2014/
 Student applicat...
Ceph Days
33
• inktank.com/
cephdays
• Recently:
London,
Frankfurt, NYC,
Santa Clara
• Aggressive
program
• Upcoming:
Sunn...
Meetups
34
• Community
organized
• World wide
• Wiki
• Ceph-community
• Goodies available
• Logistical support
• Drinkup t...
 We haven’t forgotten!
 Looking for potential founding members
 Especially important to keep the IP clean
Ceph Foundati...
Coordinated Efforts
36
• Always need help
• CentOS SIG
• OCP
• Xen
• Hadoop
• OpenStack
• CloudStack
• Ganetti
• Many more!
http://metrics.ceph.com
37
Copyright © 2014 by Inktank | Private and Confidential
THE PRODUCT
39
INKTANK CEPH ENTERPRISE
WHAT’S INSIDE?
Ceph Object and Ceph Block
Calamari
Enterprise Plugins (2014)
Support Services
C...
Ceph Inktank Ceph
Enterprise
Open-Source Ceph ✔ ✔
Object Storage ✔ ✔
Block Storage ✔ ✔
File System ✔
Management API ✔ ✔
Ma...
ROADMAP
INKTANK CEPH ENTERPRISE
42
Copyright © 2014 by Inktank | Private and Confidential
April 2014 September 2014 2015
RELEASE SCHEDULE
Copyright © 2014 by Inktank | Private and Confidential
43
2013 2014 2015
Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2
Copyright © 2014 by Inktank | Private and Confidential
QUESTIONS?
44
THANK YOU!
Patrick McGarry
Director, Community
patrick@inktank.co
m
irc.oftc.net/ceph
@scuttlemonkey
@ceph
Ceph, Open Source, and the Path to Ubiquity in Storage - AACS Meetup 2014
Ceph, Open Source, and the Path to Ubiquity in Storage - AACS Meetup 2014
Upcoming SlideShare
Loading in...5
×

Ceph, Open Source, and the Path to Ubiquity in Storage - AACS Meetup 2014

808

Published on

Everyone needs storage, but Open Source is changing how we think about storage infrastructure through new features, added durability, and reduced cost. New storage solutions like Ceph are providing distributed, flexible, powerful options that can support a myriad of use cases across object, block, and file system applications. This talk will explore the history and basics of Ceph, the current status of the community, and where the project is headed in the near future.

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
808
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
40
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide
  • Make it a horizontal timeline
  • Transcript of "Ceph, Open Source, and the Path to Ubiquity in Storage - AACS Meetup 2014"

    1. 1. Ceph, Open Source, and the Path to Ubiquity in Storage AACS – 2014 Apr 02
    2. 2. Copyright © 2014 by Inktank | Private and Confidential WHO? 2
    3. 3. Copyright © 2014 by Inktank | Private and Confidential AGENDA 3
    4. 4. THE FORECAST By 2020 over 15 ZB of data will be stored. 1 .5 ZB are stored today. 4 Copyright © 2014 by Inktank | Private and Confidential
    5. 5. THE PROBLEM  Existing systems don’t scale  Increasing cost and complexity  Need to invest in new platforms ahead of time2010 2020 IT Storage Budget Growth of data 5 Copyright © 2014 by Inktank | Private and Confidential
    6. 6. THE SOLUTION PAST: SCALE UP FUTURE: SCALE OUT 6 Copyright © 2014 by Inktank | Private and Confidential
    7. 7. INTRO TO CEPH
    8. 8. HISTORICAL TIMELINE Copyright © 2013 by Inktank | Private and Confidential 8 RHEL-OSP & RHEV Support FEB 2014 MAY 2012 Launch of Inktank OpenStack Integration 2011 2010 Mainline Linux Kernel Open Source 2006 2004 Project Starts at UCSC Production Ready Ceph SEPT 2012 2012 CloudStack Integration OCT 2013 Inktank Ceph Enterprise Launch Xen Integration 2013
    9. 9. ARCHITECTURAL COMPONENTS 9 Copyright © 2014 by Inktank | Private and Confidential APP HOST/VM CLIENT
    10. 10. CEPH UNIFIED STORAGE FILE SYSTEM BLOCK STORAGE OBJECT STORAGE Keystone Geo-Replication Native API 10 Multi-tenant S3 & Swift OpenStack Linux Kernel iSCSI Clones Snapshots CIFS/NFS HDFS Distributed Metadata Linux Kernel POSIX Copyright © 2014 by Inktank | Private and Confidential
    11. 11. TRADITIONAL STORAGE VS. CEPH TRADITIONAL ENTERPRISE STORAGE 11 Copyright © 2014 by Inktank | Private and Confidential
    12. 12. CEPH WITH OPENSTACK 12 Copyright © 2014 by Inktank | Private and Confidential
    13. 13. WEB APPLICATION STORAGE 13 Copyright © 2014 by Inktank | Private and Confidential S3/Swift S3/Swift S3/Swift S3/Swift
    14. 14. MULTI-SITE OBJECT STORAGE 14 Copyright © 2014 by Inktank | Private and Confidential
    15. 15. Copyright © 2014 by Inktank | Private and Confidential ARCHITECTURE
    16. 16. ARCHITECTURAL COMPONENTS 16 Copyright © 2014 by Inktank | Private and Confidential APP HOST/VM CLIENT
    17. 17. 1 DISK FS DISK DISK OSD DISK DISK OSD OSD OSD OSD FS FS FSFS btrfs xfs ext4 MMM 17
    18. 18. 1 M M M HUMAN 18
    19. 19. 1 Monitors: Maintain cluster map Provide consensus for distributed decision-making Must have an odd number These do not serve stored objects to clients M OSDs: •One per disk (recommended) •At least three in a cluster •Serve stored objects to clients •Intelligently peer to perform replication tasks •Supports object classes 19
    20. 20. 2 M M M CLIENT 01 10 data metadata 20
    21. 21. 2 Metadata Server Manages metadata for a POSIX- compliant shared filesystem Directory hierarchy File metadata (owner, timestamps, mode, etc.) Stores metadata in RADOS Does not serve file data to clients Only required for shared filesystem 21
    22. 22. CRUSH  Pseudo-random placement algorithm  Ensures even distribution  Repeatable, deterministic  Rule-based configuration  Replica count  Infrastructure topology  Weighting CRUSH 22
    23. 23. 2 10 10 01 01 10 10 01 11 01 10 10 10 01 01 10 10 01 11 01 10 hash(object name) % num pg CRUSH(pg, cluster state, rule set) 23
    24. 24. 2 CLIENT ?? 24
    25. 25. 2 25
    26. 26. 2 26
    27. 27. 2 CLIENT ?? 27
    28. 28. Ceph Developer Summit 29 • Recent: “Giant” • March 04-05 • wiki.ceph.com • Virtual (irc, hangout, pad, blueprint, youtube) • 2 days (soon to be 3?) • Discuss all work • Recruit for your projects!
    29. 29. New Contribute Page 30 • http://ceph.com/ community/ Contribute • Source tree • Issues • Share experiences • Standups • One-stop shop
    30. 30. New Ceph Wiki 31
    31. 31.  Accepted as a mentoring organization  8 mentors from Inktank & Community  http://ceph.com/gsoc2014/  Student applications being evaluated  Hope to turn this into academic outreach Google Summer of Code 2014 32
    32. 32. Ceph Days 33 • inktank.com/ cephdays • Recently: London, Frankfurt, NYC, Santa Clara • Aggressive program • Upcoming: Sunnyvale, Austin, Boston, Kuala Lumpur
    33. 33. Meetups 34 • Community organized • World wide • Wiki • Ceph-community • Goodies available • Logistical support • Drinkup to tradeshow
    34. 34.  We haven’t forgotten!  Looking for potential founding members  Especially important to keep the IP clean Ceph Foundation 35
    35. 35. Coordinated Efforts 36 • Always need help • CentOS SIG • OCP • Xen • Hadoop • OpenStack • CloudStack • Ganetti • Many more!
    36. 36. http://metrics.ceph.com 37 Copyright © 2014 by Inktank | Private and Confidential
    37. 37. THE PRODUCT
    38. 38. 39 INKTANK CEPH ENTERPRISE WHAT’S INSIDE? Ceph Object and Ceph Block Calamari Enterprise Plugins (2014) Support Services Copyright © 2014 by Inktank | Private and Confidential
    39. 39. Ceph Inktank Ceph Enterprise Open-Source Ceph ✔ ✔ Object Storage ✔ ✔ Block Storage ✔ ✔ File System ✔ Management API ✔ ✔ Management GUI ✔ Hyper-V Support Q3 2014 SNMP Support Q3 2014 24x7 Support ✔ Bug Prioritization ✔ FEATURE COMPARISON 41 Copyright © 2014 by Inktank | Private and Confidential
    40. 40. ROADMAP INKTANK CEPH ENTERPRISE 42 Copyright © 2014 by Inktank | Private and Confidential April 2014 September 2014 2015
    41. 41. RELEASE SCHEDULE Copyright © 2014 by Inktank | Private and Confidential 43 2013 2014 2015 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2
    42. 42. Copyright © 2014 by Inktank | Private and Confidential QUESTIONS? 44
    43. 43. THANK YOU! Patrick McGarry Director, Community patrick@inktank.co m irc.oftc.net/ceph @scuttlemonkey @ceph
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×