Webinar - DreamObjects/Ceph Case Study
Upcoming SlideShare
Loading in...5
×
 

Webinar - DreamObjects/Ceph Case Study

on

  • 2,254 views

Slides from our DreamObjects/Ceph webinar. This webinar was guest hosted by Kyle Bader of DreamHost.

Slides from our DreamObjects/Ceph webinar. This webinar was guest hosted by Kyle Bader of DreamHost.

Statistics

Views

Total Views
2,254
Views on SlideShare
1,914
Embed Views
340

Actions

Likes
2
Downloads
44
Comments
0

3 Embeds 340

http://www.inktank.com 219
http://dev.gluesys.com 118
https://twitter.com 3

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Webinar - DreamObjects/Ceph Case Study Webinar - DreamObjects/Ceph Case Study Presentation Transcript

  • DreamObjectsCloud Object Storage Powered by Ceph
  • About Me•  Senior systems engineer at dreamhost•  Been with dreamhost since 2007•  Systems lead for DreamObjects•  Excited about storage, distributed systems, networking, linux internals, automation and monitoring•  I built my first ceph cluster in april, 2010twitter: @mmgaggle
  • Anatomy of DreamObjects
  • Nuts and Bolts - Storage Nodes•  Intel nehalem or AMD lisbon•  32 GB of ram•  8-12 enterprise 3.5" sas disks•  LSI megaraid 1078/2108•  5x Gb nics•  IPMI 2.0 BMC
  • Nuts and Bolts - Support Nodes•  Intel westmere-ep•  8 GB ram•  3x Gb nics•  4x SAS drives•  IPMI 2.0 BMCceph monitors, radosgw,proxies, object sync
  • Tubes•  IPv6 native•  Dual-stack application proxy / balancer•  Application, rados, and cluster networks•  One gig links, bonded•  LLDP for discovery
  • Traffic•  IPv6 and IPv4 api endpoints•  IPv6 to radosgw nodes•  Stud for ssl termination•  HAproxy for load balancing•  10Gb backend and frontend
  • Ceph Numbers•  Two plus petabytes of raw storage•  Triplicate replication•  900+ OSDs•  7 nines durability•  Single datacenter
  • CRUSH Configuration•  Row, rack, node, and disk fault tolerant•  Weighted, even distribution of data•  Object striping•  Online expansion and contraction
  • OSD Configuration•  Single disk RAID0•  Two partitions per RAID0, data + journal•  Writeback enabled•  Disk cache disabled•  Using XFS for filesystem
  • Application Tuning•  No keepalive on apache•  Increase apache thread pool•  Increase radosgw thread pool•  Raise radosgw ulimits
  • Argonaut•  Lots of failure testing•  Tested on large clusters (~900 osds)•  Weight osds in/out of the cluster
  • Bobtail•  Tunables for less aggressive remapping•  CNAME support•  OSD performance ++
  • Control and Management•  chef•  pdsh•  runit/upstart•  jenkins•  sensu•  collectd•  graphite•  logstash
  • Chef•  Bootstrap cluster, new nodes•  Packages and configuration•  Creates, destroys and encrypts osds•  Search node metadata, restful api•  Roles mapped to pdsh genders•  User and ssh key management•  Loads more..
  • pdshParallel distributed shell• Ceph monitors are a natural control point• pdsh from monitor to other nodes• ssh-agent + ssh agent forwarding• Works when chef api doesnt
  • runit/upstartKeep ceph daemons running• ceph-osd• ceph-mon• radosgw
  • Jenkins•  Continuous api functional tests•  Monitor chef convergence•  Generates alerts on failures
  • Sensu•  Relatively new monitoring system•  Open source•  Written in ruby•  State stored in Redis•  Pub-sub with local agent•  Handlers monitor state and generate alerts
  • collectdSystem metrics• cpu• memory• disks• entropy• interfaces
  • graphite•  bucky - collectd, statsd, metricsd api•  carbon cache•  whisper for metric storage (sequential writes)•  graphite webui
  • logstash•  lumberjack agent to ship logs to logstash•  logstash graphite output•  logstash elasticsearch output•  elasticsearch restful api•  kibana webui sits on top of elasticsearch
  • Future•  East coast region•  Pool replication to another datacenter•  Object expiration, root site support•  Content delivery network..not necessarily in that order ;)
  • Thanks Questions? kyle.bader [at] dreamhost.com @mmgaggle
  • Inktank’s Professional ServicesConsulting Services: •  Technical Overview •  Infrastructure Assessment •  Proof of Concept •  Implementation Support •  Performance TuningSupport Subscriptions: •  Pre-Production Support •  Production SupportA full description of our services can be found at the following:Consulting Services: http://www.inktank.com/consulting-services/Support Subscriptions: http://www.inktank.com/support-services/
  • Check out our upcoming and ondemand webinars from InktankUpcoming: Advanced Features of Ceph Distributed Storage (delivered by Sage Weil, creator of Ceph) February 12, 2013 10:00AM PT, 12:00PM CT, 1:00PM ET http://www.inktank.com/news-events/webinars/On Demand: Getting Started with Ceph http://www.inktank.com/news-events/webinars/ Intro to Ceph with OpenStack http://www.inktank.com/news-events/webinars/
  • Contact InktankInfo@inktank.com and 1-855-INKTANKDon’t forget to follow us on:Twitter: https://twitter.com/inktankFacebook: http://www.facebook.com/inktankYouTube: http://www.youtube.com/inktankstorage
  • THANK YOU!