• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Wicked Easy Ceph Block Storage & OpenStack Deployment with Crowbar

Wicked Easy Ceph Block Storage & OpenStack Deployment with Crowbar



This was co-presented at the OpenStack Summit 2013 in Portland by Kamesh Pemmaraju, Product Manager from Dell and Neil Levine Inktank. ...

This was co-presented at the OpenStack Summit 2013 in Portland by Kamesh Pemmaraju, Product Manager from Dell and Neil Levine Inktank.

Inktank Ceph is a transformational open source storage solution fully integrated into OpenStack providing scalable object and block storage (via Cinder) using commodity servers. The Ceph solution is resilient to failures, uses storage efficiently, and performs well under a variety of VM Workloads.

Dell Crowbar is an open source software framework that can automatically deploy Ceph and OpenStack on bare metal servers in a matter of hours. The Ceph team worked with Dell to create a Ceph barclamp (a crowbar extention) that integrates Glance, Cinder, and Nova-Volume. As a result, it is lot faster and easier to install, configure, and manage a sizable OpenStack and Ceph cluster that is tightly integrated and cost- optimized.

Hear how OpenStack users can address their storage deployment challenges:

Considerations when selecting a cloud storage system
Overview of the Ceph architecture with unique features and benefits
Overview of Dell Crowbar and how it can automate and simplify Ceph/OpenStack deployments Best practices in deploying cloud storage with Ceph and OpenStack



Total Views
Views on SlideShare
Embed Views



4 Embeds 168

http://www.cloudel.com 147
http://www.linkedin.com 17
https://twitter.com 3
https://www.google.com.au 1



Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    Wicked Easy Ceph Block Storage & OpenStack Deployment with Crowbar Wicked Easy Ceph Block Storage & OpenStack Deployment with Crowbar Presentation Transcript

    • Wicked Easy Ceph BlockStorage & OpenStackDeployment withCrowbarKamesh Pemmaraju, DellNeil Levine, Inktank
    • Outline• Introduction– Dell OpenStack-Powered Cloud Solution– Ceph– Crowbar• Why Crowbar + Ceph in OpenStack?– OpenStack Block Storage gap– Automation, scale• What we have done to enable things?• Customer benefits, drivers – an example• What’s coming next?
    • • Clouds demand significant operationaland process controls• Operational decisions drive hardwareand software decisions• We are finding ways to productizeoperations into best practicesHWSWOPSClouds Require an Operational Focus
    • OPSSWHWSWOPS“Crowbar”CloudOpsSoftwareServices &ConsultingReferenceArchitectureDell OpenStack-Powered Cloud Solution
    • Dell OpenStack-Powered Cloud SolutionHW + SW + ServicesHardwareHW referencearchitecture• C6220, C6105,R720, R720XD servers• Storage and compute• Force 10 S60 and PowerConnectConfiguration • Min 6 nodes. Max 60 nodesSoftwareSoftware• OpenStack Installer and continuous integration (Crowbar)• OpenStack cloud SW• Ceph for distributed storageOperating system• Ubuntu 12.04 (host)• Windows (guests)• Linux (guests)Hypervisor • KVMServicesDeployment andconsulting• Hardware integration – onsite or merge center• Software installation – onsite• OpenStack consulting services− Assessment, design and installation− Delivered via partnerships with Mirantis and CanonicalSupport• HW/Crowbar: Dell ProSupport• OpenStack support via Canonical (coming soon)
    • OBJECTS VIRTUAL DISKS FILES & DIRECTORIESCEPHFILE SYSTEMA distributed, scale-outfilesystem with POSIXsemantics that providesstorage for a legacy andmodern applicationsCEPHOBJECT STORAGEA powerful S3- and Swift-compatible gateway thatbrings the power of theCeph Object Store tomodern applicationsCEPHBLOCK STORAGEA distributed virtual blockdevice that delivers high-performance, cost-effectivestorage for virtual machinesand legacy applicationsCeph – Unified Storage Software
    • Ceph – Key Differentiators• Unified storage platform (Object + Block + File)• Ceph Intelligent Placement (CRUSH)• Ceph Intelligent Devices (Self-healing, P2P)• Ceph Intelligent Objects (Embedded Software Classes)• Ceph Integration (Linux Kernel, OpenStack, Cloudstack)
    • Ceph Storage Cluster (RADOS)A reliable, autonomous, distributed object store comprised of self-healing, self-managing, intelligent storage nodesLIBRADOSA library allowingapps to directlyaccess RADOS,with support forC, C++, Java,Python, Ruby,and PHPCeph BlockDevice(RBD)A reliable and fully-distributed blockdevice, with a Linuxkernel client and aQEMU/KVM driverCeph FileSystem(CephFS)A POSIX-compliantdistributed filesystem, with a Linuxkernel client andsupport for FUSECeph ObjectGateway(RGW)A bucket-based RESTgateway, compatiblewith S3 and SwiftAPP APP HOST/VM CLIENTCeph Architecture
    • Ceph Monitor Nodes:– Maintain cluster map– Provide consensus fordistributed decision-making– Must have an odd number– These do not serve storedobjects to clientsM Ceph OSD Nodes:• One Ceph OSD daemon per disk(recommended)• At least three nodes in a cluster• Serve stored objects to clients• Intelligently peer to performreplication tasks• Supports object classesCeph RADOS Components
    • Mission: “A Zero Touch Cloud Installer”Servers in boxes to full function cloud in under 2 hours• Fast & Flexible– Bare metal install including BIOS & RAID config– Users can choose how their system is configured (“barclamps”)• DevOps Embracing– Ongoing Operations Model (DevOps for Clouds)– Leverages & Wraps Opscode Chef• Open– Not specific to OpenStack – Dell using for other Apps– Not restricted to Dell hardware– Apache 2 licensedWhat is Crowbar?
    • Images vs. Layers: OverviewImages: Single Unit Layers: Stacked PiecesIntegrations +Applications +Utilities + OperatingSystemConfigurationOperating SystemUtilitiesApplication BarApplication FooIntegrationsConfiguration
    • Images vs. Layers: LifecycleImages: Replacement Layers: UpgradeI+A+U+O/SConfigOSUBar v1FooIConfigI+A+U+O/SConfigI+A+U+O/SConfigI+A+U+O/SConfigOSUBar v2FooIConfigBar v2
    • Crowbar Uses the Layer Model for DeploymentDell“Crowbar”OpsManagementCore Components &Operating SystemsCloudInfrastructure &Dell IP ExtensionsPhysical ResourcesAPIs, User Access,& EcosystemPartners* Crowbar has potential to service other programs beyond OpenStack• Includes all the components required to implement anentire cloud infrastructure including ecosystems partners• Pluggable components deploy cloud infrastructure• Allow for addition of Dell IP and expansion by thecommunity services and customers• Can integrate with Dell existing products• Delivers basic data center services and required cloudinfrastructure.• Provision bare-metal servers from box to cloud WITHOUTuser intervention (other than racking/cabling and someminimal configuration questions) NOT EXCLUSIVE TO DELLHARDWARE
    • ServerHow Does Crowbar Work?Dell“Crowbar”OpsManagementOperating SystemNetworkConfigChefServerNetworkConfigAppsDHCPCrowbarMagicHappens!OrchestrationState MachineRun ListConfig.Hardware Config(uses “Sledgehammer”)Apps
    • Dell“Crowbar”OpsManagementWhat is a Barclamp?BarclampChef RecipesO/S AppPackagesCrowbar API &Partial UIComponents &Scripts• Crowbar API & Partial UI• Adds states and transitions to Orchestration• Customizes UI specific to barclamp• Visual extensions to nodes/networks/utils• Chef Recipes• DevOps description of application• Enforced repeatable configuration• O/A App Packages• Operating system dependencies (Debs & RPMs)• Components & Scripts• Ad hoc configuration not covered by Chef
    • Modular Design: Barclamps The Configuration YouWant to DeployDell“Crowbar”OpsManagementCore Components &Operating SystemsCloud Infrastructure& Dell IP ExtensionsPhysical ResourcesAPIs, User Access, &Ecosystem PartnersCrowbarDeployerProvisionerNetwork RAIDBIOS IPMINTPDNS LoggingNova Swift GlanceNagios GangliaenStratus RightScaleDashboardCephHorizonKeystoneForce10
    • “We should dothings the newway!”Use ObjectStorage“App can’t bere-architected!”Use local storage!“No SPOFsplease!”Use this expensiveSAN!“How do the Web2.0 folks do it?”Use distributedstorage on utilityhardware!The 4 Stages of Cloud Storage Thinking
    • Ceph Block Storage for Dell OpenstackIntegrated: with Cinder and Nova (via qemu-kvm)Compatible: with GlanceFoundations: no SPOFs, self-healing, CRUSHFeatures: Copy-on-Write, Snapshots, CloningIntegrated: with Crowbar for automation
    • If you can’t automate it, you can’t scale itOSUNovaNodeIConfigOSUNovaAPIIConfigOSUNovaNodeIConfigOSUNovaNodeIConfigOSUNovaNodeIConfigOSUNovaNodeIConfigOSUNovaNodeIConfigOSUNovaDBIConfigOSUNovaCntrlIConfigOSUDashBoardIConfigOSUNagiosIConfigOSUCrowbarChefIConfigOSUCeph MonNodeIConfigOSUCeph MonNodeIConfigOSUCeph MonNodeIConfigOSUCeph OSDNodeIConfig OSUCeph OSDNodeIConfigOSUCeph OSDNodeIConfigOSUCinderAPIIConfigAPI &ServicesComputeNodesStorageNodesScale-Out Operations With OpenStack
    • Ceph Related Barclamps for Crowbar• https://github.com/ceph/barclamp-ceph– ceph.com packages into the Crowbar Admin Node– Ceph OSD roles (ceph-store)– Ceph Mon roles (ceph-mon)• https://github.com/crowbar/barclamp-nova– Hypervisor with RBD (ceph-client)
    • • Dell and Inktank have partnered to bring a complete solution forOpenStack + Ceph + Automated deployment with Crowbar• The joint solution provides:– Crowbar barclamp to– Deploy Ceph clusters automatically in a very short time– Connect those clusters to OpenStack– Professional Services, Support, and Training› Collaborative Support for Dell hardware customers– Joint Solution› Validated against Dell Reference Architectures via theTechnology Partner programWhat Have We Done to Enable?
    • Customer:University to Deploy OpenStack + Ceph Using CrowbarSituation• University employs close to 900 researchers and receiveshundreds of million dollars in Grants as a Top 10 ResearchInstitution primarily working on Cancer and GenomicProjects. The University has a need to provide a centralizeddata repository for Researchers in order to insurecompliances concerning retention of data.• The intent of the Data Repository is to provide 2TB of freestorage space to each Researcher, with opportunity topurchase from the university more capacity at a veryreasonable cost.Decision Drivers• University investigated using an traditional SAN storagesolution which was very expenses on a per TB basis. Theyalso investigated using public cloud storage options whichalso proved to be too expensive. Finally, they looked atHadoop for the project but found that it was not a good fit forthis use case.• In the end, Universitychose Dell/Inktank toarchitect a platformthat would be veryscalable and providelost costs per TB andwas the best of allworlds that providecompute and storageon the samehardware.
    • What’s Next?• May– Ceph Cuttlefish• Summer 2013– Dell Grizzly support– Ceph Dumpling (August)• https://github.com/crowbar/crowbar/wiki/Crowbar-2.0– Crowbar v2.0 work is happening in the open. Check it out andparticipate!
    • Don’t Miss the OpenStack Summit SessionsHavana Sessions:– Planning the Ceph Roadmap for OpenstackWednesday April 17, 20131:50pm - 2:30pm in Room - B113– Features for Ceph with Cinder and BeyondThursday, April 18, 20139:50 – 10:30am in Rooms – C120+121+122
    • Contact InformationReach Kamesh and Neil for additional information:Dell.com/OpenStackDell.com/CrowbarInktank.com/DellKamesh_Pemmaraju@Dell.com@kpemmarajuNeil.Levine@Inktank.com@neilwlevineVisit the Dell and Inktank booths in the OpenStack Summit Expo Hall