www.it-novum.com
© it-novum GmbH
Seite 1
Building a flexible and large-scale software-
defined storage platform for OpenStack
OPENSTACK CEE DAY
May 26th 2014, Budapest
Michael Kienle, CEO
it-novum GmbH
michael.kienle@it-novum.com
www.it-novum.com
© it-novum GmbH
Seite 2
Source: RED HAT
Open Clouds are picking up right now!
www.it-novum.com
© it-novum GmbH
Seite 3
Open Stack: Open cloud operating system supported by the
OpenStack Foundation backed up by ### well-known companies
Want to learn more: listen to the other tracks 
Major Design Guidelines are OPEN:
Open source (apache), Open design (blueprints), Open development
(branches), Open community (tech leads)
Although OpenStack is purposely open it partly relies on underlying
proprietary storage products.
To build your cloud upon a truly OPEN software-defined storage
platform (SDS) you might achieve greater flexibility and $$$$-
savings.
openATTIC is a SDS cloud storage platform optimized for
OpenStack's Cinder component.
What is this all about?
www.it-novum.com
© it-novum GmbH
Seite 4
Introduction & Agenda
OpenStack
Focus on Storage within OpenStack, esp. Cinder
Requirements & Optimization of current approaches
SDS – software-defined storage
How it works
SDS as part of the software-defined datacenter (SDDC)
Why now?
Advantages
openATTIC
How to bring SDS and OpenStack together
Video/Examples
Agenda
www.it-novum.com
© it-novum GmbH
Seite 5
Leading provider of open source business solutions
75 employees currently; approx. 20% growth p.a.
Headquarters in Fulda/DE; Branch office in Vienna/AT
Hundreds of mid-sized and large corporate clients
Operating public listed parent company‘s data center for 20+ years
and migrated most of this to open source
Mission: Help customers to build and run a 100% open-source software-
defined datacenter
Proven Expertise in Business Open Source Infrastructure and
Applications
SAP Extended Business Partner (for integrating SAP into open source
solutions)
Open Business Solutions. Delivered!
www.it-novum.com
© it-novum GmbH
Seite 6
Generic Cloud Drivers: Abstraction, Automation, Self-Service,
efficient use of resource pools, dynamic & massive scale, common
APIs
Manage Virtualization across different Hypervisors (KVM, VMware,
Xen, Hyper-V etc.)
Support the whole IaaS stack (Storage, Security, Networking etc.)
Scale far beyond the limitations of an „old virtualized landscape“
Enable the software-defined datacenter (SDDC)
OpenStack: Value Proposition
www.it-novum.com
© it-novum GmbH
Seite 7
Generic Cloud Drivers: Abstraction, Automation, Self-Service,
efficient use of resource pools, dynamic & massive scale, common
APIs
Manage Virtualization across different Hypervisors (KVM, VMware,
Xen, Hyper-V etc.)
Support the whole IaaS stack (Storage, Security, Networking etc.)
Scale far beyond the limitations of an „old virtualized landscape“
Enable the software-defined datacenter (SDDC)
OpenStack: Value Proposition
www.it-novum.com
© it-novum GmbH
Seite 8
Storage costs optimization ($$$$)
Data growth of >>40% p.a. needs to be managed automatically and way
cheaper
Flexibility and reduced vendor lock-in (open)
Incumbent players are perfect in securing their huge margins
Using innovative technology without renewing existing assets
Cloud Computing has characterized the next generation IT delivery model,
easy orchestration/automation of (storage) services is essential
need scale-up/scale-out of underlying storage at lesser costs
Software-Defined DataCenter (SDDC) becomes a major disruptive IT trend
(SDS beeing part of this)
Standardisation enables automation & abstraction of software from hardware -
> Commodity hardware and SDS gained strength
Customers‘ Perspective on Storage
www.it-novum.com
© it-novum GmbH
Seite 9
Cinder / Block Storage Swift / Object Storage
Objectives • Storage for running VM disk
volumes on a host
• Ideal for performance
sensititve apps
• Ideal for cost effective,
scale-out storage
• Backup & Archive
• Dropbox-like services
Use Cases • Production Applications
• Traditional IT Systems
• Database-driven Apps
• Dev-/Test Landscape
• ISO Image- and VM-
Templates
• Snapshots
• Backup & Archive
• Repository
Workloads • High chance content
• Smaller, Random R/W
• Higher / Bursty IO
• More static content
• Larger, sequential r/w
• Lower IOPS
Storage in OpenStack
www.it-novum.com
© it-novum GmbH
Seite 10
Architecture to provide block-level resources to OpenStack („block storage
as a service“)
Persistent block-level storage volumes for use with OpenStack Nova
compute instances (e.g. iSCSI target)
Manages creation, attach/detach, clone of these storage volumens
Intention of Cinder: keep you from beeing locked-in:
Ability to change the backend (the storage system) easily
Migrate data & expect the same
base level of functionality
Many vendor/storage plug-ins for Cinder available;
the choice between functionality and $$$$
Cinder - Details
Legend
Presistent volume control
Presistent volume data
VM instance
Note that iSCSI is
just an example –
several additional
protocols are
supported (e.g.,
FC, NFS)
Nova
/dev/vda
iSCSI
initiator
Cinder
KVM
iSCSI
target
www.it-novum.com
© it-novum GmbH
Seite 11
Limitations of Commodity Storage (like a simple Linux server or
comparable) because of lacking features like:
node redundancy & clustering & HA, scaling
QoS, Deduplication/Compression, Replication
Example: Image Cloning (e.g. image from Glance) could be
faster than block-based copy&paste…
Limitations of Enterprise Storage (the big names):
Vendor lock-in
$$$$$
Consider Open Source & SDS !
Choose your underlying storage system!
www.it-novum.com
© it-novum GmbH
Seite 12
SDS is the abstraction of the software from the hardware.
Benefits:
Independence from big-name commercial developer of proprietary
systems (bundle of hardware and storage os) for better pricing and
freedom of choice
Hardware is treated as a pool, scalable across multiple storage
systems
Innovative functionality (e.g. top-down policy driven), easier to
administrate, faster to implement (software can be extended and
maintained more easily than hardware)
Part of the Software-defined DataCenter (SDDC)
By 2016 the avail market for OpenSource-Storage is 2,5$B (Gartner,
“Tech Go-to-Market: Storage Providers Should Focus on Three Growth
Opportunities Through 2016”, Nov 2013), by 2018 OpenSource Storage
will gain 20% of the market , approx. 10$B (Gartner, “2014 Strategic
Roadmap for Storage”, March 2014).
Software-defined Storage (SDS)
www.it-novum.com
© it-novum GmbH
Seite 13
Configuration
Management
System
(CMS)
Service Transition Service OperationService Design
Monitoring
Event Correlation
User
Other
Monitors
Capacity
Management
Risk
Management
Cloud
Self Service
System
Application
End-2-End
Process
Change
Management
Problem
Management
SAP
ApplicationsService Desk
Network Operation
Center
Release
Management
Cloud Management
Incident
Management
Infrastructure
Software-defined
Storage
Virtualization
CMDB
Discovery
ReportingServiceLevelManagement
The Big Picture: SDDC with 100% open source
www.it-novum.com
© it-novum GmbH
Seite 14
ITSM
i-doit & OTRS
Umbrella Monitoring
openITCOCKPIT
Cloud Management
OpenStack
Reporting
Pentaho
Billing, Backup,...
SAP, SEP sesam
Configuration
Deployment
Services & Tools
API
Web
CLI
Apps
Status
Checking
Application
Management
Access
Samba, NFS,
AFP, iSCSI, FC
Availability
DRBD, Pacemaker,
Corosync, ...
Efficiency
ZFS, LVM, BTFRS,
XFS
Backup
Snapshotting, Tape-
Backup, ...
Analysis
openITCOCKPIT,
Nagios, ...
Back end
Core
Front end
openATTIC Framework - Architecture
Interfaces
www.it-novum.com
© it-novum GmbH
Seite 15
Mirroring
Block- & File-based
Synchronous & asynchronous
Object-based
Clustering
Automated Failover
Multi Node Support
Support & Services
Training & Optimization
Integration & Development
Guaranteed Response Time
Guaranteed Restore Time
(Appliance only)
Protection
Integration Tape-Systems
Snapshotting
Consistent Snapshotting
Efficiency
Deduplication
Compression
Performance Config-Set
Config-Set for Virtualization
Unified and versatile
SAN / Block Storage
(FC, iSCSI, FCoE)
NAS / File-based (CIFS, NFS)
Object Storage
Central Configuration
Web Frontend
Command Line Interface
Dynamic Monitoring
Availability
Performance
Capacity
Cloud Support
OpenStack/Cinder
openQRM
Umbrella Management
Master/Slave Architecture
Current FeaturesOpen API, Scalable, Cinder-ready
www.it-novum.com
© it-novum GmbH
Seite 16
Volume create/delete
Snapshot create/delete
Volume attach/detach (called by Nova)
Create Volume from Snapshot
Copy Image to Volume / Volume to Image
Get Volume Stats, Clone Volume
Extend Volume (mandatory since Icehouse)
openATTIC Cinder plug-in: Video
We show you some examples how this works…
How it works: Cinder API (Icehouse)
www.it-novum.com
© it-novum GmbH
Seite 17
Unit tests
Cinder driver certification file
Release hopefully next one…
More Info: www.openattic.org/en
Integration in other cloud platforms has been done already
OpenStack Cinder / openAttic: Status
www.it-novum.com
© it-novum GmbH
Seite 18
Open and comprehensive API achivies great flexibility for integration
into different cloud platforms
Optimization of huge data through intelligent openATTIC
functionalities
Consistent snapshots relieve backup, accelerate desaster
recovery and reduce downtimes
Deduplication, Compression, efficient mirroring and redundancy
Huge cost savings:
openATTIC runs on commodity hardware
reducing vendor lock-in
Open Source approach saves license costs
Are there limitations of software-defined storage / software-defined
datacenter?
Summary: openATTIC within OpenStack
www.it-novum.com
© it-novum GmbH
Seite 19
Open and comprehensive API achivies great flexibility for integration
into different cloud platforms
Optimization of huge data through intelligent openATTIC
functionalities
Consistent snapshots relieve backup, accelerate desaster
recovery and reduce downtimes
Deduplication, Compression, efficient mirroring and redundancy
Huge cost savings:
openATTIC runs on commodity hardware
reducing vendor lock-in
Open Source approach saves license costs
However, the bad news is: even software-defined datacenters still
need some underlying hardware… 
Summary: openATTIC within OpenStack
www.it-novum.com
© it-novum GmbH
Seite 20
Thank you!
Questions? - More than welcome!
Cloud
Storage
Intelligent
SDS
Unified
Storage
Power your cloud Rock-solid storage Scale out data center
Power your cloud & scale out your data center with rock-solid Software-
Defined Storage (SDS) truly on open source, join the community & more
about the openATTIC project at http://openattic.org/en

Building an open source cloud storage platform for OpenStack - openATTIC

  • 1.
    www.it-novum.com © it-novum GmbH Seite1 Building a flexible and large-scale software- defined storage platform for OpenStack OPENSTACK CEE DAY May 26th 2014, Budapest Michael Kienle, CEO it-novum GmbH michael.kienle@it-novum.com
  • 2.
    www.it-novum.com © it-novum GmbH Seite2 Source: RED HAT Open Clouds are picking up right now!
  • 3.
    www.it-novum.com © it-novum GmbH Seite3 Open Stack: Open cloud operating system supported by the OpenStack Foundation backed up by ### well-known companies Want to learn more: listen to the other tracks  Major Design Guidelines are OPEN: Open source (apache), Open design (blueprints), Open development (branches), Open community (tech leads) Although OpenStack is purposely open it partly relies on underlying proprietary storage products. To build your cloud upon a truly OPEN software-defined storage platform (SDS) you might achieve greater flexibility and $$$$- savings. openATTIC is a SDS cloud storage platform optimized for OpenStack's Cinder component. What is this all about?
  • 4.
    www.it-novum.com © it-novum GmbH Seite4 Introduction & Agenda OpenStack Focus on Storage within OpenStack, esp. Cinder Requirements & Optimization of current approaches SDS – software-defined storage How it works SDS as part of the software-defined datacenter (SDDC) Why now? Advantages openATTIC How to bring SDS and OpenStack together Video/Examples Agenda
  • 5.
    www.it-novum.com © it-novum GmbH Seite5 Leading provider of open source business solutions 75 employees currently; approx. 20% growth p.a. Headquarters in Fulda/DE; Branch office in Vienna/AT Hundreds of mid-sized and large corporate clients Operating public listed parent company‘s data center for 20+ years and migrated most of this to open source Mission: Help customers to build and run a 100% open-source software- defined datacenter Proven Expertise in Business Open Source Infrastructure and Applications SAP Extended Business Partner (for integrating SAP into open source solutions) Open Business Solutions. Delivered!
  • 6.
    www.it-novum.com © it-novum GmbH Seite6 Generic Cloud Drivers: Abstraction, Automation, Self-Service, efficient use of resource pools, dynamic & massive scale, common APIs Manage Virtualization across different Hypervisors (KVM, VMware, Xen, Hyper-V etc.) Support the whole IaaS stack (Storage, Security, Networking etc.) Scale far beyond the limitations of an „old virtualized landscape“ Enable the software-defined datacenter (SDDC) OpenStack: Value Proposition
  • 7.
    www.it-novum.com © it-novum GmbH Seite7 Generic Cloud Drivers: Abstraction, Automation, Self-Service, efficient use of resource pools, dynamic & massive scale, common APIs Manage Virtualization across different Hypervisors (KVM, VMware, Xen, Hyper-V etc.) Support the whole IaaS stack (Storage, Security, Networking etc.) Scale far beyond the limitations of an „old virtualized landscape“ Enable the software-defined datacenter (SDDC) OpenStack: Value Proposition
  • 8.
    www.it-novum.com © it-novum GmbH Seite8 Storage costs optimization ($$$$) Data growth of >>40% p.a. needs to be managed automatically and way cheaper Flexibility and reduced vendor lock-in (open) Incumbent players are perfect in securing their huge margins Using innovative technology without renewing existing assets Cloud Computing has characterized the next generation IT delivery model, easy orchestration/automation of (storage) services is essential need scale-up/scale-out of underlying storage at lesser costs Software-Defined DataCenter (SDDC) becomes a major disruptive IT trend (SDS beeing part of this) Standardisation enables automation & abstraction of software from hardware - > Commodity hardware and SDS gained strength Customers‘ Perspective on Storage
  • 9.
    www.it-novum.com © it-novum GmbH Seite9 Cinder / Block Storage Swift / Object Storage Objectives • Storage for running VM disk volumes on a host • Ideal for performance sensititve apps • Ideal for cost effective, scale-out storage • Backup & Archive • Dropbox-like services Use Cases • Production Applications • Traditional IT Systems • Database-driven Apps • Dev-/Test Landscape • ISO Image- and VM- Templates • Snapshots • Backup & Archive • Repository Workloads • High chance content • Smaller, Random R/W • Higher / Bursty IO • More static content • Larger, sequential r/w • Lower IOPS Storage in OpenStack
  • 10.
    www.it-novum.com © it-novum GmbH Seite10 Architecture to provide block-level resources to OpenStack („block storage as a service“) Persistent block-level storage volumes for use with OpenStack Nova compute instances (e.g. iSCSI target) Manages creation, attach/detach, clone of these storage volumens Intention of Cinder: keep you from beeing locked-in: Ability to change the backend (the storage system) easily Migrate data & expect the same base level of functionality Many vendor/storage plug-ins for Cinder available; the choice between functionality and $$$$ Cinder - Details Legend Presistent volume control Presistent volume data VM instance Note that iSCSI is just an example – several additional protocols are supported (e.g., FC, NFS) Nova /dev/vda iSCSI initiator Cinder KVM iSCSI target
  • 11.
    www.it-novum.com © it-novum GmbH Seite11 Limitations of Commodity Storage (like a simple Linux server or comparable) because of lacking features like: node redundancy & clustering & HA, scaling QoS, Deduplication/Compression, Replication Example: Image Cloning (e.g. image from Glance) could be faster than block-based copy&paste… Limitations of Enterprise Storage (the big names): Vendor lock-in $$$$$ Consider Open Source & SDS ! Choose your underlying storage system!
  • 12.
    www.it-novum.com © it-novum GmbH Seite12 SDS is the abstraction of the software from the hardware. Benefits: Independence from big-name commercial developer of proprietary systems (bundle of hardware and storage os) for better pricing and freedom of choice Hardware is treated as a pool, scalable across multiple storage systems Innovative functionality (e.g. top-down policy driven), easier to administrate, faster to implement (software can be extended and maintained more easily than hardware) Part of the Software-defined DataCenter (SDDC) By 2016 the avail market for OpenSource-Storage is 2,5$B (Gartner, “Tech Go-to-Market: Storage Providers Should Focus on Three Growth Opportunities Through 2016”, Nov 2013), by 2018 OpenSource Storage will gain 20% of the market , approx. 10$B (Gartner, “2014 Strategic Roadmap for Storage”, March 2014). Software-defined Storage (SDS)
  • 13.
    www.it-novum.com © it-novum GmbH Seite13 Configuration Management System (CMS) Service Transition Service OperationService Design Monitoring Event Correlation User Other Monitors Capacity Management Risk Management Cloud Self Service System Application End-2-End Process Change Management Problem Management SAP ApplicationsService Desk Network Operation Center Release Management Cloud Management Incident Management Infrastructure Software-defined Storage Virtualization CMDB Discovery ReportingServiceLevelManagement The Big Picture: SDDC with 100% open source
  • 14.
    www.it-novum.com © it-novum GmbH Seite14 ITSM i-doit & OTRS Umbrella Monitoring openITCOCKPIT Cloud Management OpenStack Reporting Pentaho Billing, Backup,... SAP, SEP sesam Configuration Deployment Services & Tools API Web CLI Apps Status Checking Application Management Access Samba, NFS, AFP, iSCSI, FC Availability DRBD, Pacemaker, Corosync, ... Efficiency ZFS, LVM, BTFRS, XFS Backup Snapshotting, Tape- Backup, ... Analysis openITCOCKPIT, Nagios, ... Back end Core Front end openATTIC Framework - Architecture Interfaces
  • 15.
    www.it-novum.com © it-novum GmbH Seite15 Mirroring Block- & File-based Synchronous & asynchronous Object-based Clustering Automated Failover Multi Node Support Support & Services Training & Optimization Integration & Development Guaranteed Response Time Guaranteed Restore Time (Appliance only) Protection Integration Tape-Systems Snapshotting Consistent Snapshotting Efficiency Deduplication Compression Performance Config-Set Config-Set for Virtualization Unified and versatile SAN / Block Storage (FC, iSCSI, FCoE) NAS / File-based (CIFS, NFS) Object Storage Central Configuration Web Frontend Command Line Interface Dynamic Monitoring Availability Performance Capacity Cloud Support OpenStack/Cinder openQRM Umbrella Management Master/Slave Architecture Current FeaturesOpen API, Scalable, Cinder-ready
  • 16.
    www.it-novum.com © it-novum GmbH Seite16 Volume create/delete Snapshot create/delete Volume attach/detach (called by Nova) Create Volume from Snapshot Copy Image to Volume / Volume to Image Get Volume Stats, Clone Volume Extend Volume (mandatory since Icehouse) openATTIC Cinder plug-in: Video We show you some examples how this works… How it works: Cinder API (Icehouse)
  • 17.
    www.it-novum.com © it-novum GmbH Seite17 Unit tests Cinder driver certification file Release hopefully next one… More Info: www.openattic.org/en Integration in other cloud platforms has been done already OpenStack Cinder / openAttic: Status
  • 18.
    www.it-novum.com © it-novum GmbH Seite18 Open and comprehensive API achivies great flexibility for integration into different cloud platforms Optimization of huge data through intelligent openATTIC functionalities Consistent snapshots relieve backup, accelerate desaster recovery and reduce downtimes Deduplication, Compression, efficient mirroring and redundancy Huge cost savings: openATTIC runs on commodity hardware reducing vendor lock-in Open Source approach saves license costs Are there limitations of software-defined storage / software-defined datacenter? Summary: openATTIC within OpenStack
  • 19.
    www.it-novum.com © it-novum GmbH Seite19 Open and comprehensive API achivies great flexibility for integration into different cloud platforms Optimization of huge data through intelligent openATTIC functionalities Consistent snapshots relieve backup, accelerate desaster recovery and reduce downtimes Deduplication, Compression, efficient mirroring and redundancy Huge cost savings: openATTIC runs on commodity hardware reducing vendor lock-in Open Source approach saves license costs However, the bad news is: even software-defined datacenters still need some underlying hardware…  Summary: openATTIC within OpenStack
  • 20.
    www.it-novum.com © it-novum GmbH Seite20 Thank you! Questions? - More than welcome! Cloud Storage Intelligent SDS Unified Storage Power your cloud Rock-solid storage Scale out data center Power your cloud & scale out your data center with rock-solid Software- Defined Storage (SDS) truly on open source, join the community & more about the openATTIC project at http://openattic.org/en

Editor's Notes

  • #14 In Anlehnung an das Big Picture des Datacentermanagements wird die Integrationsfähigkeit der vier Open Source Werkzeuge präsentiert.