Ceph and Storage Management
with openATTIC
solutions.hamburg 2016, Hamburg, Germany
2016-09-09
Lenz Grimmer <lenz@openattic.org>
2
openATTIC – Our Vision
 Develop an open source alternative to proprietary
storage management systems
 „Traditional” unified storage (NAS/SAN)
 Support Ceph for scale-out scenarios
 Backed with commercial support and services
3
openATTIC – Notable Recent Changes
 Removed Enterprise/Community Edition split
 Now fully under the GPLv2
 Removed requirement for CLA
 DCA (Signed-off-by) is all that's needed
 Public Jira Bug Tracker (Issue tracking & roadmap)
 Public pull requests / code reviews on BitBucket
 Development/Release Branch
 Monthly releases / nighly snapshot builds
 Entire code base (Backend/WebUI/Test/Docs) in one branch
4
openATTIC – What Sets us Apart?
 Focus on data center storage management
 Support both SAN and NAS functionality without limitations
 Fully Open Source (GPLv2)
 No arbitrary functional restrictions
 Low entrance barrier for adoption
 Based on Linux / OSS tools
 Multiple Linux distributions (Debian/Ubuntu/Red Hat/SUSE)
 Well-established technology stack (e.g. drivers, hardware support)
 Broad user base
5
 Modern Web UI
 RESTful API (Software-
Defined Storage)
 Unified Storage
 NAS (NFS, CIFS, HTTP)
 SAN (iSCSI, Fibre Channel)
 LVM, XFS, ZFS, Btrfs,
ext3/4
 Volume mirroring (DRBD®)
 Multi-node support
 Monitoring (Nagios/Icinga)
built-in
 Ceph management and
monitoring (WIP)
 Development sponsored
by it-novum
openATTIC – Open Source Storage Management
6
 Backend
 Python (Django)
 Django REST Framework
(RESTful API)
 Linux tools for storage
management, e.g. LVM, LIO,
filesystem utilities, DRBD,
etc.
 Nagios/Icinga &
PNP4Nagios (Monitoring
and Graphing)
 Web Frontend
 AngularJS (JS framework)
 Bootstrap (HTML, CSS, and
JS framework)
 Uses REST API exclusively
 Automated Test Suites
 Python Unit Tests
 REST Backend Tests
(Gatling)
 WebUI Tests
(Protractor/Jasmine)
openATTIC – Components
7
openATTIC – High Level Architecture
8
openATTIC – Storage Development Roadmap
 Add Disk and Storage Pool Management to the API & WebUI
 Creating/Modifying LVM Volume Groups / MD RAID setups
 Creating/Modifying Btrfs/ZFS Pools (incl. RAID setups)
 Automatic discovery of disks/pools (via udev)
 Monitoring Disk health (SMART)
 Manage HW RAID controllers
 Add DRBD volume mirroring support to the WebUI
 Extend SAN functionality (more iSCSI/FC features)
 Public Roadmap on the openATTIC Jira/Wiki to solicit
community feedback
9
openATTIC – Current Ceph Development Status
 Ceph Cluster Status Dashboard incl. Performance Graphs
 “NoDB” and “TaskQueue” backend architecture in place
 Pool management (view/create/delete)
 OSD management (view)
 RBD management (view/create/delete/map)
 Cluster health/performance & pool monitoring
 CRUSH map editor
 Support for managing multiple Ceph clusters
10
https://wiki.openattic.org/display/OP/openATTIC+Ceph+REST+API+overview
openATTIC – Ceph Management REST API
11
openATTIC – Ceph Development Roadmap
 Extend Pool Management
 OSD Monitoring/Management
 RBD Management/Monitoring
 CephFS Management
 RGW Management (users, buckets keys)
 Deployment, remote configuration of Ceph nodes (via Salt
Open)
 Remote node monitoring (via collectd)
 Public Roadmap on the openATTIC Wiki: http://bit.ly/28PCTWf
12
openATTIC – Storage Management Dashboard
13
openATTIC – Storage Dashboard Preview
14
openATTIC – Volume Management
15
openATTIC – Volume Utilization
16
openATTIC – Volume Performance
17
openATTIC – API Recorder
18
openATTIC – Ceph Cluster Dashboard Preview
19
openATTIC – Ceph Pool List
20
openATTIC – Ceph OSD List
21
openATTIC – Ceph RBD list
22
openATTIC – Create Ceph RBD
23
openATTIC – Ceph CRUSH map editing
24
 www.openattic.org
 demo.openattic.org
 blog.openattic.org
 docs.openattic.org
 bitbucket.org/openattic
 tracker.openattic.org
 Twitter: @openATTIC
 G+: openATTIC
openATTIC – Resources
powered by it-novum
Questions / Discussion
powered by it-novum
Thank you!

Ceph and Storage Management in openATTIC - solutions.hamburg - 2016-09-09

  • 1.
    Ceph and StorageManagement with openATTIC solutions.hamburg 2016, Hamburg, Germany 2016-09-09 Lenz Grimmer <lenz@openattic.org>
  • 2.
    2 openATTIC – OurVision  Develop an open source alternative to proprietary storage management systems  „Traditional” unified storage (NAS/SAN)  Support Ceph for scale-out scenarios  Backed with commercial support and services
  • 3.
    3 openATTIC – NotableRecent Changes  Removed Enterprise/Community Edition split  Now fully under the GPLv2  Removed requirement for CLA  DCA (Signed-off-by) is all that's needed  Public Jira Bug Tracker (Issue tracking & roadmap)  Public pull requests / code reviews on BitBucket  Development/Release Branch  Monthly releases / nighly snapshot builds  Entire code base (Backend/WebUI/Test/Docs) in one branch
  • 4.
    4 openATTIC – WhatSets us Apart?  Focus on data center storage management  Support both SAN and NAS functionality without limitations  Fully Open Source (GPLv2)  No arbitrary functional restrictions  Low entrance barrier for adoption  Based on Linux / OSS tools  Multiple Linux distributions (Debian/Ubuntu/Red Hat/SUSE)  Well-established technology stack (e.g. drivers, hardware support)  Broad user base
  • 5.
    5  Modern WebUI  RESTful API (Software- Defined Storage)  Unified Storage  NAS (NFS, CIFS, HTTP)  SAN (iSCSI, Fibre Channel)  LVM, XFS, ZFS, Btrfs, ext3/4  Volume mirroring (DRBD®)  Multi-node support  Monitoring (Nagios/Icinga) built-in  Ceph management and monitoring (WIP)  Development sponsored by it-novum openATTIC – Open Source Storage Management
  • 6.
    6  Backend  Python(Django)  Django REST Framework (RESTful API)  Linux tools for storage management, e.g. LVM, LIO, filesystem utilities, DRBD, etc.  Nagios/Icinga & PNP4Nagios (Monitoring and Graphing)  Web Frontend  AngularJS (JS framework)  Bootstrap (HTML, CSS, and JS framework)  Uses REST API exclusively  Automated Test Suites  Python Unit Tests  REST Backend Tests (Gatling)  WebUI Tests (Protractor/Jasmine) openATTIC – Components
  • 7.
    7 openATTIC – HighLevel Architecture
  • 8.
    8 openATTIC – StorageDevelopment Roadmap  Add Disk and Storage Pool Management to the API & WebUI  Creating/Modifying LVM Volume Groups / MD RAID setups  Creating/Modifying Btrfs/ZFS Pools (incl. RAID setups)  Automatic discovery of disks/pools (via udev)  Monitoring Disk health (SMART)  Manage HW RAID controllers  Add DRBD volume mirroring support to the WebUI  Extend SAN functionality (more iSCSI/FC features)  Public Roadmap on the openATTIC Jira/Wiki to solicit community feedback
  • 9.
    9 openATTIC – CurrentCeph Development Status  Ceph Cluster Status Dashboard incl. Performance Graphs  “NoDB” and “TaskQueue” backend architecture in place  Pool management (view/create/delete)  OSD management (view)  RBD management (view/create/delete/map)  Cluster health/performance & pool monitoring  CRUSH map editor  Support for managing multiple Ceph clusters
  • 10.
  • 11.
    11 openATTIC – CephDevelopment Roadmap  Extend Pool Management  OSD Monitoring/Management  RBD Management/Monitoring  CephFS Management  RGW Management (users, buckets keys)  Deployment, remote configuration of Ceph nodes (via Salt Open)  Remote node monitoring (via collectd)  Public Roadmap on the openATTIC Wiki: http://bit.ly/28PCTWf
  • 12.
    12 openATTIC – StorageManagement Dashboard
  • 13.
    13 openATTIC – StorageDashboard Preview
  • 14.
  • 15.
  • 16.
  • 17.
  • 18.
    18 openATTIC – CephCluster Dashboard Preview
  • 19.
  • 20.
  • 21.
  • 22.
  • 23.
    23 openATTIC – CephCRUSH map editing
  • 24.
    24  www.openattic.org  demo.openattic.org blog.openattic.org  docs.openattic.org  bitbucket.org/openattic  tracker.openattic.org  Twitter: @openATTIC  G+: openATTIC openATTIC – Resources
  • 25.
  • 26.