Cinder
Project overview and update
Jay Bryant (Cinder PTL for Train)
Brian Rosmaita (Cinder PTL for Ussuri)
IRC: jungleboyj , rosmaita
Twitter: @jungleboyj , @br14nr
November 2019
What does Cinder do?
• Provide block storage service
• Implement services and
libraries to provide on demand,
self-service access to Block
Storage resources. Provide
Software Defined Block
Storage via abstraction and
automation on top of various
traditional backend block
storage devices.
Project background
• Founded during the Folsom
release of OpenStack
• 150 contributors in Stein (51
companies)
• 115 contributors in Train (39
companies)
Project background
Latest user survey adoption
numbers:
• Deployed in 98% of production
deployments
(out of 463 deployments)
Are we satisfied with 98%? NO!!!
● cinderlib
○ Reuse Cinder drivers in any Python code
■ No API, Scheduler, or Volume services
■ No Keystone, MySQL, or RabbitMQ
○ Used in:
○ “Cycle-trailing” release (since it depends on Cinder)
■ Stein: v0.9.0
■ Train: v1.0.0 on the way
→ Kubernetes: Ember-CSI
→ oVirt: Managed Block Storage
→ Ansible: Storage Role PoC
Mid-Cycle Meeting
● August 21-23 at Morrisville (North Carolina,
USA) Lenovo Site
● Approximately 5 people in Physical
Attendance
● Approximately 8 people remotely
participated
● Was a productive 3 days
Seriously, we do more than just eat at these meetings ----->
Agenda
● The State of Cinder
● Update on Train release
● Priorities for Ussuri
The State of Cinder
Contributions
● Slow decline of commits during the last few releases
● Why?
○ Transition from new feature development to bug fixing and
User Experience improvements
○ Drivers have stabilized and are more reliable
○ Deprioritization of upstream development by some
companies
● Is this good?
○ Yes ... and No
○ Cinder is a more mature and stable offering
○ The software doesn’t stabilize itself!
Participation in Train
Red Hat
22.9%
Dell EMC 14.2%
SUSE 9.5%
Others
21.5%
Lenovo 5.7%
NEC 7.1%
Huawei 5.1%
Drivers
● 59 supported drivers
○ All have third-party CI running in Python 3.7
○ Has remained stable for the last few releases with
about the same number going out as coming in
● 17 unsupported drivers
○ Some of these are unsupported due to their 3rd-party
CI systems not being able to handle Python 3.7
○ Will hopefully get this solved early in the Ussuri cycle
Bottom Line
● Cinder’s participation remains fairly healthy
● With cinderlib, the project has relevance in the
containerized world
○ This is particularly true for the backend drivers, so
hopefully vendors will beef up their support
Train Release Update
New Drivers
● New drivers in Train
○ Infortrend (restored; had been removed in Queens)
○ LINSTOR
○ RackScale Design NVMe-oF
○ Seagate -- FC and iSCSI
Additionally, many current drivers added enhanced
functionality. See the Train release notes for details.
Removed Drivers
● The following drivers, deprecated in Stein, failed to restore
their 3rd party CI during the Train cycle and were removed:
○ Nexenta Edge
○ Veritas HyperScale
○ Tintri
● The DRBDManage driver was removed; it is replaced by
the new LINSTOR driver
Unsupported Drivers
● Some due to lack of interest in keeping 3rd Party CI
running, and some due to unanticipated problems
converting 3rd Party CI to running under Python 3.7
○ See the “Unsupported Drivers” section of the
“Available Drivers” page in the Cinder documentation
○ https://docs.openstack.org/cinder/latest/drivers.html
Multi-Attach
● Several drivers added multi-attach support
○ HPE 3PAR and MSA
○ NEC
○ NexentaStor5 iSCSI and NFS
○ StorPool
Compression of volumes uploaded as images
● Admin-facing
○ uploaded qcow2 images are compressed using the native
qemu-img compression
○ Less data to upload/store, but requires more CPU
○ “On” by default (image_compress_on_upload option)
Compression of volumes uploaded as images
● User-facing
○ support added for hardware accelerated compression
○ User selects ‘compressed’ container format for the image
○ Has a software fallback if a HW accelerator is not configured
○ “Off” by default (allow_compression_on_image_upload
option)
○ See the Train release notes and “Accelerate image
compression” in the Cinder Administration Guide
No Untyped Volumes
● It’s now impossible to have untyped volumes
● There is a default volume type cleverly named __DEFAULT__
● It is assigned when:
○ A new volume is created without a type, and
○ The default_volume_type option is unset in cinder.conf
Upgrade Checks
● Allow administrators to check their environment to ensure
compatibility with the new Cinder release
● cinder-status upgrade check
● Upgrade-to-Stein checks were included in Cinder 14.0.1 (the
first Stein update release)
● Upgrade-to-Train checks are included in Cinder 15.0.0
There was a forum session about this yesterday -- if you missed it,
see the etherpad:
https://etherpad.openstack.org/p/shanghai-forum-upgrade-checker
Priorities for Ussuri
New features & enhancements
planned for Ussuri
● A reminder that this is just a statement of plan … actual
mileage may vary.
● Priorities will be discussed at the Ussuri PTG later this week.
○ https://etherpad.openstack.org/p/shanghai-ptg-cinder
● What follows is a short list of topics off the top of my head
Theme: Stability
● Want to increase automated test coverage to handle
scenarios not currently covered by unit, functional, or tempest
tests
○ Use the cinder-tempest-plugin to use the tempest
framework to do more thorough testing
○ Should be an easy integration for third party CI systems
to run these more thorough tests as well
OSSN-0085
● Applies to Ceph backend, but only when the rbd_keyring_conf
option is set
○ Option is unset by default
○ Vulnerability is: Ceph credentials can be leaked
○ Mitigation is: do not use the option
○ Option is deprecated in Ussuri for removal in “V”
○ Migration path: none
■ Contact me if you have a use case for this functionality
● https://wiki.openstack.org/wiki/OSSN/OSSN-0085
Driver Capabilities Reporting
● Not currently easy to see capabilities reported by drivers
enabled in an environment
● Working to make the information more readily available and
usable
More default volume types
● Having a single volume type default is too restrictive for
bigger clouds with multiple AZs and many tenants/projects
● There are use cases for per-project default volume types
○ There are some inelegant workarounds, we’d like to
enable a better user experience
Removal of V2 API
● V3 is a supserset of V2. Would like to remove duplicate V2
code
● Working with API consumers to determine possible impacts
● Did not happen in Train -- maybe in Ussuri?
Reference Links
● Release notes
○ https://docs.openstack.org/releasenotes/cinder/train.html
● Launchpad
○ https://launchpad.net/cinder
● Cinder wiki
○ https://wiki.openstack.org/wiki/Cinder
● Cinder YouTube
○ https://www.youtube.com/channel/UCJ8Koy4gsISMy0qW3CWZmaQ
Moar contributors!
● Everyone in this room can be a contributor
● “10 ways to contribute to an open source project without
writing code”
○ A 2013 article by Heiko W. Rupp, but still very relevant
○ http://tiny.cc/10-ways
@OpenSta
ck
THANKS.
Questions?
openstack openstack OpenStackFoundation

OpenStack Cinder Project Update - Shanghai 2019

  • 1.
    Cinder Project overview andupdate Jay Bryant (Cinder PTL for Train) Brian Rosmaita (Cinder PTL for Ussuri) IRC: jungleboyj , rosmaita Twitter: @jungleboyj , @br14nr November 2019
  • 2.
    What does Cinderdo? • Provide block storage service • Implement services and libraries to provide on demand, self-service access to Block Storage resources. Provide Software Defined Block Storage via abstraction and automation on top of various traditional backend block storage devices.
  • 3.
    Project background • Foundedduring the Folsom release of OpenStack • 150 contributors in Stein (51 companies) • 115 contributors in Train (39 companies)
  • 4.
    Project background Latest usersurvey adoption numbers: • Deployed in 98% of production deployments (out of 463 deployments)
  • 5.
    Are we satisfiedwith 98%? NO!!! ● cinderlib ○ Reuse Cinder drivers in any Python code ■ No API, Scheduler, or Volume services ■ No Keystone, MySQL, or RabbitMQ ○ Used in: ○ “Cycle-trailing” release (since it depends on Cinder) ■ Stein: v0.9.0 ■ Train: v1.0.0 on the way → Kubernetes: Ember-CSI → oVirt: Managed Block Storage → Ansible: Storage Role PoC
  • 6.
    Mid-Cycle Meeting ● August21-23 at Morrisville (North Carolina, USA) Lenovo Site ● Approximately 5 people in Physical Attendance ● Approximately 8 people remotely participated ● Was a productive 3 days Seriously, we do more than just eat at these meetings ----->
  • 7.
    Agenda ● The Stateof Cinder ● Update on Train release ● Priorities for Ussuri
  • 8.
  • 9.
    Contributions ● Slow declineof commits during the last few releases ● Why? ○ Transition from new feature development to bug fixing and User Experience improvements ○ Drivers have stabilized and are more reliable ○ Deprioritization of upstream development by some companies ● Is this good? ○ Yes ... and No ○ Cinder is a more mature and stable offering ○ The software doesn’t stabilize itself!
  • 10.
    Participation in Train RedHat 22.9% Dell EMC 14.2% SUSE 9.5% Others 21.5% Lenovo 5.7% NEC 7.1% Huawei 5.1%
  • 11.
    Drivers ● 59 supporteddrivers ○ All have third-party CI running in Python 3.7 ○ Has remained stable for the last few releases with about the same number going out as coming in ● 17 unsupported drivers ○ Some of these are unsupported due to their 3rd-party CI systems not being able to handle Python 3.7 ○ Will hopefully get this solved early in the Ussuri cycle
  • 12.
    Bottom Line ● Cinder’sparticipation remains fairly healthy ● With cinderlib, the project has relevance in the containerized world ○ This is particularly true for the backend drivers, so hopefully vendors will beef up their support
  • 13.
  • 14.
    New Drivers ● Newdrivers in Train ○ Infortrend (restored; had been removed in Queens) ○ LINSTOR ○ RackScale Design NVMe-oF ○ Seagate -- FC and iSCSI Additionally, many current drivers added enhanced functionality. See the Train release notes for details.
  • 15.
    Removed Drivers ● Thefollowing drivers, deprecated in Stein, failed to restore their 3rd party CI during the Train cycle and were removed: ○ Nexenta Edge ○ Veritas HyperScale ○ Tintri ● The DRBDManage driver was removed; it is replaced by the new LINSTOR driver
  • 16.
    Unsupported Drivers ● Somedue to lack of interest in keeping 3rd Party CI running, and some due to unanticipated problems converting 3rd Party CI to running under Python 3.7 ○ See the “Unsupported Drivers” section of the “Available Drivers” page in the Cinder documentation ○ https://docs.openstack.org/cinder/latest/drivers.html
  • 17.
    Multi-Attach ● Several driversadded multi-attach support ○ HPE 3PAR and MSA ○ NEC ○ NexentaStor5 iSCSI and NFS ○ StorPool
  • 18.
    Compression of volumesuploaded as images ● Admin-facing ○ uploaded qcow2 images are compressed using the native qemu-img compression ○ Less data to upload/store, but requires more CPU ○ “On” by default (image_compress_on_upload option)
  • 19.
    Compression of volumesuploaded as images ● User-facing ○ support added for hardware accelerated compression ○ User selects ‘compressed’ container format for the image ○ Has a software fallback if a HW accelerator is not configured ○ “Off” by default (allow_compression_on_image_upload option) ○ See the Train release notes and “Accelerate image compression” in the Cinder Administration Guide
  • 20.
    No Untyped Volumes ●It’s now impossible to have untyped volumes ● There is a default volume type cleverly named __DEFAULT__ ● It is assigned when: ○ A new volume is created without a type, and ○ The default_volume_type option is unset in cinder.conf
  • 21.
    Upgrade Checks ● Allowadministrators to check their environment to ensure compatibility with the new Cinder release ● cinder-status upgrade check ● Upgrade-to-Stein checks were included in Cinder 14.0.1 (the first Stein update release) ● Upgrade-to-Train checks are included in Cinder 15.0.0 There was a forum session about this yesterday -- if you missed it, see the etherpad: https://etherpad.openstack.org/p/shanghai-forum-upgrade-checker
  • 22.
  • 23.
    New features &enhancements planned for Ussuri ● A reminder that this is just a statement of plan … actual mileage may vary. ● Priorities will be discussed at the Ussuri PTG later this week. ○ https://etherpad.openstack.org/p/shanghai-ptg-cinder ● What follows is a short list of topics off the top of my head
  • 24.
    Theme: Stability ● Wantto increase automated test coverage to handle scenarios not currently covered by unit, functional, or tempest tests ○ Use the cinder-tempest-plugin to use the tempest framework to do more thorough testing ○ Should be an easy integration for third party CI systems to run these more thorough tests as well
  • 25.
    OSSN-0085 ● Applies toCeph backend, but only when the rbd_keyring_conf option is set ○ Option is unset by default ○ Vulnerability is: Ceph credentials can be leaked ○ Mitigation is: do not use the option ○ Option is deprecated in Ussuri for removal in “V” ○ Migration path: none ■ Contact me if you have a use case for this functionality ● https://wiki.openstack.org/wiki/OSSN/OSSN-0085
  • 26.
    Driver Capabilities Reporting ●Not currently easy to see capabilities reported by drivers enabled in an environment ● Working to make the information more readily available and usable
  • 27.
    More default volumetypes ● Having a single volume type default is too restrictive for bigger clouds with multiple AZs and many tenants/projects ● There are use cases for per-project default volume types ○ There are some inelegant workarounds, we’d like to enable a better user experience
  • 28.
    Removal of V2API ● V3 is a supserset of V2. Would like to remove duplicate V2 code ● Working with API consumers to determine possible impacts ● Did not happen in Train -- maybe in Ussuri?
  • 29.
    Reference Links ● Releasenotes ○ https://docs.openstack.org/releasenotes/cinder/train.html ● Launchpad ○ https://launchpad.net/cinder ● Cinder wiki ○ https://wiki.openstack.org/wiki/Cinder ● Cinder YouTube ○ https://www.youtube.com/channel/UCJ8Koy4gsISMy0qW3CWZmaQ
  • 30.
    Moar contributors! ● Everyonein this room can be a contributor ● “10 ways to contribute to an open source project without writing code” ○ A 2013 article by Heiko W. Rupp, but still very relevant ○ http://tiny.cc/10-ways
  • 31.