Presented by Akshai Parthasarathy, Sean Cohen, and Tom Bechtold about file-share services (Manila) in OpenStack

1. Manila An Update from Liberty
October 29, 2015
OpenStack Summit -
Tokyo
Akshai Parthasarathy, NetApp
Sean Cohen, Red Hat
Thomas Bechtold, SUSE

2. Akshai Parthasarathy
Technical Marketing
Engineer, NetApp
@akshai
Sean Cohen
Principal Product Manager,
Red Hat
@SeanCohen_RH
Thomas Bechtold
OpenStack Cloud Engineer,
SUSE
Today's Presenters

3. Agenda
• What is Manila?
• Why use Manila?
• Use Cases
• Sahara
• Containers
• Liberty Updates
• Distributions Integration
• SUSE Demo
• RedHat Demo
• Upcoming in Mitaka
• Share Replication Demo
• Q+A

4. Manila: The OpenStack Shared File Service Program
Bringing self-service, shared file services
to the cloud

5. Manila History
 Beginnings
 Juno
 Incubated Project
 Community Inception
 Puppet Support
 Share Servers
 Tempest Integration
 Kilo
 Driver Modes
 DevStack Plug-in
 Storage Pools
 Default Share Type
 Manage/Unmanage
 Manila UI

6. Manila Today -
 Production Ready
 Number of Drivers: 14
 Blue Prints Completed: 51
 Major Blue Prints:
 Share Instances
 REST API Microversions
 Experimental APIs
 Extend & Shrink
 Consistency Groups (CGs)
 Share Migrations
 etc.

7. Manila Deployment Options and Benefits
 Single Storage Virtual Machine (SVM) / Multi SVM
§ driver_handles_share_servers=False/True
 Benefit
 Network Plugins
 Standalone Network Plugin
 Nova Network Plugin
 Neutron Network Plugin
 Benefit

8. Contributions/New Entrants
Welcome and Thank You:
● CloudBase Solutions
● Fujitsu
● Scality
● NEC
● NTT
● Letv Cloud Computing
● and others
Source:
http://stackalytics.com/?module=
manila&metric=loc&release=liber
tyCode Contributions Reviews

9. Why Use Manila?
 Shared File Services
 Spending
 Total Capacity
 IDC: Age of Cloud File Services
 Why not?

10. Use Cases

11. Standalone Share File Services Management
 Replace Legacy Systems
 Improve SLA
 Integrate with Automation Frameworks

12. Enterprise Applications
 Lower Cost, Heterogeneous Infrastructure
 Application Porting: Move Workloads to OpenStack

13. DevOps/Continuous Integration
 Pluggable Infrastructure
 Storage Service Catalog
 Analytics
 Containers
 Snapshots & Clones
 Consistency Groups

14. New in Liberty

15. Share Instances
Needed for features like:
● share migrations
● replication

16. REST API Microversions
http://docs.openstack.org/developer/manila/devref/api_microversion_dev.html
http://docs.openstack.org/developer/manila/devref/api_microversion_history.html
Example:
● client 1 with server 1 => version 2.3
● client 1 with server 2 => version 2.3
● client 2 with server 1 => version 2.5
● client 2 with server 2 => version 2.10

17. Experimental APIs
 Expected to change at any time
 can be removed without
deprecation period
 Usage: needs to set header “X-
OpenStack-Manila-API-
Experimental: true”
http://docs.openstack.org/developer/manila/devref/experimental_apis.html
http://developer.openstack.org/api-ref-share-v2.html
@api_version(min_version='2.1', max_version='2.9')
def show(self, req, id):
.... stuff ....
@api_version(min_version="2.4",
experimental=True)
def my_api_method(self, req, id):
.... stuff ....

18. Extend & Shrink
http://docs.openstack.org/developer/manila/devref/share_back_ends_feature_support_mapping.html

19. Consistency Groups (CGs)
 grouping different shares together for the purpose of application data
protection (focus of snapshots for disaster recovery)
 example use case: database data and log files are on different shares
 when doing snapshots for both shares, the data on the shares must be consistent
 supported actions
 create CG with mutliple shares
 create snapshot of CG
 create CG from snapshot

20. External CI For All Drivers
And others ...

21. Oversubscription
 Tunable for setting provisioned
capacity and a subscription ratio
added
 Addresses 'infinite' and 'unknown'
drivers reported capacity that may
lead to oversubscription
 thin_provisoning support needed

22. Share Migrations
 Share Migration allows a share to be
migrated from one host pool to
another hostpool through the "manila
migrate <share> <host#pool>"
command, as well as allowing to
perform migration between different
backends.
 Basic implementation
 The fallback approach for migration is rsync.
Slow, inefficient
 Vendors can utilize API for optimized
migration

23. Availability Zones
 The availability zones support which was
inherited from Cinder was reworked this
cycle:
 Added public API extension
 Allow to preserve AZ if creating a share
from a snapshot and set AZ in Share API
or Share Manager.
 AZs will benefit share replication and also
give end users control of the locality of
their data w.r.t. consumers of the data.

24. Sahara Integration
 Use cases
 Stores binaries for job templates - NFS is ideal for this case
 Input and output data sources - Manila-provisioned HDFS and NFS offers more
options
 Mount NFS share API
 Binaries and data I/O from an NFS share path
 New development in Sahara this cycle
 Mount shares at cluster creation or auto-mount when a share is used for EDP
 Manila-provisioned HDFS
 Data sources and data processing on Sahara-external clusters
 Testing and process verification of extant Manila features this cycle
 Coming soon - NFS Hadoop driver (run jobs on your NFS shares)

25. Sahara Integration - Current Implementations
 Data sources on Manila-provisioned
HDFS
 API to mount NFS shares to clusters (job
binaries and data sources)
Images by Weiting Chen (Intel)

26. Manila + Containers Ceph Example
 Simply mount --bind share into
 container namespace
 NFS re-export from host
 mount and export fs on host
 private host/guest net
 avoid network hop from NFS
service VM
 Host mounts CephFS
 Bind Manila share/volume into
container
 Further integration requires
integration with both Nova and
Manila to manage the
attach/detach process
HOST
M M
RADOS CLUSTER
CONTAINER
MANILA
NATIVE CEPH
CEPH.KO
NOVA

27. Distributions
Integration

28. Manila in SUSE OpenStack Cloud
 The Manila service is tech preview in SUSE Cloud 5
 Fully supported in SUSE OpenStack Cloud 6
 Crowbar deployment tool integration
 Controller HA
 NetApp driver
 Custom driver possible

30. Manila in RHEL OpenStack Platform 7
 The Manila service is tech preview in
RHEL OpenStack Platform 7
 RHEL OpenStack director deployment
tool
 Offer integration with
 GlusterFS native
 Gluster NFS
 NetApp drivers
 Manila Certification program in RHEL
OpenStack Platform 8
 Introducing NFS-Ganesha and Gluster
Automated Volume Management
(based on Heketi)

32. Upcoming
in Mitaka

33. Upcoming in Mitaka
 Mount Automation
 Rolling Updates
 Export Location Metadata
 Manila QoS
 Capability Lists
 Interaction Between New Features
 Share Migration Loose-ends
 Remove All Extensions
 Architectural Directions For New 1st-party Drivers
 Share Replication

34.  Non Disruptive Operations
 High Availability
 Availability Zones
 Failures within an AZ
 High Availability Solution
 Clustered Storage
 Failure of an AZ
 High Availability Solution
 Share Replication
Manila State of the Art: Share Replication

35. Demo #3: Share Replication

37. GET INVOLVED WITH MANILA!
 Manila Resources
 https://github.com/openstack/manila
 https://github.com/openstack/python-manilaclient
 https://github.com/openstack/manila-ui
 https://github.com/openstack/manila-image-elements
 https://wiki.openstack.org/wiki/Manila
 https://launchpad.net/manila
 #openstack-manila on IRC (Freenode)
 Weekly meetings @ Thursday, 15:00 UTC
 NetApp: http://netapp.github.io
 Red Hat: https://www.redhat.com/en/insights/openstack
 Suse: https://www.suse.com/products/suse-cloud/

38. MANILA RELATED SESSIONS
 MANILA GENERAL SESSIONS
 Manila and Sahara: Crossing the Desert to the Big Data Oasis: Tuesday, Oct 27 12:05pm
 OpenStack Manila Hands-on Lab Session: Tuesday, Oct 27 2:00pm
 The State of Ceph, Manila, and Containers in OpenStack: Wednesday, Oct 28 4:40pm
 UPCOMING MANILA SESSION
 Manila contributors meetup: Friday, Oct 30, 9:00am

39. Q&A
October 29, 2015
OpenStack Summit -
Tokyo