Senlin clustering service deep dive on Austin summit. This slide introduces the background, the overall architecture of the senlin project. It also highlights features released in 1.0.0 and features planed for 2.0.0.

1. Senlin Clustering Service Deep Dive
Ethan Lynn, IBM, IRC: elynn
Qiming Teng, IBM, IRC: Qiming
Yanyan Hu, IBM, IRC: yanyanhu

2. Agenda
• Why did we start senlin project
• Senlin overview
• Senlin status and roadmap

3. Why did we start senlin

4. Background
• Why Senlin?
• CNCP (IBM research project)
• Heat AutoScaling Evaluation
• Heat AutoScaling Rework
• Customer Engagement
Gap Analysis
Heat AutoScaling ReworkTalk on Atlanta Summit
Evaluation Report

5. Background
• Why Not Heat?
• Heat AutoScaling today is based on
AWS design, with gaps
• Heat's mission
• Defined in project governance
• It's NOT about autoscaling, NOT about HA,
...
• It's NOT Heat's job to provide new service(s)
To orchestrate composite cloud applications
using a declarative template format through an
OpenStack-native REST API.
Heat Mission Statement
https://github.com/openstack/governance/blob/
master/reference/projects.yaml
• Proposal:
https://wiki.openstack.org/wiki/Heat/AutoScaling
• API Design: http://docs.heatautoscale.apiary.io/
• Targeting at many use cases not covered by Heat
today
AutoScaling: a new design
Heat as a Compiler
YAML
Template
Heat
Nova
Cinder
Neutron
Keystone
• CREATE
• GET
• UPDATE
• DELETE

6. Blueprints on Reworking Heat AutoScaling
BP Priority Description
autoscaling-
api-resources
high Heat resources invoking AS APIs
as-api-group-
resource
high ScalingGroup resource wrapping AS API's
group functionality
as-api-policy-
resource
high ScalingPolicy resource wrapping AS API's
policy functionality
as-api-
webhook-res
high Webhook resource wrapping AS API's
execution of webhooks
autoscaling-
api-client
high A python client for Heat to interact with AS
API
autoscaling-
api
- A separate service for the implementation
of autoscaling w/ Heat
as-engine - A separate engine/service for autoscaling
support AS API
as-engine-db - A DB dedicated to autoscaling, using
schema created in as-lib-db
as-lib - A separate module to be used by the AS
service
as-lib-db - A DB for autoscaling bookkeeping

7. What Do We Really Need?
Scalable
Load-Balanced
Highly-Available
Manageable
......
of any (OpenStack) objects
- What is missing from
OpenStack?
A Clustering Service
- Auto-scaling?
Just one of the usage
scenario of a cluster.
- Auto-Healing (HA)?
Just another usage
scenario.
- We can address the
concerns by making policies
orthogonal

8. Senlin Overview

9. Senlin Architecture
REST RPC
Profiles
Policies

10. Senlin Features
• Profiles: A specification for the objects to be managed
• Policies: Rules to be checked/enforced before/after actions are performed
10
(others)
Senlin
Nova
Docker
Heat
Ironic BareMetal
VMs
Stacks
Containers
placement
deletion
scaling
health
load-balance
affinity
Policies as Plugins Profiles as Plugins Cluster/Nodes Managed

11. Senlin Server Architecture
openstacksdk
identity
compute
orchestration
network
...
engineengine lock
scheduler
actions
nodecluster
service
registry
receiverparser
drivers
openstack
dummy
(others)
dbapi
rpc client
policies
placement
deletion
scaling
health
load-balance
affinity
receiver
webhoook
MsgQueue
extension points
for external
monitoring
services
extension points
facilitating a
smarter cluster
management
extension points to talk to different
endpoints for object CRUD operations
extension points for interfacing
with different services or clouds
profiles
os.heat.stack
(others)
os.nova.server
senlin-api
WSGI
middleware
apiv1

12. Senlin Operations (Actions)
• Cluster
• CREATE
• DELETE
• UPDATE
• LIST
• SHOW
• ADD_NODES
• DEL_NODES
• SCALE_OUT
• SCALE_IN
• RESIZE
• POLICY_ATTACH
• POLICY_DETACH
• POLICY_UPDATE
12
 Node
• CREATE
• DELETE
• UPDATE
• LIST
• SHOW
 Policy
• CREATE
• UPDATE
• DELETE
• LIST
• SHOW
 Profile
• CREATE
• UPDATE
• DELETE
• LIST
• SHOW
 Action
• LIST
• SHOW
Event
• LIST
• SHOW
 Receiver
• CREATE
• DELETE
• LIST
• SHOW

13. profile/cluster/node operations
13
$ senlin profile-create -s <spec> <name>
$ senlin profile-list
$ senlin profile-show <profile>
$ senlin profile-delete <profile>
$ senlin profile-update -n <name> <profile>
$ senlin cluster-create -p <profile> -n <size> <name>
$ senlin cluster-delete <cluster>
$ senlin cluster-list
$ senlin cluster-show <cluster>
$ senlin node-create -p <profile> -c <cluster> <name>
$ senlin node-delete <node>
$ senlin node-show <node>
$ senlin node-join -c <cluster> <node>
$ senlin node-leave <node>
$ senlin cluster-node-add -n <nodes> <cluster>
$ senlin cluster-node-del -n <nodes> <cluster>
$ senlin cluster-node-list <cluster>
Profile CRUD
Cluster
CRUD
Node CRUD
Cluster
Membership

14. policies operations
14
$ senlin policy-create -s <spec> <name>
$ senlin policy-list
$ senlin policy-show <policy>
$ senlin policy-delete <policy>
$ senlin policy-update -n <name> ... <policy>
$ senlin cluster-policy-attach <cluster> <policy>
$ senlin cluster-policy-detach <cluster> <policy>
$ senlin cluster-policy-list <cluster>
$ senlin cluster-policy-show <cluster> <policy>
$ senlin cluster-policy-enable <cluster> <policy>
$ senlin cluster-policy-disable <cluster> <policy>
$ senlin cluster-policy-update <cluster> <policy>
Policy
Management
Cluster-Policy
Bindings

15. Other Useful Operations
15
$ senlin cluster-scale-in [-c <count>] <cluster>
$ senlin cluster-scale-out [-c <count>] <cluster>
$ senlin cluster-resize <options> <cluster>
$ senlin profile-type-list
$ senlin profile-type-show <type>
$ senlin policy-type-list
$ senlin policy-type-show <type>
$ senlin action-list
$ senlin action-show <action-id>
$ senlin event-list
$ senlin event-show <event-id>
Cluster
Resize
Profile Type
Policy Type
Action
Event

16. senlin cluster-resize
16
usage: senlin cluster-resize [-c <CAPACITY>] [-a <ADJUSTMENT>] [-p <PERCENTAGE>]
[-t <MIN_STEP>] [-s] [-n MIN] [-m MAX] <CLUSTER>
Positional arguments:
<CLUSTER> Name or ID of cluster to operate on.
Optional arguments:
-c <CAPACITY>, --capacity <CAPACITY> The desired capacity the cluster.
-a <ADJUSTMENT>, --adjustment <ADJUSTMENT>
A positive integer meaning the number of nodes to add,
or a negative integer indicating the number of nodes
to remove.
-p <PERCENTAGE>, --percentage <PERCENTAGE>
A value that is interpreted as the percentage of size
adjustment. This value can be positive or negative.
-t <MIN_STEP>, --min-step <MIN_STEP>
An integer specifying the number of nodes for
adjustment when <PERCENTAGE> is specified.
-s, --strict A boolean specifying whether the resize should be
performed on a best-effort basis when the new capacity
may go beyond size constraints.
-n MIN, --min-size MIN New lower bound of cluster size.
-m MAX, --max-size MAX New upper bound of cluster size. A value of -1
indicates no upper limit on cluster size.
Adjustment Types
• EXACT CAPACITY
• CAPACITY CHANGE
• CAPACITY PERCENTAGE
Optional Modifiers
• Strict vs. Best-Effort
• Minimum Step
Dealing with Constraints
• MIN_SIZE
• MAX_SIZE

17. Senlin Roadmap

18. Senlin Journey
Talk on China
OpenStack/Docker
community joint
meetup
2015-04-25
• IBM internal
announce
• Email to OpenStack
Heat Core team
2015-02-10
Project
accepted to
StackForge
2015-03-16
Community
Announcem
ent
2015-03-26
Git repo
migrate to
github.com
2014-12-25
Initial Git
Repository
inside
CRL
2014-12-10
First weekly
IRC
meeting
2015-06-10
Talk on
OPNFV
meeting
2015-07-08
Start work
with TOSCA
std team
2015-07-09
Talk on 2nd China
OpenStack
Hackathon
2015-08-20
Senlin
dashboard
project
initiated
2015-08-24
Talk on Tokyo
Summit -- Scaling
for Containers
2015-10-29
Senlin
docs
went
online
2015-12-03
Project
accepted
into Big
Tent
2015-11-11
Mitaka
Mid-Cycle
Meetup
2016-01-11
Vancouver
Heat design
summit
2015-05-20
2015
Jan
2015
Jul
2016
Jan
2015
Apr
2015
Feb
2015
Mar
2015
May
2015
Jun
2015
Oct
2015
Aug
2015
Sep
2015
Nov
2015
Dec
2014
Dec
2016
Feb
2016
Mar
2016
Apr
Version
1.0
released
2016-04-07
Senlinclient
0.4.0
released
2016-03-19

19. Mitaka Features
• API Changes and API Reference
• OpenStackClient Support
• Heat Resources
• Health Management primitive
• User/Developer Documentation
• Senlin Dashboard

20. Newton Outlook
API micro-version
Health management
Container Cluster
Testing:
API/Scenario/Performance...
Additional Clustering Operations
Queue (e.g. Zaqar) type Receiver
Notifications to other services
Batched Operations
User Defined Actions
Access Control
More communications, more collaborations

21. Relation To Other Projects
21
Senlin
Ceilometer
Heat
Nova Cinder Neutron Swift Keystone
Horizon
Primitive Data Types
Complex Data Types
struct person {
int age;
char name[0];
}
person team[10]; // Senlin cluster of Heat stacks
// Senlin cluster of nova servers
// Heat stack containing senlin clusters
Mistral MagnumSaharaMonasca Tacker Zaqar
Senlin provides the array data type for cloud programming

22. We Are Hiring, 
Standard
OpenStack
Customer
IBMTacker
Contributors
Zaqar

23. Questions?

24. Thank You!