1. The document discusses integrating Magnum and Senlin for autoscaling containers across multiple levels. Magnum provisions and manages container clusters while Senlin provides clustering and autoscaling capabilities. 2. A design is proposed where Senlin policies and triggers drive autoscaling at both the container and cluster level. Scaling would be coordinated between applications and clusters through control flows in both directions. 3. Integration with existing autoscaling in container orchestration engines like Kubernetes is also considered, where Senlin could leverage autoscalers but also handle scaling requirements not met by the native solutions.

1. 1
Exploring Magnum and Senlin integration
for autoscaling containers
Hongbin Lu (Huawei)
Jay Lau, Ton Ngo (IBM)
Qiming Teng, Julio Ruano (IBM)
Exploring Magnum and Senlin integration

2. Contributors
Exploring Magnum and Senlin integration2
Zilian Ji, Linjuan Xia, Shijia E, Peiyu Lin, Tianyuan Zhuang, Ji Jiang, Yongkui Wang

3. Talk Outline
1. Use cases
2. Magnum
3. Senlin
4. How it works
5. Demo
Exploring Magnum and Senlin integration3

4. Use cases
Exploring Magnum and Senlin integration4

5. Use case
• Shared COE cluster
– Autoscale on some policies (utilization, etc.)
– Scale number of nodes
• Multiple apps on cluster
– Autoscale on some policies (request rate, etc.)
– Scale number of containers
• Example:
– 2, 3-tiered web apps: handle requests
– SLA determines policy
Exploring Magnum and Senlin integration5

6. Scaling scenarios (1)
Exploring Magnum and Senlin integration6
node
c c c
node
c cc
node
c scale app
scale cluster
Single app on cluster
• scale out containers as needed
• scale cluster by utilization
• coordination = utilization

7. Scaling scenarios (2)
Exploring Magnum and Senlin integration7
node
c
node
c c
node
c c c c scale 2 apps
Two apps on cluster
• scale out containers as needed
• scale cluster by utilization
• coordination = utilization
scale cluster
unlimited
scale cluster
limited

8. Scaling scenarios (3)
Exploring Magnum and Senlin integration8
node
c c
node
scale 2 apps
scale cluster
limited
c
node
c c c c c c
Two apps on cluster, limited cluster size
• blue scales first, retains containers
• red scales later, fails
• maybe acceptable
• what if red is critical and blue is not
• inverted priority: blue < red

9. Scaling scenarios (4)
Exploring Magnum and Senlin integration9
node
c c scale 2 apps
Limit on apps
• Place max on blue
• blue scales, red does not scale
• cannot use full resources
scale cluster
limited
c
nodenode
c
limited

10. Scaling scenarios (5)
Exploring Magnum and Senlin integration10
node
c c
node
scale 2 apps
scale cluster
limited
c
node
c c c c c cc c c c
Priority on apps
• blue scales first, red scales later
• blue scales back to free up resource for red
• 2 ways linkage: container cluster host cluster

11. Magnum
Exploring Magnum and Senlin integration11

12. Instance
Magnum Overview
Exploring Magnum and Senlin integration12
• Provision
• Kubernetes
• Docker Swarm
• Mesos
• Scale
• Add instances
• Remove instances
• Manage
• Pod
• Service
• Replication Controller
• Container
Nova Instance
container
container
Magnum CLI
Magnum
Heat

13. Magnum Architecture
Exploring Magnum and Senlin integration13

14. Magnum: Autoscale Bay
1. Raw data: Magnum pulls raw
data from Bays.
2. Metrics: Metrics, like memory
utilization, are computed and sent
them to Ceilometer.
3. Alarm: Ceilometer triggers alarms
based on user-specified scenario
and metrics.
4. Scale: Heat scales the bays.
Exploring Magnum and Senlin integration14
Ceilometer
Magnum Heat
Bay
1) Raw data
2) Metrics 3) Alarm
4) Scale

15. Kubernetes: Autoscale Pod
1. Metrics: The autoscaler
periodically queries pods’ metrics.
2. Scale: Based on collected
metrics, the autoscaler uses a
built-in algorithm to trigger the
scaling.
3. Resize: Replication Controller
resize the pods.
Exploring Magnum and Senlin integration15
Horizontal
Pod
Autoscaler
Pods
1) Metrics
3) Resize
Replication
Controller
2) Scale

16. Future direction
• Magnum AutoScaler
– A new component in Magnum
• Scaling Bays
• Scaling Applications
– Interacts with AutoScaler from COEs
• Kubernetes AutoScaler as a driver for container-level scaling
• Magnum responsible for scaling RCs without Kubernetes autoscaler
– Customizability
• data sources
• metrics and thresholds
• standby bay nodes (i.e. resource pool)
Exploring Magnum and Senlin integration16

17. Two-Levels of AutoScaling
Exploring Magnum and Senlin integration17
Working
K8S Bay
NodeNode
K8S
Master
Heapster
Kublet
cAdvisor
Kublet
cAdvisor
Storage
Backend
RC AutoScaler
Pod
Magnum AutoScaler Service
K8S Driver
Collector
Analyzer
Operator
Swarm
Driver
Mesos
Driver
Senlin
Magnum
Conductor
Standby

18. Senlin
Exploring Magnum and Senlin integration18

19. Clustering - A Missing Service in OpenStack
Exploring Magnum and Senlin integration19
Scalable
Load-Balanced
Highly-Available
Manageable
......
of any (OpenStack) objects

20. Senlin Architecture
Exploring Magnum and Senlin integration20
REST RPC
Senlin
Database
Policies
Profiles

21. Clustering/AutoScaling/LoadBalancing
Exploring Magnum and Senlin integration21
Nova
Kubernetes
Heat
Ironic BareMetal
VMs
Stacks
Containers
Senlin
placement
deletion
scaling
health
load-balance
batching

22. Scaling A Cluster using Senlin
Exploring Magnum and Senlin integration22
type: os.nova.server
version: 1.0
properties:
image: fedora-21
flavor: 3
networks:
- network: private
userdata:
...
Profile
Cluster
type: senlin.policy.scaling
version: 1.0
properties:
event: CLUSTER_SCALE_IN
adjustment:
type: CHANGE_IN_CAPACITY
number: 1
min_step: 1
best_effort: True
attach
Policy
type: senlin.policy.scaling
version: 1.0
properties:
event: CLUSTER_SCALE_OUT
adjustment:
. . .
attach
Policy
webhook
RESIZE

23. Policies
Exploring Magnum and Senlin integration23
type: senlin.policy.scaling
version: 1.0
properties:
adjustment:
min_step: 1
best_effort: true
number: 1
type: CHANGE_IN_CAPACITY
event: CLUSTER_SCALE_OUT
• Declarative
• Can be mapped to from a standard
• Example: SUR_Cluster
(Add k8s Minion / Add Spark worker node)

24. Trigger
Exploring Magnum and Senlin integration24
type: CeilometerThresholdAlarm
version: 1.0
rule:
meter_name: memory_util
comparison_operator: gt
threshold: 70
period: 60
evaluation_periods: 1
statistic: avg
query:
- field: resource_metadata.cluster
op: eq
value: SUR_Cluster
• Generic abstraction
• alarms in OpenStack
monitoring services
• e.g. Ceilometer Alarm

25. How it works
Exploring Magnum and Senlin integration25

26. Design goals
1. Coordinate scaling at apps and cluster level
• Control flows in both direction
• Apps -> Cluster: utilization, other methods
• Cluster -> Apps: priority, other methods
2. Integrate scaling support from COE if available
• Kubernetes autoscaler
3. Policy driven
• Easy to express, manage complex cases
Exploring Magnum and Senlin integration26

27. template
Outer Stack
Autoscaling: OpenStack-Only Solution (Today)
27 Exploring Magnum and Senlin integration
AutoScalingGroup Stack
Heat
Ceilometer
AH
PUp PDn
AL
ASGS1 S2 S3
update
ASG

28. Senlin
profile
Autoscaling: OpenStack-Only Solution (New)
28 Exploring Magnum and Senlin integration
Cluster
Ceilometer
AH
PUp
PDn
AL
SCALE_IN
SCALE_OUT
Node1 Node2 Node3
webhookspolicies triggers

29. Demo
Exploring Magnum and Senlin integration29

30. Autoscaling with Kubernetes
30 Exploring Magnum and Senlin integration
VMs
K8S Bay
NodeNode
Heapster
Kublet
cAdvisor
Kublet
cAdvisor
Pod
Senlin
Magnum
profile
CeilometerWorkload
policy

31. Questions
Exploring Magnum and Senlin integration31

32. Thank You
Exploring Magnum and Senlin integration32