OpenStack Summit Berlin
Rico Lin, irc: ricolin <rico.lin@easystack.cn>
Zane Bitter irc: zaneb <zbitter@redhat.com>
Improvements in OpenStack
Integration for Application
Developers
Nov 2018
An Application
An Application Developer
● Who care about real use cases cross
node, cluster, and even platform
● Who need to stay awake 24 hr when
service is added/upgraded
● Who have to care about multiple projects
● Who will always been called when
environment failed
● A lot more in Application itself...
Where it needs to be improved
● Cross Community Integration
● Cross Project Integration
● Cross Site/Platform Integration
● Community Improvement
● Upgrade
Are we there yet?!
Are we there yet?!
Are we there yet?!
Are we there yet?!
Not yet, my little Smurfs!
But here are some things we started
Application Credentials [more]
clouds:
openstack:
auth:
auth_url: https://cloud.example.com/identity/v3
username: "Rico"
user_domain_name: "openstack.de"
application_credential_name:
"volume_backups_001"
application_credential_secret: "{Credential ID}"
region_name: "RegionOne"
interface: "public"
identity_api_version: 3
auth_type: "v3applicationcredential"
Cross Site/Platform Integration
An application credential is a scoped auth method
that a user creates to delegate a subset of their role
assignments on a single project to something else -
whoever or whatever possesses knowledge of the
identifier and the secret belonging to the application
credential.
● Has its own secret
● Can only access one project, no matter how
many projects the user is in
● Can have all or a subset of the roles the user
has on that project
● Is user-lived - when the user is deleted, the app
credential dies
● User can have many
Application Credential Capabilities
POST /v3/capability-templates
{
"capability_template": {
"allow_chained": true,
"role_id": "0dbbcb80-9d70-4c86-b38a-ae826e501885",
"path": "/v2.1/servers/**",
"substitutions": {},
"service": "67764758-3bdb-462e-babf-537c8fbe7bcd",
"type": "GET"
}
}
Cross Site/Platform Integration
Nova instance credentials
Cross Project Integration
Once we are able to lock down application credentials to particular
capabilities, it would be useful to automatically assign a set of these
credentials to a Nova server so that the application running on it can use
them to access the cloud. Depending on where the server gets its
metadata from, it should even be possible to regularly rotate the
credentials so that a temporary breach of the server doesn’t necessarily
result in ongoing access.
Pre-signed URLs
Several services in OpenStack make use of pre-signed URLs that can be
given to applications to allow them limited access only to resources
explicitly specified by the user. Examples include reading and writing data
in Swift (requires the TempURL middleware), or sending and receiving
messages in Zaqar. You can both limit the type of access (e.g. read-only
or write-only) and the duration for which it is valid. Heat also makes use
of pre-signed URLs in some circumstances to allow workloads to
communicate status back to it.
Cross Project Integration
Self-healing with Heat
Cross Project Integration
XXX::Server
XXX::Signal XXX::Alarm
XXX::Workflow
Signal
Meter
Trigger
How you
metering
?
How you
handle
signal?
How you
trigger a
fix job
What's
meter to
you?
Fix
server:
type: OS::Nova::Server
properties:
...
alarm_queue:
type: OS::Zaqar::Queue
error_event_alarm:
type: OS::Aodh::EventAlarm
properties:
event_type: compute.instance.update
query:
- field: traits.instance_id
value: {get_resource: server}
op: eq
- field: traits.state
value: error
op: eq
alarm_queues:
- {get_resource: alarm_queue}
alarm_subscription:
type: OS::Zaqar::MistralTrigger
properties:
queue_name: {get_resource: alarm_queue}
workflow_id: {get_resource: autoheal}
input:
stack_id: {get_param: "OS::stack_id"}
root_stack_id:
if:
- is_standalone
- {get_param: "OS::stack_id"}
- {get_param: "root_stack_id"}
autoheal:
type: OS::Mistral::Workflow
properties:
description: >
Mark a server as unhealthy and commence a stack update
to replace it.
input:
stack_id:
root_stack_id:
type: direct
tasks:
- name: resources_mark_unhealthy
action:
list_join:
- ' '
- - heat.resources_mark_unhealthy
- stack_id=<% $.stack_id %>
- resource_name=<%
env().notification.body.reason_data.event.traits.where($[0] =
'instance_id').select($[2]).first() %>
- mark_unhealthy=true
- resource_status_reason='Marked by alarm'
on_success:
- stacks_update
- name: stacks_update
action: heat.stacks_update stack_id=<% $.root_stack_id
%> existing=true
Self-healing with Heat
Vision for OpenStack Clouds
TC is working with the community to produce a vision
document that will help guide future development of
OpenStack. The document defines giving applications
control over their own infrastructure as one of the
pillars of cloud computing that sets it apart from earlier
models.
Community Improvement
Expose SIGs and WGs
Community Improvement
Current
Scenario(Story)
Project A team
Project B team
Project C team
Tasks
Project A team
Project B team
Project C team
Scenario(Story)
SIGs/WGs
Exposed
OpenStack site A
Multi-Cloud Orchestration
Cross Site/Platform Integration
Stack2Stack
Stack2
OpenStack site B
resources:
app_stack_at_site_B:
type: OS::Heat::Stack
properties:
context:
credential_secret_id: {Barbican Secret ID}
openstack secret store -n site-b-cred --payload
'{
"auth_type": "v3applicationcredential",
"auth": {
"auth_url": "URL",
"application_credential_id": "Credential ID",
"application_credential_secret": "sec"},
"roles": ["admin"],
"project_id": "Project_ID"
}'
K8s Cluster Autoscaling on OpenStack
Cross Community Integration
Kubernetes OpenStack
Node
ResourceGroup
Instance Instance
K8s K8s
ResourceGroup
Instance Instance
K8s K8s
Magnum
Pod
Node
Pod Pod
Node
Pod
Pod Pod Pod Pod
Kubernetes/Autoscaler
Autoscaling improvement
Cross Project Integration
Scaling group
Instance Instance
Application Application
Scaling group
Instance Instance
Application Application
Heat
InstanceGroup AutoScalingGroup
Cluster
Senlin
ResourceGroup
Just maybe…
we need new mascot for this...
Heat Kubernetes Resource
Cross Community Integration
Software Deploy
Nova Server
os-collect-config
os-refresh-config
os-apply-config
kubelet-hook
$ kubelet
Webserver
done
config-notify
Signal
Software Config
Software Deploy
Kubernetes
Software Config
Credential to
Kubernetes
Barbican
Stack
Software
Config
Software
Deploy
Kubernetes
Heat Ansible Resource
Cross Community Integration
Software Deploy
Nova Server
os-collect-config
os-refresh-config
os-apply-config
ansible-hook
$ ansible-playbook
Webserver
done
config-notify
Signal
Software Config
Software Deploy
Ansible
Software Config
Credential to
Ansible
Barbican
Stack
Software
Config
Software
Deploy
Ansible
Rico Lin, irc: ricolin <rico.lin@easystack.cn>
Zane Bitter irc: zaneb <zbitter@redhat.com>
Thank You!
Danke!
Nov 2018
Q & A
Rico Lin, irc: ricolin <rico.lin@easystack.cn>
Zane Bitter irc: zaneb <zbitter@redhat.com>
Thank You!
Danke!
Nov 2018
Q & A
Rico Lin, irc: ricolin <rico.lin@easystack.cn>
Zane Bitter irc: zaneb <zbitter@redhat.com>
Thank You!
Danke!
Nov 2018
Q & A

Improvements in OpenStack Integration for Application Developers

  • 1.
    OpenStack Summit Berlin RicoLin, irc: ricolin <rico.lin@easystack.cn> Zane Bitter irc: zaneb <zbitter@redhat.com> Improvements in OpenStack Integration for Application Developers Nov 2018
  • 2.
  • 3.
    An Application Developer ●Who care about real use cases cross node, cluster, and even platform ● Who need to stay awake 24 hr when service is added/upgraded ● Who have to care about multiple projects ● Who will always been called when environment failed ● A lot more in Application itself...
  • 4.
    Where it needsto be improved ● Cross Community Integration ● Cross Project Integration ● Cross Site/Platform Integration ● Community Improvement ● Upgrade
  • 5.
    Are we thereyet?! Are we there yet?! Are we there yet?! Are we there yet?!
  • 6.
    Not yet, mylittle Smurfs! But here are some things we started
  • 7.
    Application Credentials [more] clouds: openstack: auth: auth_url:https://cloud.example.com/identity/v3 username: "Rico" user_domain_name: "openstack.de" application_credential_name: "volume_backups_001" application_credential_secret: "{Credential ID}" region_name: "RegionOne" interface: "public" identity_api_version: 3 auth_type: "v3applicationcredential" Cross Site/Platform Integration An application credential is a scoped auth method that a user creates to delegate a subset of their role assignments on a single project to something else - whoever or whatever possesses knowledge of the identifier and the secret belonging to the application credential. ● Has its own secret ● Can only access one project, no matter how many projects the user is in ● Can have all or a subset of the roles the user has on that project ● Is user-lived - when the user is deleted, the app credential dies ● User can have many
  • 8.
    Application Credential Capabilities POST/v3/capability-templates { "capability_template": { "allow_chained": true, "role_id": "0dbbcb80-9d70-4c86-b38a-ae826e501885", "path": "/v2.1/servers/**", "substitutions": {}, "service": "67764758-3bdb-462e-babf-537c8fbe7bcd", "type": "GET" } } Cross Site/Platform Integration
  • 9.
    Nova instance credentials CrossProject Integration Once we are able to lock down application credentials to particular capabilities, it would be useful to automatically assign a set of these credentials to a Nova server so that the application running on it can use them to access the cloud. Depending on where the server gets its metadata from, it should even be possible to regularly rotate the credentials so that a temporary breach of the server doesn’t necessarily result in ongoing access.
  • 10.
    Pre-signed URLs Several servicesin OpenStack make use of pre-signed URLs that can be given to applications to allow them limited access only to resources explicitly specified by the user. Examples include reading and writing data in Swift (requires the TempURL middleware), or sending and receiving messages in Zaqar. You can both limit the type of access (e.g. read-only or write-only) and the duration for which it is valid. Heat also makes use of pre-signed URLs in some circumstances to allow workloads to communicate status back to it. Cross Project Integration
  • 11.
    Self-healing with Heat CrossProject Integration XXX::Server XXX::Signal XXX::Alarm XXX::Workflow Signal Meter Trigger How you metering ? How you handle signal? How you trigger a fix job What's meter to you? Fix
  • 12.
    server: type: OS::Nova::Server properties: ... alarm_queue: type: OS::Zaqar::Queue error_event_alarm: type:OS::Aodh::EventAlarm properties: event_type: compute.instance.update query: - field: traits.instance_id value: {get_resource: server} op: eq - field: traits.state value: error op: eq alarm_queues: - {get_resource: alarm_queue} alarm_subscription: type: OS::Zaqar::MistralTrigger properties: queue_name: {get_resource: alarm_queue} workflow_id: {get_resource: autoheal} input: stack_id: {get_param: "OS::stack_id"} root_stack_id: if: - is_standalone - {get_param: "OS::stack_id"} - {get_param: "root_stack_id"} autoheal: type: OS::Mistral::Workflow properties: description: > Mark a server as unhealthy and commence a stack update to replace it. input: stack_id: root_stack_id: type: direct tasks: - name: resources_mark_unhealthy action: list_join: - ' ' - - heat.resources_mark_unhealthy - stack_id=<% $.stack_id %> - resource_name=<% env().notification.body.reason_data.event.traits.where($[0] = 'instance_id').select($[2]).first() %> - mark_unhealthy=true - resource_status_reason='Marked by alarm' on_success: - stacks_update - name: stacks_update action: heat.stacks_update stack_id=<% $.root_stack_id %> existing=true Self-healing with Heat
  • 13.
    Vision for OpenStackClouds TC is working with the community to produce a vision document that will help guide future development of OpenStack. The document defines giving applications control over their own infrastructure as one of the pillars of cloud computing that sets it apart from earlier models. Community Improvement
  • 14.
    Expose SIGs andWGs Community Improvement Current Scenario(Story) Project A team Project B team Project C team Tasks Project A team Project B team Project C team Scenario(Story) SIGs/WGs Exposed
  • 15.
    OpenStack site A Multi-CloudOrchestration Cross Site/Platform Integration Stack2Stack Stack2 OpenStack site B resources: app_stack_at_site_B: type: OS::Heat::Stack properties: context: credential_secret_id: {Barbican Secret ID} openstack secret store -n site-b-cred --payload '{ "auth_type": "v3applicationcredential", "auth": { "auth_url": "URL", "application_credential_id": "Credential ID", "application_credential_secret": "sec"}, "roles": ["admin"], "project_id": "Project_ID" }'
  • 16.
    K8s Cluster Autoscalingon OpenStack Cross Community Integration Kubernetes OpenStack Node ResourceGroup Instance Instance K8s K8s ResourceGroup Instance Instance K8s K8s Magnum Pod Node Pod Pod Node Pod Pod Pod Pod Pod Kubernetes/Autoscaler
  • 17.
    Autoscaling improvement Cross ProjectIntegration Scaling group Instance Instance Application Application Scaling group Instance Instance Application Application Heat InstanceGroup AutoScalingGroup Cluster Senlin ResourceGroup
  • 18.
    Just maybe… we neednew mascot for this...
  • 19.
    Heat Kubernetes Resource CrossCommunity Integration Software Deploy Nova Server os-collect-config os-refresh-config os-apply-config kubelet-hook $ kubelet Webserver done config-notify Signal Software Config Software Deploy Kubernetes Software Config Credential to Kubernetes Barbican Stack Software Config Software Deploy Kubernetes
  • 20.
    Heat Ansible Resource CrossCommunity Integration Software Deploy Nova Server os-collect-config os-refresh-config os-apply-config ansible-hook $ ansible-playbook Webserver done config-notify Signal Software Config Software Deploy Ansible Software Config Credential to Ansible Barbican Stack Software Config Software Deploy Ansible
  • 21.
    Rico Lin, irc:ricolin <rico.lin@easystack.cn> Zane Bitter irc: zaneb <zbitter@redhat.com> Thank You! Danke! Nov 2018 Q & A
  • 22.
    Rico Lin, irc:ricolin <rico.lin@easystack.cn> Zane Bitter irc: zaneb <zbitter@redhat.com> Thank You! Danke! Nov 2018 Q & A
  • 23.
    Rico Lin, irc:ricolin <rico.lin@easystack.cn> Zane Bitter irc: zaneb <zbitter@redhat.com> Thank You! Danke! Nov 2018 Q & A