How to Develop for OpenStack APIs

Developing for
Openstack APIs
Yoram Weinreb, CTO Office
Ran Ziv, Cloudify R&D
Gigaspaces
Download the deck

Agenda
- Introduction to Openstack API
- Exposing the APIs in an Orchestrator
- Openstack API Quirks and Pitfalls
- Testing with Openstack API

Introduction to
OpenStack API
How do we interact with OpenStack?
- Send Actions/Commands
- Collect information

Interact with OpenStack
Openstack Endpoints RESTful API
Horizon Openstack CLI Tools
Openstack Official Python
Client Libraries
SDKs for other
Languages

RESTful endpoints
- API Versioning
- For Example Identity ver. 1, 2, 3…
- Interface
- Public
- Internal
- Admin
- Format (JSON, XML)
The northbound interfaces

Some Endpoint
examples
- Identity API v3
- Compute API v2.1
- Image service API v2
- Block Storage API v2
- Networking API v2.0
- Object Storage API v1
- Bare Metal API v1
- Orchestration API v1
- Telemetry API v2
- Clustering API v1
- Data Processing v1.1
- Database Service API
- Shared File Systems API v2

Debugging OpenStack API
- CLI --debug
- Python SDK debug params
- VM Logs on horizon & CLI
- Access to Openstack logs /
DevStack

Debugging OpenStack
API
Demo time...

Debugging using Python SDK
Another option to turn on logging is
with environment variables:
os.environ['NEUTRONCLIENT_DEBUG'] = 'true'

Viewing VM logs
- VM Logs on horizon
- VM logs on CLI
nova console-log [--length <length>] <server>
- VM logs in Python client:
server.get_console_output()

Viewing OpenStack logs
- Most logs are located at
/var/log/<service name>
- Devstack:
[[local|localrc]]
DEST=/opt/stack/
LOGFILE=$DEST/stack.sh.log
SCREEN_LOGDIR=$DEST/logs/screen

Exposing OpenStack APIs in
Cloudify

Exposing OpenStack APIs in Cloudify
- Cloudify is an open-source, pure-play orchestrator
- Based on the TOSCA open standard
- Describe your application components, their
lifecycle, and their relationships to one another
- Use either YAML format or the Cloudify Composer
- Supports both hybrid and multi cloud environments
- Open architecture allows for extending Cloudify to
work on any environment through plugins

- Cloudify supports Openstack via the Openstack plugin
- Strives to be unopinionated, support any use-case
- Offers simplicity and ease-of-use, yet not
compromise on flexibility
- Robust at dealing with cloud errors
- Makes abstractions common to other
environments where possible

- Resources are exposed via types defined by the plugin
- Prominent resource parameters are exposed explicitly
- Provides sensible default values, syntactic
sugaring, etc.
- Other parameters are configurable via direct override
- Parameters for Openstack clients used are also
configurable

- An example:
my_subnet_node:
type: cloudify.openstack.nodes.Subnet
properties:
resource_id: my-subnet
cidr: 1.2.3.4/24
dns_nameservers: [8.8.8.8]
Subnet: # direct override
enable_dhcp: false

- A Server example
- Keypair, SGs, networks/ports may be configured
directly or, preferably, via relationships
my_server_node:
type: cloudify.openstack.nodes.KeyPair
properties:
resource_id: my-server
image: Ubuntu-14.04 # translated to ID
flavor: x3.medium # translated to ID

- Some server operations take a while to complete
- VMs take long to boot (reach “ACTIVE”)
- Sometimes boot in errored state, or have
connectivity issues
- Cloudinit startup
- retrieve a password from the metadata service
- sshd service to start up
- An orchestrator needs to be able to handle
asynchronous operations

Volumes considerations:
- Volumes may also require asynchronous handling
- Volumes often require a specific set of operations to
become usable
- format
- mkfs
- mount

Security-groups considerations:
- In Openstack, a default SG always exists and will be
attached automatically to a new VM which doesn’t
declare otherwise
- Orchestrator must create SGs prior to the VM
and declare them at boot time
- When creating a new SG, it’s instantiated with
default, permissive egress rules
- disable_default_egress_rules: false

- Can override Openstack clients configuration
- Configurable on multiple scopes (deployment, node,
operation) to achieve both ease of use and flexibility
my_floating_ip_node:
type: cloudify.openstack.nodes.FloatingIP
properties:
openstack_config:
nova:
http_log_debug: true
neutron:
endpoint_url: https://1.2.3.4:9696

OpenStack API
Quirks and Pitfalls

OpenStack API Quirks and Pitfalls
- Some of the APIs aren’t necessarily intuitive
- Some due to historic and legacy reasons; some have
been or will be declared as bugs
- The orchestrator can help with handling these quirks
and making things more intuitive where possible

- A network may contain more than a single subnet
- When creating a server (or a port), required
parameter is “network_id” - subnet is chosen
arbitrarily
- To place a server on a specific subnet, must first
create a port and supply it with the “fixed_ips” param
{'name': 'my-port',
'network_id': '4c6d….aba9',
'fixed_ips': [{'subnet_id': '4e70...f2a6'}]}

- Unlike the rest of the resource types, Keypairs are
managed on a per-user basis, not per-tenant
- Can lead to funny behavior:
- Breaks isolation between tenants - resource
cleanup on one tenant may affect another
- Heat stack created by one user may not be
destroyable by another user on same tenant
- Keypair quotas are set per tenant but apply per
user

- In Openstack, floating IPs are first allocated and then
attached to a server or a port
- There’s no validation when attempting to attach a
floating IP which is already attached to a resource
- The floating IP will simply detach from the
previous resource
- race-condition scenarios
- checking an IP is allocated doesn’t necessarily
mean it’s safe to attach it to a resource

- The Nova API for adding a SG to a server is not
thread-safe (launchpad bug)
- An orchestrator that runs concurrent operations
should verify the addition after making the API call
- Neutron API for adding a SG to a port is thread-safe
- But not concurrency-friendly as it requires stating
all SGs rather than only the newly added one

- SG rules are defined over a port or a range of ports
- ICMP rules have a “type” and a “code” but no port
association
- Neutron’s security-groups API translates:
“port_range_max” → “code”
“port_range_min” → “type”
- E.g. - Allowing Ping from anywhere:
{'protocol': 'icmp',
'port_range_min': 0, # Ping’s ICMP Type
'port_range_max': 0, # Ping’s ICMP Code
'remote_ip_prefix': '0.0.0.0/24',
…}

- Nova’s API for adding a security-group to a server
may receive either a security-group’s name or ID
- If the security-group was created using Nova (i.e. it’s
a Nova-net security-group), passing the ID will fail
- Poses a problem when there are multiple security-
groups with the same name
- Poses a problem for orchestrator - the code for a
separate resource (Server) needs to be aware of
the SG’s type

- Ports can be created either explicitly or implicitly (e.g.
by connecting a server to a network
- Explicit ports should have to be deleted explicitly
- Up until Kilo release, deleting a server which was
connected to a port that had been created explicitly
would’ve deleted the port as well
- From an orchestrator’s POV, this is an abstraction
breach, as one resource affects the lifecycle of
another

- Keystone roles are assigned per tenant
- However, a user who’s assigned an admin role on one
tenant becomes an admin across all tenants
- Should stick to setting admin users only within an
“Admin” project
- Admin users can manage resources on any tenant
- May actually see resources of tenant A even
while using tenant B

- Unit tests
- mock library
- custom mocks
- Mimic project
- Integration tests
- Test the plugin’s operations against real
Openstack deployments, of different versions
- System tests
- End-to-end tests for the plugin
- The plugin is used to run many other
of Cloudify’s end-to-end tests
Testing with OpenStack API

- Testing environments:
- Integration tests and system tests run in parallel
over multiple Openstack tenants
- Pretty good isolation
- Easy clean-up
- Os-purge
- The tenants can be pre-existing
- Tenant deletion requires clean-up anyway
- Allows setting up an environment for
multiple tests to run serially
- Take resources snapshot before and
after each test - clean-up delta

- Tenants for tests are set across multiple
Openstack deployments
- Clouds have problems:
- VMs starting up in erroneous states
- Connectivity issues
- Maintenance and downtime
- ...
Relying on a single environment is risky
- Possibly tests multiple Openstack
versions / distributions

- When cleaning up a tenant’s resources after a
test, keypairs make an exception
- since they’re per-user rather than per-tenant
- Each test cleans up any Keypairs it created on its
own, whether it succeeds or fails
- An independent process is run when tests are
inactive to clean up any Keypair leftovers from
tests that’ve been stopped abruptly

- Testing multiple Openstack versions:
- Devstack
git clone git://git.openstack.
org/openstack-dev/devstack -b <release
branch name>
- Ravello
- A double virtualized environment in the
cloud.
- Define Ravello blueprints that will start and
stop isolated Openstack clouds in minutes

Thanks!
Come visit us at booth #C20!

- A more advanced example:
my_subnet_node:
type: cloudify.openstack.nodes.Subnet
properties:
resource_id: my-subnet
cidr: 1.2.3.4/24
relationships:
- target: my_network_node
type: cloudify.relationships.contained_in

How to Develop for OpenStack APIs

Recommended

Recommended

More Related Content

More from Cloudify Community

More from Cloudify Community (20)

Recently uploaded

Recently uploaded (20)

How to Develop for OpenStack APIs