The document summarizes the OpenStack Interoperability Working Group's efforts to promote interoperability across OpenStack distributions and products. It discusses how the group develops guidelines specifying required capabilities and tests. Products must pass these tests to be considered interoperable and qualify for the OpenStack logo program. The guidelines aim to ensure a consistent user experience while allowing flexibility in implementations. The document also outlines the group's governance process and opportunities for participants to provide feedback to help improve interoperability standards over time.

1. © 2016 VMware Inc. All rights reserved.
“Together, or Not At All?”
Interop WG: The Interoperability Standard for OpenStack
Mark T. Voelker, OpenStack Architect
Updated May 2017

2. 2
Perhaps inherent in that ambition is the promise of interoperability.
Code against one set of capabilities and API’s, take your pick of many
public clouds, distributions, appliances, and services.
As it turns out, even clouds that are on some level the same
can look and act very differently.
OpenStack aims to be a “ubiquitous Open Source Cloud Computing platform that will meet the
needs of public and private clouds, regardless of size…”

3. 3
There are ~4,650 config options in the tc-approved-release OpenStack projects.
And ~1,073 policy.json configurations (as of late 2015).
OpenStack has a lot of nerd knobs that can
affect how a cloud behaves. In fact…

4. 4
You can also put things in front of OpenStack that change the behavior users
see…like firewalls.
…and load
balancers, API
gateways, SSL
configurations…

5. The mechanics of what’s under the hood may change behavior too…
(take image formats supported by various virt drivers & storage platforms for example) 5

6. 6
And of course different types of workloads
use the cloud in different ways…

7. 7Think you can code against that?

8. Mark T. Voelker (@marktvoelker)
• OpenStack Architect @ VMware, Interop Working Group co-chair
• Fact: can be bribed with doughnuts
• In copious (hah!) spare time: OpenStack solutions, Big Data, Massively Scalable Data Centers, DevOps,
making sawdust with extreme prejudice, raising two great kids with my awesome wife in North Carolina
“A computer nerd….is somebody who uses a computer in order to use a computer.”
–Douglas Adams

9. 9
[note: this talk will be slightly more entertaining if you’re a science fiction fan…
…otherwise it will merely be somewhat informative.]

10. 10
Why Should Vendors Care
About Interoperability
Standards?
• It’s good for your users
• It helps you promote your product
• It helps applications be developed
for your platform
• It’s now required if you want to
call your product OpenStack.

11. 11
def list_images():
“”” A function to list images. Because all OpenStack Powered Platforms can do that…somehow.””"
if $cloud == ‘vendorA’:
# TODO: this also works for vendorX
list_images_via_nova_image_api()
elif $cloud == ‘vendorB’:
# TODO: this also worked for vendorY last week but now, um?
list_images_via_glance_v1()
elif $cloud == ‘vendorC’:
list_images_via_glance_v2()
else:
# I dunno what cloud this is, but it’s OpenStack Powered! So something must work.
# Resort to trial and error since we don’t know.
try:
list_images_via_nova_image_api()
except NopeError:
# D’oh, guess that wasn’t it…
try:
list_images_via_glance_v1()
except StillNopeError:
# Aww…well third time’s the charm?
try:
list_images_via_glance_v2()
except NopeNopeNopeError:
rage_quit()
This function could
also be called:
not_winning()

12. 12
Someone should make a standard.
Situation:
Interoperability is hard.

13. 13
Soon
Soon:
Want to prevent that?
When the community
works together with
the Interop Working
Group, we can.
To understand how,
you need to know how
OpenStack is
governed.

14. 14
OpenStack is primarily
governed by two bodies:
Technical Committee
& the
Board of Directors
(you decide which is which)

15. 15
The Technical Committee provides:
“technical leadership for OpenStack as
a whole…..enforces OpenStack ideals
(Openness, Transparency,
Commonality, Integration, Quality…),
decides on issues affecting multiple
projects…..”

16. 16
“The Board of Directors provides strategic & financial oversight of Foundation resources and staff.”
Which
includes
these things

openstack.org/marketplace

17. 17
Interop WG is a Board activity.
• One Director serves as co-chair, other
co-chair elected by participants.
• It’s work and procedures must ultimately
be approved by a vote of the Board, not
the +2’s of it’s most trusted reviewers.
• It produces “Guidelines”, not
infrastructure code.
• It can use the OpenStack trademark and
logo as both a carrot and stick.
• It can make requirements for products
that call themselves OpenStack.

18. 18
So what’s a Guideline, then?
• A list of Capabilities that products
must support.
• A list of Tests products must pass
to prove it.
• A list of Designated Sections of
OpenStack code they must use to
provide those Capabilities.
(also: a list of exceptions & things that
might be required in the future)

19. 19
What’s the cadence of Guidelines?
• New Guideline every 6 months
• Each Guideline covers 3 OpenStack releases
• Only the newest 2 can be used for new logo
program qualification

20. 20
Things you won’t find in
Interoperability Guidelines:
• Stuff that end users don’t
see or can’t use:
• Admin-only API’s
• RPC API’s
• DB Schema
• HA Requirements
• Stuff that’s intentionally
pluggable:
• Virt/net/storage drivers
• Middlewares
• Specific databases
• Stuff that doesn’t have tests
• Stuff that’s being
deprecated (usually…more
on that in a minute)

21. 21
How do we decide what
gets in?
10 Guiding Principles
12 Core Criteria
A giant list of tests
New Capabilities must be
“advisory” (non-required)
for one Guideline before
becoming required.

22. 22
Interop Criteria
(Currently all weighted fairly
equally with a few small
deviations)
https://github.com/openstack/interop/blob/master/doc/source/process/CoreCriteria.rst

23. 23
First, we talk to
each other.

24. 24
Then, we
write things
down.

25. 25
When it’s all
ready, we
send it to the
Board of
Directors for
approval.

26. 26
Say more to me about these tests….
• Must be under TC governance
• All tests today are in Tempest (per the TC’s request)
• Must work on all releases covered by a Guideline
• Typically run via the RefStack wrapper which reports results to
refstack.openstack.org and shows which Guidelines you meet
Details:
http://openstack.org/interop
http://refstack.openstack.org

27. 27
Sometimes things go amiss…
Tests can be “flagged” (not required
for the duration of the Guideline) in
some cases:
• Capability fails to meet Criteria
(e.g. was scored incorrectly)
• Test fails/is skipped due to an
accepted bug in the project
• Test fails/is skipped due to an
accepted bug in the test
• Test fails because it requires non-
required Capabilities
• Test reflects an implementation
choice that isn’t widely deployed
even though the Capability itself
is.

28. 28
When does all this happen?
• Summit-3 months: preliminary draft
• Summit-2 months: ID new Capabilities
• Summit-1 month: Score Capabilities
• Summit: “Solid” draft
• Summit+1 month: Self-testing
• Summit+2 months: Test Flagging
• Summit+3 months: Board Approval
(Note: 2015 was weird in that we had a very
accelerated schedule to get DefCore
bootstrapped…above is what it looks like
from now on.)

29. 29
Why isn’t $my_favorite_thing in
the current Interoperability
Guideline?

30. 30

31. 31
• It didn’t meet Criteria
(scored too low)
• It wasn’t scored in time
(scoring is surprisingly hard
to get right)
• It was admin-only or driver-
specific
• That project isn’t yet widely
deployed
• There wasn’t a test for it
• It didn’t score highly across
all releases covered in that
Guideline
• Nobody brought it up yet

32. 32
I’m an OpenStack
Developer
I have a really cool
new feature...
How do I get it into the
Interoperability
Guidelines?

33. 33
I’m the Interop WG co-chair.
I have a blog post you should read!
http://markvoelker.github.io/blog/defcore-for-devs/
• Document it well
• Ensure it has usable tests
• Foster adoption among users,
SDK’s, & other projects
• Be patient: needs to be in 3
OpenStack releases

34. 34
Do we get to offer feedback?
Absolutely!
• Feedback built into Interop WG
scoring cycles
• Feedback encouraged for
Advisory capabilities
• Feedback encouraged via flag
requests
• Feedback via User Survey
• Feedback via RefStack
community-visible results (you may also buy me a beer & bend
my ear about interoperability anytime)

35. 35
How do I make RefStack work for me?
It’s actually not that hard.
Instructions here which boil down to:
1. Download refstack-client.
2. Run the “setup_env” script.
3. Configure tempest for your cloud.
4. Run refstack-client to execute tests.
5. Upload results to refstack.openstack.org and review.

36. 36
Why run all the tests &
upload results?
Because more data is
better.
This gives us additional data
about what works in your cloud.
With data from lots of clouds, we
can make better scoring
decisions in the future when
considering adding Capabilities
to Guidelines.

37. 37
What if I don’t pass all the
required tests?
Don’t panic.
• Figure out why some tests failed.
• Was it environmental (e.g. a timeout due to
storage being slow)? Tweaking tempest
config may help.
• Was it due to a bug in the test?
• Was it due to a bug in OpenStack?
• Do you have grounds for requesting a flag?
• Valid reasons for flagging a test and how to
do it can be found here.
• Talk to us!
• We have an interest in helping you succeed.
• interop@openstack.org is here to help!
• Catch us on IRC at #openstack-interop or
#openstack-refstack.

38. 38
What’s Next?
• Work begins on 2017.08
• More testing
• Vendors currently testing against just-released 2017.01
• Vertical Programs
• Example: what would “OpenStack Powered NFV” look like?
• New add-on programs and/or “vertical” programs
• Example: what would “OpenStack Powered Compute with
Database” look like?
• Could these be a conduit to projects defining (early on, for
themselves) what interoperability looks like to users of their
project?

39. 39
Sometimes the enemy is us.
• Sometimes projects are slow
to adopt support for each
others’ new API’s and features.
• Sometimes projects provide
multiple ways to do the same
things (and sometimes they’re
mutually exclusive).
• Sometimes we don’t have
good data about what’s really
supported.
• Sometimes tests use admin
credentials unnecessarily or
lump many Capabilities into
one test.

40. 40
For this to work, we have to communicate as one community.
• Board of Directors/Interop
Working Group
• Technical
Committee/technical
contributors
• End Users
• Vendors
• Ecosystem

41. 41
But what interoperability issues
should we attack first?
How about the ones in
this report?

42. 42
What interoperability issues
should I, an end user, watch
out for ?
Here’s that report again.
1. How we test interop
2. Variance in API evolution
3. External network connectivity
4. API and Policy discoverability
5. Understanding the Interop WG

43. 43
Add-on
Programs

44. 44
I need some
additional
things in my
cloud beyond
what’s
covered by
Powered…
...wish I had interop for those too.

45. 45
If you use some less
widely deployed
projects, you
probably care about
them…a lot.
(Project devs do too.)

46. 46
• Build on existing
Powered
programs
• Let project teams
define what
interop looks like
for them
Platform
D
N
S
D
B
.
.
.
.
.
.

47. 47
When projects think
about interop early,
everyone wins.
When consumers
have a way to see
what products
support, everyone
wins.

48. 48
Now, did you
mention
something
about….
...verticals?

49. 49
Think about
the Linux
ecosystem.
The kernel &
userland you
run in your
datacenter…

50. 50
…are
probably a
bit different
than the one
you have on
your phone.
(or in your router,
submarine, cash
register, seatback
entertainment unit,
or drone)

51. 51
We still think of both
as “Linux.”
They’re just
specialized for
specific use cases.
We’re seeing
specialization in
OpenStack now too.

52. 52
Within products
addressing a
particular use case,
interoperability is
still useful and
important.
Otherwise things
might not behave
as expected when
moving from one
cloud or product to
another.

53. 53
We’re just getting started
designing vertical programs.
Much depends on the use case.
Some things a use case might
need:
• Capabilities not commonly
exposed in general purpose
compute clouds
• Specific attributes in the API
• Scenario tests (beyond API
availability, maybe beyond
Tempest)
• Specific control plane design
choices
• Performance criteria.
• Admin API’s
• Different OpenStack projects

54. 54
The key is to take the general
methodology and criteria we’ve
already developed….
...and apply them in a way that’s
specific to a vertical use case.

55. 55
• Same BoD approval
• Same schema
• Same general process
• Same criteria (?)
• Builds on Powered (?)
• Different tests
• Different reporting
• Different mechanics

56. 56
And of course it sometimes helps to work
with our friends in adjacent
communities…

57. 57
Some examples of
things a general-
purpose compute
clouds might not need
for interoperability that
an NFV cloud might:
• SR-IOV support
• High PPS network
dataplane rates
• Provider network
support
• NUMA topology
aware scheduling
• CPU pinning
• Active-active control
plane design
choices

58. 58
We’d like to get
moving quickly on
developing programs
for vertical use cases.
There seems to be
demand, and people
ready to help. NFV
seems like a good first
target.
Buckle up!

59. 59
I’ll be jumping in myself.
Speaking of which, we should really wrap up today’s
talk and get on with it…

60. Want to learn more?
• 2017.01 Guideline
• 2016.08 Guideline
• Next Guideline draft
• Public RefStack Server
• OpenStack Interop Homepage
• Core Criteria
• Interop WG procedural overview
• Lexicon of Interop WG terms
• Interop WG wiki & meeting Info
• How to submit patches

61. 61
“Do what I do. Hold on tight and pretend it’s a plan!”

62. 62
Thank you.
[with apologies to fine the folks at the BBC’s “Doctor Who”]
(please don’t have me arrested)