Open stack qa and tempest

3,866 views
3,629 views

Published on

Tempest is an Openstack test suite which runs against all the OpenStack service endpoints. It makes sure that all the OpenStack components work together properly and that no APIs are changed. Tempest is a "gate" for all commits to OpenStack repositories and will prevent merges if tests fail.

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
3,866
On SlideShare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
118
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Open stack qa and tempest

  1. 1. OpenStack QA and TempestDavid KranzMay 6, 2013Monday, May 06, 2013
  2. 2. Outline OpenStack QA Tempest Future directions How to contributeMonday, May 06, 2013
  3. 3. Current OpenStack QA Landscape Upstream (OpenStack QA project) Tempest (functional and integration tests) Grenade (upgrade testing) Gerrit/Jenkins (continuous integration and gating) Projects Unit and functional tests Packagers and Deployers Testing in real multi-node environments Stress and performance testingMonday, May 06, 2013
  4. 4. What is Tempest? Comprehensive test suite for OpenStack API validation Integration testing Can be configured to run against realdeployments Runs as part of the gating process or on aperiodic basis Written in PythonMonday, May 06, 2013
  5. 5. What is Devstack? Opinionated single-node deployment Creates complete OpenStack system All components configurable Used by developers, testers and by jenkins The jenkins “gate” uses devstack and Tempest https://github.com/openstack-dev/devstack.gitMonday, May 06, 2013
  6. 6. What is “the gate”? Tests that run on every commit before merging Includes per-project unit tests Tempest runs on all commits to all projects Integration tests help make sure commits donot break other projects Blocks incompatible API changes https://wiki.openstack.org/wiki/Governance/Approved/APIStabilityMonday, May 06, 2013
  7. 7. Running Tempest with Devstack Stack.sh runs devstack and creates/opt/stack/etc/tempest.conf Tempest will spin up images, servers, volumes,etc., do things to them, bring them down. Takes about 45 minutes to run all tests Edit tempest.conf to run against real deploymentMonday, May 06, 2013git clone git://github.com/openstack-dev/devstack.gitcd devstack./stack.shcd /opt/stack/tempest./run_tests.sh [tempest/tests/...]
  8. 8. Example of Tempest Test CaseMonday, May 06, 2013def test_create_with_existing_server_name(self):# Creating a server with a name that already exists is allowedserver_name = rand_name(server)resp, server = self.create_server(name=server_name, wait_until=ACTIVE)id1 = server[id]resp, server = self.create_server(name=server_name, wait_until=ACTIVE)id2 = server[id]self.assertNotEqual(id1, id2, "Did not create a new server")resp, server = self.client.get_server(id1)name1 = server[name]resp, server = self.client.get_server(id2)name2 = server[name]self.assertEqual(name1, name2) Test cases use unittest2 From tempest/tests/compute/servers/test_servers.py
  9. 9. Tempest in the Gate Three tempest runs per commit MySQL PostGresSQL Quantum Full tempest run Spins up blank Ubuntu 12.04 node Installs with devstack Runs 700+ tests Takes about 40 minutes in gateMonday, May 06, 2013
  10. 10. Opentack QA Roadmap Move more testing upstream (shared effort!) Support multi-node testing in jenkins (triple-o) Integrate reports from downstream tempest runs usingvarious real hardware configurations Performance testing Better upgrade testing Add newly incubated projects to Tempest Use Tempest as validation suite for the OpenStacktrademark Improve Tempest performance by parallel execution Make stress testing reliableMonday, May 06, 2013
  11. 11. Where Tempest needs Help New tests to cover rest of API Strong on Nova, weaker on other projects Lots of extensions untested New integrated projects: ceilometer and heat Improving performance Upgrade testing Stress testingMonday, May 06, 2013
  12. 12. Getting Involved with Tempest Hang out with us in #openstack-qa Weekly QA meetings Thursday 17:00 UTC Get on openstack-qa mailing list http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa Always looking for help filling test gapsMonday, May 06, 2013
  13. 13. Questions?Monday, May 06, 2013

×