This document summarizes Chmouel Boudjnah's presentation on software testing in OpenStack. It discusses different types of testing like unit testing, integration testing, and performance testing. It also covers testing methodologies like test-driven development and behavior-driven development. Finally, it provides examples of testing tools used in OpenStack like Tempest, static code analysis with flake8, and performance benchmarking with Rally.

1. Chmouel Boudjnah!
OpenStack in Action // Spring 2014
The art of Software testing in OpenStack
Chmouel Boudjnah - @chmouel - eNovance

2. Chmouel Boudjnah - @chmouel - eNovance
About me

3. Chmouel Boudjnah - @chmouel - eNovance
About this session

4. Chmouel Boudjnah - @chmouel - eNovance
Software Testing
overview

5. Chmouel Boudjnah - @chmouel - eNovance
• Unit Testing
• Integration Testing
• Performance Testing
• Scalability Testing.
Different type of Testing

6. Chmouel Boudjnah - @chmouel - eNovance
• Pentesting
• Usability Testing
• Regression Testing
• Compatibility Testing.
More testing

7. Chmouel Boudjnah - @chmouel - eNovance
Methodologies

8. Chmouel Boudjnah - @chmouel - eNovance
• Code a bit the test.
• Start a bit of coding.
• Make the test pass.
• Repeat until the feature is implemented.
TDD

9. Chmouel Boudjnah - @chmouel - eNovance
• Business Driven Development
Introducing BDD

10. Chmouel Boudjnah - @chmouel - eNovance
• Business Driven Development
• Behaviour Driven Development
Introducing BDD

11. Chmouel Boudjnah - @chmouel - eNovance
• Business Driven Development
• Behaviour Driven Development
• BS Driven Development (i.e: Vaporware)
Introducing BDD

12. Chmouel Boudjnah - @chmouel - eNovance
• Business Driven Development
• Behaviour Driven Development
• BS Driven Development
• Some people get it confused with “Bourrinage
Driven Development”
Introducing BDD

13. Chmouel Boudjnah - @chmouel - eNovance
Methodology
“You shall not pass a commit before each iterations has been
fully tested”

14. Chmouel Boudjnah - @chmouel - eNovance
Unit Testing

15. Chmouel Boudjnah - @chmouel - eNovance
The five year old technology mashup.

16. Chmouel Boudjnah - @chmouel - eNovance
The five year old technology mashup.
"well it is just like when you turn the wheel before we
are going on a ride you test that it turns properly
before actually riding it. If it didn’t turn then that means
I may going to have a crash and have a big bobo.”
Noah Boudjnah

17. Chmouel Boudjnah - @chmouel - eNovance
• Swift 2371 unit tests
• Ceilometer 2583
• Heat 2710
• Keystone 3216
• Nova 10653
In OpenStack

18. Chmouel Boudjnah - @chmouel - eNovance
• Mostly for code styling
• flake8 for pep8 and common
errors
• Openstack hacking
Static code analysis.

19. Chmouel Boudjnah - @chmouel - eNovance
Tooling and Libraries

20. Chmouel Boudjnah - @chmouel - eNovance
Devstack

21. Chmouel Boudjnah - @chmouel - eNovance
• tox
• pbr
• testrunner
• coverage
Tools

22. Chmouel Boudjnah - @chmouel - eNovance
• testtools
• mock
• httppretty
• six
Libraries

23. Chmouel Boudjnah - @chmouel - eNovance
Integration tests

24. Chmouel Boudjnah - @chmouel - eNovance
• run against any OpenStack clouds
• auto discover features
• only OpenStack public interfaces
• Black box testing
• can be reused as load generation
• self tested
• around 2000 functional tests run on each commit iterations.
Tempest

25. Chmouel Boudjnah - @chmouel - eNovance
WhatTheStack? A Tool Based on Tempest to Verify your Deployments

26. Chmouel Boudjnah - @chmouel - eNovance
• external CI plug into openstack
infrastructure.
• able to vote +1 but not -1
• Usually used by vendors for drivers
testing in Neutron / Nova / Cinder
• Used as well for other distros (only
RedHat at the moment).
External CI

27. Chmouel Boudjnah - @chmouel - eNovance
Performances testing

28. Chmouel Boudjnah - @chmouel - eNovance
Performance testing moto
One shall always testing the underlying hardware
before even thinking about performance testing.
Erwan Velu - eNovance Tester allumni

29. Chmouel Boudjnah - @chmouel - eNovance
• at eNovance we have a tool called AHC which benchmark
underlying platform before every deliveries.
• tempest can be used for performance testing of the
software itself.
• There is OpenStack Rally as well a tool that automate the
full thing with nicely generated reports and such.
• Highly dependent of your deployment (i.e: your hardware).
Performance testing.

30. Chmouel Boudjnah - @chmouel - eNovance
That’s it