This document summarizes Chmouel Boudjnah's presentation on software testing in OpenStack. It discusses different types of testing like unit testing, integration testing, and performance testing. It also covers testing methodologies like test-driven development and behavior-driven development. Finally, it provides examples of testing tools used in OpenStack like Tempest, static code analysis with flake8, and performance benchmarking with Rally.
8. Chmouel Boudjnah - @chmouel - eNovance
• Code a bit the test.
• Start a bit of coding.
• Make the test pass.
• Repeat until the feature is implemented.
TDD
9. Chmouel Boudjnah - @chmouel - eNovance
• Business Driven Development
Introducing BDD
10. Chmouel Boudjnah - @chmouel - eNovance
• Business Driven Development
• Behaviour Driven Development
Introducing BDD
11. Chmouel Boudjnah - @chmouel - eNovance
• Business Driven Development
• Behaviour Driven Development
• BS Driven Development (i.e: Vaporware)
Introducing BDD
12. Chmouel Boudjnah - @chmouel - eNovance
• Business Driven Development
• Behaviour Driven Development
• BS Driven Development
• Some people get it confused with “Bourrinage
Driven Development”
Introducing BDD
13. Chmouel Boudjnah - @chmouel - eNovance
Methodology
“You shall not pass a commit before each iterations has been
fully tested”
15. Chmouel Boudjnah - @chmouel - eNovance
The five year old technology mashup.
16. Chmouel Boudjnah - @chmouel - eNovance
The five year old technology mashup.
"well it is just like when you turn the wheel before we
are going on a ride you test that it turns properly
before actually riding it. If it didn’t turn then that means
I may going to have a crash and have a big bobo.”
Noah Boudjnah
17. Chmouel Boudjnah - @chmouel - eNovance
• Swift 2371 unit tests
• Ceilometer 2583
• Heat 2710
• Keystone 3216
• Nova 10653
In OpenStack
18. Chmouel Boudjnah - @chmouel - eNovance
• Mostly for code styling
• flake8 for pep8 and common
errors
• Openstack hacking
Static code analysis.
24. Chmouel Boudjnah - @chmouel - eNovance
• run against any OpenStack clouds
• auto discover features
• only OpenStack public interfaces
• Black box testing
• can be reused as load generation
• self tested
• around 2000 functional tests run on each commit iterations.
Tempest
25. Chmouel Boudjnah - @chmouel - eNovance
WhatTheStack? A Tool Based on Tempest to Verify your Deployments
26. Chmouel Boudjnah - @chmouel - eNovance
• external CI plug into openstack
infrastructure.
• able to vote +1 but not -1
• Usually used by vendors for drivers
testing in Neutron / Nova / Cinder
• Used as well for other distros (only
RedHat at the moment).
External CI
28. Chmouel Boudjnah - @chmouel - eNovance
Performance testing moto
One shall always testing the underlying hardware
before even thinking about performance testing.
Erwan Velu - eNovance Tester allumni
29. Chmouel Boudjnah - @chmouel - eNovance
• at eNovance we have a tool called AHC which benchmark
underlying platform before every deliveries.
• tempest can be used for performance testing of the
software itself.
• There is OpenStack Rally as well a tool that automate the
full thing with nicely generated reports and such.
• Highly dependent of your deployment (i.e: your hardware).
Performance testing.