A tutorial about Testing in ROS that I gave to the members of my lab. Code Examples and Exercises in: https://github.com/VGonPa/ros-testing-tutorial

1. Test Driven
Development in ROS
A Gentle Introduction
to Gtest
Víctor González Pacheco
victor.gonzalez.pacheco@gmail.com
March 2014

2. Outline
What to expect
Motivation
Testing Levels
Writing your first tests
Advice on making good tests
Resources
Exercise

3. Start with an example
Download material at:
https://github.com/VGonPa/ros-testing-seminar
git clone https://github.com/VGonPa/ros-testing-seminar.git

4. What to expect
from this seminar

5. What to expect
“Will I become a TDD master with this
seminar?”

6. “Actually you might, eventually,
if you start writing tests today!”

7. What to expect
You will learn where to start,
the basics, and where to find more
information and help.

8. Motivation

9. Motivation
"Tests + Programming" is faster
than just "Programming" [1]

10. Motivation
SW Evolves
Requirements change
Bug corrections
Optimizations
Improvements on design
How do you know if a change in your code
didn’t break your previous work?

11. Motivation
NOT testing is very dangerous:
“(Making large changes without tests) is like doing
aerial gymnastics without a net.” [1]
The later you detect a bug, the more expensive is to
fix it:
A bug detected in production may be REALLY expensive
Let alone if tomorrow you have a demo!

12. Motivation
Some advantages of writing tests
It forces you to design better code
Faster incremental upgrades
A clear metric of your progress
It lets you refactor with greater confidence
Prevents recurring bugs
It enables you to blame others!
Other people can work with your code more easily

13. Motivation
The worst disadvantage
It will affect your thinking process...
...and your programming style.
But you will become a better
programmer!

14. Motivation
“I don’t have time to write tests!”
Had you done the tests before,
now you'd have time to do them!
More tests = less time debugging
More tests = more time to program features

15. Test Driven Development (TTD)
The process
Before programming, write tests
Write only the needed code to make the tests
pass
Refactor to eliminate duplicated code, etc.
Repeat

16. Test Driven Development (TTD)
“Do I really need to write tests first?”
It’s much better to write them first,
but if you don’t, be sure that
your code is properly tested

17. Testing Levels

18. Imagine a simple scenario
N1 N2

19. Each node has its own code...
N1 N2

20. This is better...
N1 N2

21. There you have the first level
N1 N2
Library Unit Testing

22. Library Unit testing
Library Unit Testing Test only your libraries
N1
Code must be ROS Agnostic
GTest/Unittest(python)

23. Library Unit testing
N1
Test each component
separately

24. Library Unit Testing
N1
Test for:
common cases
extreme cases
Input errors
methods pre and post
conditions

25. Now let’s test the node itself
Node Level
Testing
N1 N2

26. Node Level Unit testing
Node Level
Testing
N1
Test Node start/shutdown
Test Node external API
services, published topics, subscribed topics
(params?)
RosTest + GTest/Unittest

27. And, finally, the whole system
Integration/Regression Testing
N1 N2

28. Integration Testing
N1 N2
Test multiple nodes working as
expected
RosTest + GTest/Unittest

29. Testing Levels: Overview
Integration/Regression Testing
N1 N2
Library Unit
Testing
Node Level
Testing

30. Writing your first tests
in ROS

31. Gtest and rostest
ROS provides two tools for executing tests:
gtest and rostest

32. Gtest and rostest
GTest
Google’s tool for unit testing
Regular cpp files
Executed with any of the following:
./my_test_file
make test # needs macro in CMakeLists.txt
rosmake <package_name> -t # needs macro in CMakeLists.txt
http://wiki.ros.org/gtest

33. Gtest and rostest
GTest
Gtest tutorials are quite good.
We won’t cover them here.
Read them if you don’t know how to write Gtests.

34. Gtest and rostest
rostest
roslaunch extension to launch tests
.launch syntax with added <test> tag
executed with any of the following:
rostest <pgk_name> <test_name>.test
make test # needs macro in CMakeLists.txt
rosmake <package_name> -t # needs macro in CMakeLists.txt
http://wiki.ros.org/rostest
http://wiki.ros.org/rostest/Writing
http://wiki.ros.org/roslaunch/XML/test <------ <test> tag reference

35. Where should I use each one?
Integration/Regression
Node
N1 N2
Library
Gtest
Rostest (+ Gtest)

36. Package structure with tests
my_ros_pkg/
CMakeLists.txt
bin/
build/
msg/
...
src/
test/ <--- gtests go here
test/ <--- rostests go here

37. Installation
GTest
Install: bash> sudo apt-get install libgtest-dev
In your header files:
#include <gtest/gtest.h>
GMock
Install bash> sudo apt-get install google-mock
In your header files:
#include <gmock/gmock.h>

38. Gtest CMakeLists.txt
# add gtest
rosbuild_add_gtest(test/my_test test/mytest.cpp
[other sources])
# link required libraries
target_link_libraries(test/mytest linked_libraries)

39. rostest CMakeLists.txt
# add the test executable,
# keep it from being built by "make all"
rosbuild_add_executable(test_mynode EXCLUDE_FROM_ALL
src/test/test_mynode.cpp)
# Link test_mynode against gtest
# and add a dependency to the "test" target
rosbuild_add_gtest_build_flags(test_mynode)
# Make sure rostest launches test/mynode.test during "make test"
rosbuild_add_rostest(test/mynode.test)

40. Let’s see an example
Checkout the package
rostest_node_example from the material

41. Some advise
How to make good tests

42. What to test
Example: max(list_of_ints)
Normal cases
max([10,3,0,-1,8])
Extreme cases
max([3,3,3,3,3,3])
max(2)
Error cases
max([‘aaa’,3,nan,None])
max([])

43. What not to test
The test itself
Modules that cannot be broken (or that there is no
solution):
System calls
Hardware failures
Modules from which your code depends on:
Standard Libraries, modules written by others, etc.
They already have (or should have) their own tests
Exhaustive tests

44. Writing more “testable” code
Sometimes it’s difficult to test a component
in isolation
Dependencies between components
Some components might be on the network (eg. sockets)
Some components might needuser input
Some comopnents might just be slow to test

45. Writing more “testable” code
Solution: Break dependencies
Program against interfaces
Specify dependencies in the constructor
Use mocks in your tests

46. Common features of good tests
A good test should be:
Independent
1. You do not need to read other tests to understand what a test does
2. If a test fails, it should be easy to find the bug
3. Each test focuses on a single aspect
Repetible
Quick: Use mocks
Small: Enables you to easily spot bugs.
Big tests functions have many parts affecting each other

47. Beware the Unit-Integration Chimera!
It will eat your productivity!
Unit Testing
Sweet spot
Integration Testing
Sweet spot

48. Bibliography and
Resources

49. Bibliography and Resources
B. Eckel, "Thinking in C++ Volume 2". (online version)
Google C++ Testing Framework project
Project: http://code.google.com/p/googletest/
Guía inicial: http://code.google.com/p/googletest/wiki/V1_6_Primer
Avanzado: http://code.google.com/p/googletest/wiki/V1_6_AdvancedGuide
Ejemplos: http://code.google.com/p/googletest/wiki/V1_6_Samples
Dependency injection, mocks:
Google Mock: http://code.google.com/p/googlemock/

50. Bibliography and Resources
ROS Unit Testing: http://wiki.ros.org/UnitTesting
ROS GTest: http://wiki.ros.org/gtest
ROS rostest: http://wiki.ros.org/rostest

51. Bibliography and Resources
This seminar has been greatly inspired from this talk
from Zhanyong Wan:
Effective C++ Testing Using Google Test
There you will find more information regarding Gtest
and Unit testing in general

52. Questions?
https://github.com/VGonPa/ros-testing-seminar

53. Excercise

54. /do_increment /counter /factorial
counter_node factorial_node
1. Counter node’s counter should be initiated by
parameter to any number
2. Factorial calculates factorial of counter
3. Test the library, node and integration using gtest +
rostests

55. This seminar is licensed under the Creative Commons license
CC Attribution 4.0 International
You are free to use and adapt this presentation as long as you give credit to the author.
More information can be found here:
http://creativecommons.org/licenses/by/4.0/