Robot simulators, like the MORSE project, provide a safe and readily available environment for robot system testing, reducing the effort for testing drastically. In principle, simulation testing is automatable, and thus a good target for Continuous Integration (CI) testing. However, so far, high-level scenario tests still require complex component setup and configuration before they can be run in the simulator. An added complication is, that there is no standard for starting, configuring, or monitoring software components on todays robots. Often, high-level tests are carried out manually, implementing a tailored solution, e.g, via shell scripts or launch files, for a specic system setup. Besides the eort of manual execution and supervision, current tests mostly do not take timing and orchestration, i.e., required process start-up sequence, into account. Furthermore, successful execution of components is not verfied, which might lead to subsequent errors during the execution chain. Most importantly, all this knowledge about the test and its environment is implicit, often hidden in the actual implementation of the tailored test suite. To overcome these issues, this contribution introduces a generic and congfiurable state-machine based process to automate a) environment setup, b) system bootstrapping, c) system tests, d) result assessment, and e) exit and clean-up strategy.
8. (my) Motivation
Robots operating in domestic environments
... rich interaction with humans
... object detection and recognition
... cluttered “real world” environment
... speech recognition and synthesis
... autonomous navigation and localization
... competitive event, which is “thrilling”
8
10. (my) Motivation
The team “ToBI” consists of
... 6-10 BA/MA students, they earn CPs!
... 1-3 PhDs, Postdocs supervision
... new students, every year...
... constantly improving/evolving system
... hackathons, during regular term
... “agile” on-site all around the world
9
14. Problem I
Approximately 10 developers but
... only 1 Robot to test with
... > 10 source code repos to “watch”
... frequent changes everywhere
... local file system vs. robot file system
12
20. Solution I
Yes! A Continuous Integration Server
... monitors all build statuses (red, green)
... checks robot vs. local file system sync
... delivers instant feedback
... automated build process, static checks
... de-stresses situation, best-practice, etc
18
21. Problem II
“I tested the task in the area everything ..., but the robot doesn’t move!”
19
22. Solution II
Yes! Simulation for everyone!
... but,
... > 15 components to start, manually
... operate the simulator, manually
... based on subjective measurements
... first test prob. != second test
... timing, sequence, operator, machine
... you name it
20
24. Proposal
Yes! Combine CI and simulation testing
... serialize building an testing automatically
... start the simulation and all required components
... deliver feedback and results
... frequent checks (per check-in)
... works also outside RoboCup ;)
22
26. Goals of SM based testing
No “tailored” solution for a specific system
Minimize the effort, let the machine do the work
Enable developers to design a well-structured test
Explicit environment setup and configuration
Transparent orchestration, repeatable and comparable
Verify system start-up and tear-down
24
30. Benefits
Well-structured, explicit definition of env. and comp.
Fully automatable (CI enabled and CLOUD enabled)
Repeatable, exchangeable, composable and comparable
Execution checks, early exit (error states)
Timing and sequence is taken into account
Language, tool, and framework independent (!= roslaunch)
(OS independent, not tested on windows yet)
Distribution (STRANDS?)
28
31. Benefits // MORSE
Morse setup and execution
Fully automatable (CI enabled and CLOUD enabled)
Repeatable, exchangeable, composable and comparable
Execution checks, early exit (error states)
Timing and sequence is taken into account
Language, tool, and framework independent (!= roslaunch)
(OS independent, not tested on windows yet)
X