Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

State Machine Based Testing

452 views

Published on

Robot simulators, like the MORSE project, provide a safe and readily available environment for robot system testing, reducing the e ffort for testing drastically. In principle, simulation testing is automatable, and thus a good target for Continuous Integration (CI) testing. However, so far, high-level scenario tests still require complex component setup and confi guration before they can be run in the simulator. An added complication is, that there is no standard for starting, con figuring, or monitoring software components on todays robots. Often, high-level tests are carried out manually, implementing a tailored solution, e.g, via shell scripts or launch files, for a speci c system setup. Besides the e ort of manual execution and supervision, current tests mostly do not take timing and orchestration, i.e., required process start-up sequence, into account. Furthermore, successful execution of components is not verfi ed, which might lead to subsequent errors during the execution chain. Most importantly, all this knowledge about the test and its environment is implicit, often hidden in the actual implementation of the tailored test suite. To overcome these issues, this contribution introduces a generic and con gfiurable state-machine based process to automate a) environment setup, b) system bootstrapping, c) system tests, d) result assessment, and e) exit and clean-up strategy.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

State Machine Based Testing

  1. 1. 1
  2. 2. State Machine Based TestingFlorian Lier, Norman Köster, Ingo Lütkebohle, and Sven Wachsmuth2
  3. 3. 3 talks mentioned CI already ;)3
  4. 4. (my) MotivationBesides working in other research projects as a PhD student...4
  5. 5. (my) Motivation...my motivation is deeply grounded in the...5
  6. 6. ROBOCUP6
  7. 7. Source: http://www.youtube.com/watch?v=YpjeNa8BAYg7
  8. 8. (my) MotivationRobots operating in domestic environments... rich interaction with humans... object detection and recognition... cluttered “real world” environment... speech recognition and synthesis... autonomous navigation and localization... competitive event, which is “thrilling”8
  9. 9. (my) MotivationSource: http://www.cit-ec.de/sites/www.cit-ec.de/files/TDP_ToBI_2011_0.pdfX
  10. 10. (my) MotivationThe team “ToBI” consists of... 6-10 BA/MA students, they earn CPs!... 1-3 PhDs, Postdocs supervision... new students, every year...... constantly improving/evolving system... hackathons, during regular term... “agile” on-site all around the world9
  11. 11. Coding at actual event10
  12. 12. Which is quite stressful, sometimes11
  13. 13. Agile methods are appliedX
  14. 14. Problem IApproximately 10 developers but... only 1 Robot to test with... > 10 source code repos to “watch”... frequent changes everywhere... local file system vs. robot file system12
  15. 15. Problem I“The task starts in 5 minutes”13
  16. 16. Problem I“Oh wait, I didn’t commit yet...”14
  17. 17. Problem I“Ok, it’s checked-in send the robot to the arena!”15
  18. 18. Problem I“Waaaait, I just realized I broke the task!!!”(RoboCup classic)16
  19. 19. Solution I17
  20. 20. Solution IYes! A Continuous Integration Server... monitors all build statuses (red, green)... checks robot vs. local file system sync... delivers instant feedback... automated build process, static checks... de-stresses situation, best-practice, etc18
  21. 21. Problem II“I tested the task in the area everything ..., but the robot doesn’t move!”19
  22. 22. Solution IIYes! Simulation for everyone!... but,... > 15 components to start, manually... operate the simulator, manually... based on subjective measurements... first test prob. != second test... timing, sequence, operator, machine... you name it20
  23. 23. “Simulation?”21
  24. 24. ProposalYes! Combine CI and simulation testing... serialize building an testing automatically... start the simulation and all required components... deliver feedback and results... frequent checks (per check-in)... works also outside RoboCup ;)22
  25. 25. ApproachCreate a state-machine for simulation tests!23
  26. 26. Goals of SM based testingNo “tailored” solution for a specific systemMinimize the effort, let the machine do the workEnable developers to design a well-structured testExplicit environment setup and configurationTransparent orchestration, repeatable and comparableVerify system start-up and tear-down24
  27. 27. 25
  28. 28. 26
  29. 29. 27
  30. 30. BenefitsWell-structured, explicit definition of env. and comp.Fully automatable (CI enabled and CLOUD enabled)Repeatable, exchangeable, composable and comparableExecution checks, early exit (error states)Timing and sequence is taken into accountLanguage, tool, and framework independent (!= roslaunch)(OS independent, not tested on windows yet)Distribution (STRANDS?)28
  31. 31. Benefits // MORSEMorse setup and executionFully automatable (CI enabled and CLOUD enabled)Repeatable, exchangeable, composable and comparableExecution checks, early exit (error states)Timing and sequence is taken into accountLanguage, tool, and framework independent (!= roslaunch)(OS independent, not tested on windows yet)X
  32. 32. Morse Demo: ATRV + ROS, rosbag, rostopic, rxplot29
  33. 33. 30Source: https://vimeo.com/67803733
  34. 34. Thank you!31
  35. 35. Discussion TopicsUseful?Problems: SCXML is pain!MORSE integrationMetricsUnitT integration32
  36. 36. 33

×