Unit testing on embedded target with C++Test


Published on

Parasoft delivers a complete framework to create, manage, and extract greater value from unit tests. We help you exercise and test an incomplete system—enabling you to identify problems when they are least difficult, costly, and time-consuming to fix. This reduces the length and cost of downstream processes such as debugging. Moreover, since all tests are written at the unit level, the test suite can be run independent of the complete system. This allows you to isolate code behavior changes, reduces setup complexities, and makes it practical to execute the test suite on a daily basis.

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Unit testing on embedded target with C++Test

  1. 1. Unit testing in embedded environments with Parasoft C++test
  2. 2. Agenda Introduction  What are the embedded systems  What is Unit Testing Unit Testing With C++test  General flow  Test cases generation Unit Testing in Embedded Environments  General flow  Runtime requirements  Building test executable  Pros, Cons, limitations Questions ....
  3. 3. Introduction
  4. 4. What is an embedded system ?An embedded system is a combination of computerhardware and the software, and perhaps additionalmechanical or other parts, designed to perform aspecific function. (Michael Barr - Programming Embedded Systems)
  5. 5. A generic embedded system Memory Functional devices Inputs Processor Outputs
  6. 6. Example 1 - VCR VCR can be made of:  8 bit processor  ROM and RAM  Mechanical components  Analog modules for signal processing  Platform targeted operating system
  7. 7. Example 2 - Opportunity Opportunity:  CPU RAD6000 (RISC)  128 MB DRAM  3 MB EEPROM  256 MB FLASH  Running on VxWORKS (WindRiver) operating system
  8. 8. Example 3 - Pendolino Composed of many sub systems running on different hardware platforms and different operating systems
  9. 9. Development process Host Machine CPU deployment Cross development environment Target Platform MEM
  10. 10. Unit TestingIn computer programming, a unit test is aprocedure used to validate that a particular moduleof source code is working properly. The procedure isto write test cases for all functions and methods sothat whenever a change causes a regression, it canbe quickly identified and fixed. (www.wikipedia.org)
  11. 11. Unit testing with C++test-test flow-test harness architecture-test cases-regression testing
  12. 12. Unit Testing with C++test Automatic test cases generation Test cases in a form of source code Coverage metrics Post-Conditions recording for regression testing Automatic Stubs configuration Automatic test flow management Command line mode Convenient reporting system
  13. 13. From source code to test results test results C++test instrumented source Parser user source code Instr Generator Test Cases test cases Test Generator source Executable runtime librarylibraries Libraries Scanner stub source Stub Manager
  14. 14. Tests architecture - example #include <header> int funcB(); int funcA(int b) { int a = funcB(); return a + b; }
  15. 15. Tests architecture - example Test Executable test runner main() { instrumented code call_test_case Int funcA(int a) { .....} instr_callback(); int b = funcB_stub(); instr_callback(); return a + b; } test_case { pre_cond Stubs funcA() funcB_stub() { post_cond return 0; } }
  16. 16. Test cases
  17. 17. Test cases architecture Test case is composed ofTestSuite_MyClass::test_foo_0() three sections:{ //PreCondition initialization int arg_1 = 0; MyClass testObject();  Pre conditions setup //Tested function call  Tested function call int ret = testObject.func(arg1);  Post conditions //PostConditions validation validation CPPTEST_ASSERT(ret == 1)}
  18. 18. Automatic test cases generationC++tests test cases generator automaticallyproduces source elements for:Initializing pre-conditions like: function arguments,global variables, this object for tested functionCalling the tested function or methodValidating post-conditions (achieved test result)
  19. 19. Initialization strategies Simple (built-in) types:  Integers (-1,0,1, min, max value)  Floating point (-1.0, 0.0, 1.0, max, min positive, max, min negative)  Heuristic values Complex types (user defined types) C++test will use available constructors and generate code necessary to initialize constructors arguments.
  20. 20. Initialization strategies Pointer types:  NULL pointer  address of a local object created on stack  address of an object created with “new” (on heap) Reference types:  local object created on stack  object created using operator “new” Heuristic values for simple types:  int  float  char *
  21. 21. Post conditionsC++test provides macros and routines for controllinghow source code test results are collected andvalidated:Test case post-conditions are controlled viaassertion macros (similar to CppUnit)  CPPTEST_ASSERT_INTEGER_EQUAL(expected,actual)Assertions perform a condition validation and sendresults back to C++test
  22. 22. Test case example with post-conditions /* CPPTEST_TEST_CASE_BEGIN test_bufLen_0 */ void TestSuite_bufLen_0::test_bufLen_0() { /* Pre-condition initialization */ /* Initializing argument 0 (this) */ /* Initializing constructor argument 1 (fill) */ char _fill_0 = 001; ::Data _cpptest_TestObject (_fill_0); /* Tested function call */ int _return = _cpptest_TestObject.bufLen(); /* Post-condition check */ CPPTEST_ASSERT_INTEGER_EQUAL(4, ( _return )) CPPTEST_ASSERT(( _cpptest_TestObject._data ) != 0) }
  23. 23. Regression test cases generation subsequent test runs generation Test results Test cases Validation Testing generation First test execution Test cases Regression test cases
  24. 24. Stubs
  25. 25. Stubs – handling missing definitions Instrumented Libraries source code & object files Auto generated stubs int func2() { return 0; }extern int func2();int func1(int b) User defined{ stubs int a = func2(); return a + b;}
  26. 26. Stubs in C++test Stubs are used:  when testing in separation from other modules  when application module is not yet ready  when hardware module is not yet ready C++test provides three modes for stubbed function:  original  user-defined  auto-generated Stubs configuration is managed from C++test GUI and persisted in C++test project
  27. 27. Unit Testing in embedded environments-test flow-runtime library-problems & limitations-why to unit test on target ?
  28. 28. Unit testing in embedded environmentC++test runtime Cross Compilerruntime linker/locatorlibrary library source Test executable Test Test harnessharness binariessource deploy code host - target communication results
  29. 29. Unit testing in embedded environment Host Development env Target C++test deploy Test executable Runtime library Listening communication agent communication channel module
  30. 30. C++test runtime library
  31. 31. C++test runtime library Provided in form of source code Implemented in plain C Needs to be cross-compiled for specific target platform Contains definition of communication channel Provided implementations:  file  TCP/IP sockets Designed to be easily customizable
  32. 32. C++test runtime libraryCommunication module
  33. 33. Target-to-host communication module Custom implementation of communication channel can be plugged in Defining communication channel requires implementing four methods:  cpptestInitializeCommunication(...);  cpptestFinalizeCommunication(...);  cpptestSendData(const char* data, unsigned size); All data transfer operations, performed inside C+ +test runtime library are implemented in terms of above interface
  34. 34. Collecting results on target Host Target C++test Test executable Manual results Runtime library transfer File communication System module Local Storage
  35. 35. “On the fly” results transfer Host Target C++test TCP/IP Sockets Test executable RS 232 RS 485 I2C ICE Runtime library File Listening communication System agent communication channel module
  36. 36. C++test runtime libraryExceptions handling module
  37. 37. Exceptions handling  C++test uses Test cases execution sequence exceptions handling facilities (try, catch, Test setjmp, longjmp, case 1 signal) to recover if exception is thrown exception during test sequencehandling enabled TestWith exceptions case 2 (e.g divide by zero)  If target platform does not provide exceptions handling, test sequence Test is broken on first case n exception and needs to be restarted
  38. 38. Test flow automation
  39. 39. Test flow automation Test sequence can be automatically performed in C++test GUI as well as in command line mode Test sequence consists of test harness preparation, building test executable, deploying test executable, starting it, and collecting results Limitations in automation of testing process can be imposed be development environment C++test provides easy to use XML-based format for defining test framework operations
  40. 40. Problems & Limitations
  41. 41. Problems & limitations Not enough memory on the target device to store test harness and tested code may block or limit testing capabilities Compilation without debug information Original file Min Instr Med Instr Max Instr GNU GCC 3.2 22 KB 22 KB (0%) 29 KB (30%) 41 KB (80%) MSVC++ 7.1 33 KB 33 KB (0%) 41 KB (24%) 69 KB (100%)Tornado simpc 23 KB 23 KB (0%) 30 KB (30%) 41 KB (78%)
  42. 42. Problems & limitations Lack of communication channel may effect in weak test automation Missing support for exceptions handling may increase the number of runs necessary to execute test cases scheduled for test session. Additional amount of work required during development process
  43. 43. Why to unit test on target ?
  44. 44. Pros of unit testing on target All well known positives from host-based unit testing (early bugs detection, increased code robustness, ...) Possibility of detecting hardware problems Possibility of detecting software/hardware interaction problems Easy stubbing of missing software (also hardware) modules Quick and automated generation of regression suites
  45. 45. Unit testing in embedded environment - examplestatic int iTemperatures[2];  “Critical section”void interrupt vReadTemperatures(void) { related bugs are hard ITemperatures[0] = /*Read sensor 1*/ to detect during host- ITemperatures[1] = /*Read sensor 2*/ based unit testing}  Target-based unitbool testSensors() { testing highly increases int iTemp0, iTemp1; probability of catching iTemp0 = ITemperatures[0]; this kind of problems, iTemp1 = ITemperatures[1]; if (iTemp0 != iTemp1) { // Alarm !!! however it does not return -1; provide 100% certainty } of catching them. return 0;}
  46. 46. Unit testing in embedded environment - example  Module for driving the position of functional mechanical equipment servo  Hardware contains “stuck-at- encoder zero” problem between the encoder and board interface int set_position(int angl) { if (angl < MIN || angl > MAX) { return INCORRECT_ANGL; } return run_servo(angl); }
  47. 47. Epilog C++test offers a high level of automation for taskswhich are usually performed manually, especially inembedded environments, therefore very often it lets us give the positive answer for a question: “to test ? or not to test ?”