Unit testing in embedded environments         with Parasoft C++test
Agenda Introduction    What are the embedded systems    What is Unit Testing Unit Testing With C++test    General flo...
Introduction
What is an embedded system ?An embedded system is a combination of computerhardware and the software, and perhaps addition...
A generic embedded system                 Memory                                       Functional devices      Inputs     ...
Example 1 - VCR                  VCR can be made of:                   8 bit processor                   ROM and RAM    ...
Example 2 - Opportunity                          Opportunity:                           CPU RAD6000                      ...
Example 3 - Pendolino                        Composed of many                        sub systems running                  ...
Development process    Host Machine                     CPU                       deployment   Cross development      envi...
Unit TestingIn computer programming, a unit test is aprocedure used to validate that a particular moduleof source code is ...
Unit testing with C++test-test flow-test harness architecture-test cases-regression testing
Unit Testing with C++test   Automatic test cases generation   Test cases in a form of source code   Coverage metrics  ...
From source code to test results                                test results            C++test                           ...
Tests architecture - example  #include <header>  int funcB();  int funcA(int b)  {     int a = funcB();     return a + b;  }
Tests architecture - example                                      Test Executable   test runner       main() {            ...
Test cases
Test cases architecture                                       Test case is composed ofTestSuite_MyClass::test_foo_0()     ...
Automatic test cases generationC++tests test cases generator automaticallyproduces source elements for:Initializing pre-c...
Initialization strategies Simple (built-in) types:    Integers (-1,0,1, min, max value)    Floating point (-1.0, 0.0, 1...
Initialization strategies Pointer types:    NULL pointer    address of a local object created on stack    address of a...
Post conditionsC++test provides macros and routines for controllinghow source code test results are collected andvalidated...
Test case example with post-conditions      /* CPPTEST_TEST_CASE_BEGIN test_bufLen_0 */      void TestSuite_bufLen_0::test...
Regression test cases generation   subsequent test runs               generation                      Test results       T...
Stubs
Stubs – handling missing definitions                                    Instrumented     Libraries                       s...
Stubs in C++test Stubs are used:   when testing in separation from other modules   when application module is not yet r...
Unit Testing in embedded environments-test flow-runtime library-problems & limitations-why to unit test on target ?
Unit testing in embedded environmentC++test                                  runtime           Cross Compilerruntime      ...
Unit testing in embedded environment   Host    Development env                               Target           C++test     ...
C++test runtime library
C++test runtime library Provided in form of source code Implemented in plain C Needs to be cross-compiled for specific ...
C++test runtime libraryCommunication module
Target-to-host communication module Custom implementation of communication channel  can be plugged in Defining communica...
Collecting results on target  Host                                       Target           C++test                         ...
“On the fly” results transfer  Host                                                   Target           C++test            ...
C++test runtime libraryExceptions handling module
Exceptions handling                                           C++test uses  Test cases execution        sequence         ...
Test flow automation
Test flow automation Test sequence can be automatically performed in  C++test GUI as well as in command line mode Test s...
Problems & Limitations
Problems & limitations Not enough memory on the target device to store  test harness and tested code may block or limit  ...
Problems & limitations Lack of communication channel may effect in weak  test automation Missing support for exceptions ...
Why to unit test on target ?
Pros of unit testing on target All well known positives from host-based unit  testing (early bugs detection, increased co...
Unit testing in embedded environment - examplestatic int iTemperatures[2];                                            “Cr...
Unit testing in embedded environment - example                          Module for driving the position                  ...
Epilog C++test offers a high level of automation for taskswhich are usually performed manually, especially inembedded envi...
Upcoming SlideShare
Loading in...5
×

Unit testing on embedded target with C++Test

1,424

Published on

Parasoft delivers a complete framework to create, manage, and extract greater value from unit tests. We help you exercise and test an incomplete system—enabling you to identify problems when they are least difficult, costly, and time-consuming to fix. This reduces the length and cost of downstream processes such as debugging. Moreover, since all tests are written at the unit level, the test suite can be run independent of the complete system. This allows you to isolate code behavior changes, reduces setup complexities, and makes it practical to execute the test suite on a daily basis.

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,424
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
37
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Unit testing on embedded target with C++Test"

  1. 1. Unit testing in embedded environments with Parasoft C++test
  2. 2. Agenda Introduction  What are the embedded systems  What is Unit Testing Unit Testing With C++test  General flow  Test cases generation Unit Testing in Embedded Environments  General flow  Runtime requirements  Building test executable  Pros, Cons, limitations Questions ....
  3. 3. Introduction
  4. 4. What is an embedded system ?An embedded system is a combination of computerhardware and the software, and perhaps additionalmechanical or other parts, designed to perform aspecific function. (Michael Barr - Programming Embedded Systems)
  5. 5. A generic embedded system Memory Functional devices Inputs Processor Outputs
  6. 6. Example 1 - VCR VCR can be made of:  8 bit processor  ROM and RAM  Mechanical components  Analog modules for signal processing  Platform targeted operating system
  7. 7. Example 2 - Opportunity Opportunity:  CPU RAD6000 (RISC)  128 MB DRAM  3 MB EEPROM  256 MB FLASH  Running on VxWORKS (WindRiver) operating system
  8. 8. Example 3 - Pendolino Composed of many sub systems running on different hardware platforms and different operating systems
  9. 9. Development process Host Machine CPU deployment Cross development environment Target Platform MEM
  10. 10. Unit TestingIn computer programming, a unit test is aprocedure used to validate that a particular moduleof source code is working properly. The procedure isto write test cases for all functions and methods sothat whenever a change causes a regression, it canbe quickly identified and fixed. (www.wikipedia.org)
  11. 11. Unit testing with C++test-test flow-test harness architecture-test cases-regression testing
  12. 12. Unit Testing with C++test Automatic test cases generation Test cases in a form of source code Coverage metrics Post-Conditions recording for regression testing Automatic Stubs configuration Automatic test flow management Command line mode Convenient reporting system
  13. 13. From source code to test results test results C++test instrumented source Parser user source code Instr Generator Test Cases test cases Test Generator source Executable runtime librarylibraries Libraries Scanner stub source Stub Manager
  14. 14. Tests architecture - example #include <header> int funcB(); int funcA(int b) { int a = funcB(); return a + b; }
  15. 15. Tests architecture - example Test Executable test runner main() { instrumented code call_test_case Int funcA(int a) { .....} instr_callback(); int b = funcB_stub(); instr_callback(); return a + b; } test_case { pre_cond Stubs funcA() funcB_stub() { post_cond return 0; } }
  16. 16. Test cases
  17. 17. Test cases architecture Test case is composed ofTestSuite_MyClass::test_foo_0() three sections:{ //PreCondition initialization int arg_1 = 0; MyClass testObject();  Pre conditions setup //Tested function call  Tested function call int ret = testObject.func(arg1);  Post conditions //PostConditions validation validation CPPTEST_ASSERT(ret == 1)}
  18. 18. Automatic test cases generationC++tests test cases generator automaticallyproduces source elements for:Initializing pre-conditions like: function arguments,global variables, this object for tested functionCalling the tested function or methodValidating post-conditions (achieved test result)
  19. 19. Initialization strategies Simple (built-in) types:  Integers (-1,0,1, min, max value)  Floating point (-1.0, 0.0, 1.0, max, min positive, max, min negative)  Heuristic values Complex types (user defined types) C++test will use available constructors and generate code necessary to initialize constructors arguments.
  20. 20. Initialization strategies Pointer types:  NULL pointer  address of a local object created on stack  address of an object created with “new” (on heap) Reference types:  local object created on stack  object created using operator “new” Heuristic values for simple types:  int  float  char *
  21. 21. Post conditionsC++test provides macros and routines for controllinghow source code test results are collected andvalidated:Test case post-conditions are controlled viaassertion macros (similar to CppUnit)  CPPTEST_ASSERT_INTEGER_EQUAL(expected,actual)Assertions perform a condition validation and sendresults back to C++test
  22. 22. Test case example with post-conditions /* CPPTEST_TEST_CASE_BEGIN test_bufLen_0 */ void TestSuite_bufLen_0::test_bufLen_0() { /* Pre-condition initialization */ /* Initializing argument 0 (this) */ /* Initializing constructor argument 1 (fill) */ char _fill_0 = 001; ::Data _cpptest_TestObject (_fill_0); /* Tested function call */ int _return = _cpptest_TestObject.bufLen(); /* Post-condition check */ CPPTEST_ASSERT_INTEGER_EQUAL(4, ( _return )) CPPTEST_ASSERT(( _cpptest_TestObject._data ) != 0) }
  23. 23. Regression test cases generation subsequent test runs generation Test results Test cases Validation Testing generation First test execution Test cases Regression test cases
  24. 24. Stubs
  25. 25. Stubs – handling missing definitions Instrumented Libraries source code & object files Auto generated stubs int func2() { return 0; }extern int func2();int func1(int b) User defined{ stubs int a = func2(); return a + b;}
  26. 26. Stubs in C++test Stubs are used:  when testing in separation from other modules  when application module is not yet ready  when hardware module is not yet ready C++test provides three modes for stubbed function:  original  user-defined  auto-generated Stubs configuration is managed from C++test GUI and persisted in C++test project
  27. 27. Unit Testing in embedded environments-test flow-runtime library-problems & limitations-why to unit test on target ?
  28. 28. Unit testing in embedded environmentC++test runtime Cross Compilerruntime linker/locatorlibrary library source Test executable Test Test harnessharness binariessource deploy code host - target communication results
  29. 29. Unit testing in embedded environment Host Development env Target C++test deploy Test executable Runtime library Listening communication agent communication channel module
  30. 30. C++test runtime library
  31. 31. C++test runtime library Provided in form of source code Implemented in plain C Needs to be cross-compiled for specific target platform Contains definition of communication channel Provided implementations:  file  TCP/IP sockets Designed to be easily customizable
  32. 32. C++test runtime libraryCommunication module
  33. 33. Target-to-host communication module Custom implementation of communication channel can be plugged in Defining communication channel requires implementing four methods:  cpptestInitializeCommunication(...);  cpptestFinalizeCommunication(...);  cpptestSendData(const char* data, unsigned size); All data transfer operations, performed inside C+ +test runtime library are implemented in terms of above interface
  34. 34. Collecting results on target Host Target C++test Test executable Manual results Runtime library transfer File communication System module Local Storage
  35. 35. “On the fly” results transfer Host Target C++test TCP/IP Sockets Test executable RS 232 RS 485 I2C ICE Runtime library File Listening communication System agent communication channel module
  36. 36. C++test runtime libraryExceptions handling module
  37. 37. Exceptions handling  C++test uses Test cases execution sequence exceptions handling facilities (try, catch, Test setjmp, longjmp, case 1 signal) to recover if exception is thrown exception during test sequencehandling enabled TestWith exceptions case 2 (e.g divide by zero)  If target platform does not provide exceptions handling, test sequence Test is broken on first case n exception and needs to be restarted
  38. 38. Test flow automation
  39. 39. Test flow automation Test sequence can be automatically performed in C++test GUI as well as in command line mode Test sequence consists of test harness preparation, building test executable, deploying test executable, starting it, and collecting results Limitations in automation of testing process can be imposed be development environment C++test provides easy to use XML-based format for defining test framework operations
  40. 40. Problems & Limitations
  41. 41. Problems & limitations Not enough memory on the target device to store test harness and tested code may block or limit testing capabilities Compilation without debug information Original file Min Instr Med Instr Max Instr GNU GCC 3.2 22 KB 22 KB (0%) 29 KB (30%) 41 KB (80%) MSVC++ 7.1 33 KB 33 KB (0%) 41 KB (24%) 69 KB (100%)Tornado simpc 23 KB 23 KB (0%) 30 KB (30%) 41 KB (78%)
  42. 42. Problems & limitations Lack of communication channel may effect in weak test automation Missing support for exceptions handling may increase the number of runs necessary to execute test cases scheduled for test session. Additional amount of work required during development process
  43. 43. Why to unit test on target ?
  44. 44. Pros of unit testing on target All well known positives from host-based unit testing (early bugs detection, increased code robustness, ...) Possibility of detecting hardware problems Possibility of detecting software/hardware interaction problems Easy stubbing of missing software (also hardware) modules Quick and automated generation of regression suites
  45. 45. Unit testing in embedded environment - examplestatic int iTemperatures[2];  “Critical section”void interrupt vReadTemperatures(void) { related bugs are hard ITemperatures[0] = /*Read sensor 1*/ to detect during host- ITemperatures[1] = /*Read sensor 2*/ based unit testing}  Target-based unitbool testSensors() { testing highly increases int iTemp0, iTemp1; probability of catching iTemp0 = ITemperatures[0]; this kind of problems, iTemp1 = ITemperatures[1]; if (iTemp0 != iTemp1) { // Alarm !!! however it does not return -1; provide 100% certainty } of catching them. return 0;}
  46. 46. Unit testing in embedded environment - example  Module for driving the position of functional mechanical equipment servo  Hardware contains “stuck-at- encoder zero” problem between the encoder and board interface int set_position(int angl) { if (angl < MIN || angl > MAX) { return INCORRECT_ANGL; } return run_servo(angl); }
  47. 47. Epilog C++test offers a high level of automation for taskswhich are usually performed manually, especially inembedded environments, therefore very often it lets us give the positive answer for a question: “to test ? or not to test ?”
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×