testIDEA v11.04

RealTime-UnitTest On Object Code LevelGet the Most out of Development and Testing with the Maker of the Blue Box V11.01

Part 1 - OverviewGet the Most out of Development and Testing with the Maker of the Blue Box V11.01

Motivation - Functional Safety Standards ISO 26262 Automotive • Process driven software development (e.g., ISO/IEC 12207, ISO/IEC 15504  DO178B/C Avionics SPICE) IEC 62304 Medical • Model based software development andIEC 61508 testing explicitly mentioned EN 50128 Railway • Intensive testing highly recommended Process IEC 61511 Industry • Minimize the risk of systematic faults in the developed product due to IEC 60880 Nuclear Power malfunctions of the software tool EN 50271, EN Gas detection, (introduce or fail to detect errors) 50402 measurement • … … Get the Most out of Development and Testing with the Maker of the Blue Box V11.01

RealTime-UnitTest - What‘s in it for you?• Characteristics ― Object Code Level ― No test driver ― Integrated in the development environment• Requirements ― For Code Coverage analysis a micro controller with trace support is a must ― The target application should already run on the target ― Some stack space is needed ― Compilers should be EABI (Embedded Application Binary Interface) compatible* * specifies standard conventions for file formats, data types, register usage, stack frame organization, and function parameter passing Get the Most out of Development and Testing with the Maker of the Blue Box V11.01

RealTime-UnitTest - What will it do for you?• Benefits ― Quick turn-around times • No additional compile, link and download cycle while testing ― Flexibility in use • Combine trace, performance analysis, code coverage and I/O stimuli with test runs • Technology may be used for unit, integration and system test ― Test environment corresponds as closely as possible to the target environment Get the Most out of Development and Testing with the Maker of the Blue Box V11.01

How does it work?• Application is running on a target ― Initialization done ― Pieces of the application already implemented• A test case is setup and will be executed on the target ― Run until main{} ― Jump to the function under test, push test parameters on the stack ― Execute the funtion under test with parameters ― Compare return values with expected values and jump back to main{}• How to setup test cases and execute them on a target? ― RealTime-UnitTest Architecture Get the Most out of Development and Testing with the Maker of the Blue Box V11.01

Part 2 - RealTime – UnitTest Architecture Get the Most out of Development and Testing with the Maker of the Blue Box V11.01

Remote Control and Test API Architecture Get the Most out of Development and Testing with the Maker of the Blue Box V11.01

Remote Control and Test API Architecture isystem.connect for Python wrapper isystem.connect for Python wrapper isystem.connect for Python wrapper isystem.connect test isystem.connect debug isystem.connect iSYSTEM IDE & Debugger winIDEA Get the Most out of Development and Testing with the Maker of the Blue Box V11.01

C/C++ API isystem.connect iSYSTEM IDE & Debugger winIDEAGet the Most out of Development and Testing with the Maker of the Blue Box V11.01

C/C++ APIVerwendung isystem.connect debug debug.download(); debug.setBP("myFunction"); debug.run(); debug.waitUntilStopped(0, 200); debug.deleteBP("myFunction");Use of isystem.connect m_pIConnectDebug.RunControl(IConnectDebug::rDownload, 0, 0); m_pIConnectDebug.SetBreakpoint(IConnectDebug::bSymbol | IConnectDebug::bSet, 0, 0, "myFunction", 0); m_pIConnectDebug.RunControl(IConnectDebug::rRun, 0, 0); IConnectDebug::SStatus status; do { m_pIConnectDebug.GetStatus(IConnectDebug::tCPU, &status); Sleep(200); } while (status.m_byStatus != IConnectDebug::stStopped); iSYSTEM IDE & Debugger winIDEAm_pIConnectDebug.SetBreakpoint(IConnectDebug::bSymbol | IConnectDebug::bClear, 0, 0, "myFunction", 0); Get the Most out of Development and Testing with the Maker of the Blue Box V11.01

C/C++ API• Write your own C/C++ application and remote control winIDEA core functionality• Use already built abstraction layers for ― National Instruments LabVIEW (VI library), The Mathworks Matlab (JAVA) ― Microsoft Automation (e.g., to read/write data from/in Excel, use C#, Jscript, ASP, VBScript, ...) ― ...• Write your own abstraction layer• To simplify the use of this API and to implement RealTime-UnitTest we added another abstraction layer ― isystem.connect debug ― Script language wrappers Get the Most out of Development and Testing with the Maker of the Blue Box V11.01

Debug Abstraction and Script Language Layer isystem.connect for Python wrapper isystem.connect for Python wrapper isystem.connect for Python wrapper isystem.connect debug isystem.connect iSYSTEM IDE & Debugger winIDEA Get the Most out of Development and Testing with the Maker of the Blue Box V11.01

Debug Abstraction and Script Language LayerUse of isystem.connect debug debug.download(); debug.setBP("myFunction"); debug.run(); debug.waitUntilStopped(0, 200); debug.deleteBP("myFunction");Use of isystem.connect m_pIConnectDebug.RunControl(IConnectDebug::rDownload, 0, 0); m_pIConnectDebug.SetBreakpoint(IConnectDebug::bSymbol | IConnectDebug::bSet, 0, 0, "myFunction", 0); m_pIConnectDebug.RunControl(IConnectDebug::rRun, 0, 0); IConnectDebug::SStatus status; do { m_pIConnectDebug.GetStatus(IConnectDebug::tCPU, &status); Sleep(200); } while (status.m_byStatus != IConnectDebug::stStopped);Debugger winIDEA iSYSTEM IDE &m_pIConnectDebug.SetBreakpoint(IConnectDebug::bSymbol | IConnectDebug::bClear, 0, 0, "myFunction", 0); Get the Most out of Development and Testing with the Maker of the Blue Box V11.01

Read variable iCounter on the target and print its value until it reaches value of 100000.import isystem.connect # Set global variable isDebugTest to 1 and run the targetfrom isystem import icutils dbg.modify(IConnectDebug.fMonitor, isDebugTest, 1) dbg.run()from isystem.connect import IConnectDebug py_iCounter = 0# First we obtain controller objects # Read value of iCounter until condition is meticFactory = icutils.getFactory() while py_iCounter < 100000:dbg = icFactory.getDebugFacade() evalResult = dbg.evaluate(IConnectDebug.fRealTime, iCounter) py_iCounter = evalResult.getInt()ide = icFactory.getIdeCtrl() print py_iCounter time.sleep(1)import time dbg.stop()# Initialize print "Target stopped, iCounter reached the limit:", py_iCounterdbg.download() raw_input(Press ENTER to exit)dbg.deleteAll()dbg.setBP(main)dbg.run()dbg.waitUntilStopped() Python Example Get the Most out of Development and Testing with the Maker of the Blue Box V11.01

Conclusion• Generic API enables you to setup an automation scenario for ― Development ― Testing• Effort doing it in C/C++ is a bit higher than using abstraction layers as well as script languages• It still has not much to do with functional testing approach Get the Most out of Development and Testing with the Maker of the Blue Box V11.01

Test APIGet the Most out of Development and Testing with the Maker of the Blue Box V11.01

Test API isystem.connect for Python wrapper isystem.connect for Python wrapper isystem.connect for Python wrapper isystem.connect test isystem.connect debug isystem.connect iSYSTEM IDE & Debugger winIDEAGet the Most out of Development and Testing with the Maker of the Blue Box V11.01

Test APIUse of isystem.connect testUse of isystem.connect debug debug.download();Use of isystem.connect m_pIConnectDebug.RunControl(IConnectDebug::rDownload, 0, 0); Get the Most out of Development and Testing with the Maker of the Blue Box V11.01

Test API + Test Specification Language +Execution Environment == RealTime-UnitTesting Get the Most out of Development and Testing with the Maker of the Blue Box V11.01

Test Specification LanguageYAML 1.1 – yaml.org # Simple test of function: int funcTestInt2(int a, int b, int c)• Similar to XML func: [funcTestInt2, [-30000000, 2, 1], retVal]• Data structures like in Perl, Python, C expect:• Human readable - retVal == -29999996• de.wikipedia.org/wiki/YAML• isystem.com/downloads/isystem.connect/api/yaml_spec.html Use of isystem.connect test testCase.itestCpp("func: [funcTestGlobals]n" Use of isystem.connect debug debug.download(); Use of isystem.connect m_pIConnectDebug.RunControl(IConnectDebug::rDownload, 0, 0); Get the Most out of Development and Testing with the Maker of the Blue Box V11.01

import isystem.connect as ic # Simple test of function: int funcTestInt2(int a, int b, int c)import isystem.itest as it testCase.itest( """def verifyResult(results): func: [funcTestInt2, [-30000000, 2, 1], retVal] for result in results: expect: if result.isError(): - retVal == -29999996 out = ic.CStringStream() """, emitter = ic.EmitterFactory.createYamlEmitter(out) None, True) emitter.startStream() emitter.startDocument(False) print funcTestInt2 test result:, result.serializeErrorsOnly(emitter, None) verifyResult(testCase.getTestResults()) emitter.endDocument(False) emitter.endStream() # Test with coverage print out.getString() testCase.itest( else: """ print "OK" Python Example func: [funcTestGlobals] coverage:cmgr = ic.ConnectionMgr() runMode: startcmgr.connectMRU() document: itestCoverage.ccv openMode: wdbg = ic.CDebugFacade(cmgr) isSaveAfterTest: yes exportFormat: XML2# initialize the target - this may differ in your environment exportFile: coverageTest.xmldbg.download() statistics:dbg.runUntilFunction("main") - func: funcTestGlobalsdbg.waitUntilStopped() """, None);testCase = it.PTestCase(cmgr) print funcTestGlobals test result:, verifyResult(testCase.getTestResults()) Get the Most out of Development and Testing with the Maker of the Blue Box V11.01

Remote Control and Test API Architecture isystem.connect for Python wrapper isystem.connect for Python wrapper isystem.connect for Python wrapper iSYSTEM Test GUI testIDEA isystem.connect test isystem.connect debug isystem.connect iSYSTEM IDE & Debugger winIDEA Get the Most out of Development and Testing with the Maker of the Blue Box V11.01

testIDEA GUIGet the Most out of Development and Testing with the Maker of the Blue Box V11.01

env: Saved Test Specification initBeforeRun: false downloadOnInit: true from within testIDEA resetOnInit: false runOnInit: true stopFunction: main deleteBreakpointsOnInit: false tests:reportConfig: - id: StubTest reportContents: full desc: Stub Test outFormat: xml init: fileName: D:isystem.testIDEATestResultsPPC554 iCounter: 0ITMPC554 GCC int RAM Test Report.xml func: [Address_DifferentFunctionParameters] xsltFull: <built-in> itestResult.blue.xslt stubs: isIncludeTestSpec: true - func: useCustomTime: true - Func2 testInfo: - retV tester: Werner assign: date: 09.11.2010 retV: 2 time: 11:33:29 expect: software: SW - iCounter == 1 hardware: HW - id: GlobalVariableTest description: Desc desc: Global Variable Test testSpecificationFile: D:isystem.testIDEAMy YAML init:FilesPPC5554 ITMPC5554 GCC iCounter: 0 Int RAM.iyaml func: [Address_DifferentFunctionParameters] wiWorkspacePath: expect:D:winIDEA2011ExamplesOnChipPowerPCMPC55xxITM - iCounter == 1PC5554GCCIntRAM Get the Most out of Development and Testing with the Maker of the Blue Box V11.01

Get the Most out of Development and Testing with the Maker of the Blue Box V11.01

Part 3 - SummaryGet the Most out of Development and Testing with the Maker of the Blue Box V11.01

testIDEA Standard• Integrated in winIDEA• Free of charge• Online help and samples• Descriptive tooltips• TestID, tags, description• Local test variables• Stubs• Derived/sub test case creation• Debugging of test cases Get the Most out of Development and Testing with the Maker of the Blue Box V11.01

testIDEA Professional• testIDEA standard features• A license fee applies• Python script generation by mouse click• Trace, Profiler, Code Coverage support• I/O module support• Script execution before and after a test run• Excel/CSV import and export• Test filter based on tags• Test specification can be saved and loaded into/from C/C++ source code• Dry run – precalculation of expected return values by test execution• Automatic wrong parameter detection• Tag for specifiying the sequence of functions called during test• Breakpoints inside functions under test• Real-time stubs (stubs executed in real-time)• Report Generation in XML and HTML• Stylesheet for HTML Report Generation• Support/update and upgrade service Get the Most out of Development and Testing with the Maker of the Blue Box V11.01

Part 4 – Use CasesGet the Most out of Development and Testing with the Maker of the Blue Box V11.01

eMOTE *E*mbedded *Mo*del-based *Te*sting• Funded by BMWi Germany• Project partners: FZI Karlsruhe, sepp.med GmbH• Project Number: KF2076903SS9 Get the Most out of Development and Testing with the Maker of the Blue Box V11.01

Regression Test Tool SuiteGet the Most out of Development and Testing with the Maker of the Blue Box V11.01

Tool Pre-Qualification Scenario Safety Manual Test Specification, Test Cases, + Consulting/Integration Services Test Reports Reference-TargetTest GUI isystem.connect for Python wrapper isystem.connect test testIDEA isystem.connect debug isystem.connect iSYSTEM Development Environment & Debugger winIDEAReference Application Reference Target Get the Most out of Development and Testing with the Maker of the Blue Box V11.01

testIDEA v11.04

by iSYSTEM AG on Oct 21, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

Statistics

testIDEA v11.04 — Presentation Transcript