Vector cast Testing

1. Logo of your
organization 1
A scalable approach for Test Automation in Vector CAST/Manage with
the Jenkins tool to achieve Requirement Based Testing.
Manoj kumar Pant -SeniorEngineer
Avneet Mishra- Senior Engineer
Mando Softtech India

2. 2
Content
• Introduction
• Automotive Software Development and ISO – 26262
• What is VectorCAST/Manage ?
• What is Jenkins?
• VectorCAST/Manage integration with Jenkins provides a test automation platform
through continuous Integration testing .
• Benefits of continuous Integration testing .
• Features provided by VectorCAST/Manage with Jenkins
• Types of Requirement based testing performed by VectorCAST/Manage
• Design methods of for Deriving test cases in VectorCAST/Manage for Software
Unit/integration Testing.
• Coverage analysis through VectorCAST/Manage

3. Introduction
• This paper explain requirement based testing approach for
ensuring the intended Automotive Software performing its
functionality with a high level of confidence by satisfying the
ISO 26262 functional safety standard to meeting the Aspice
process assessment objectives by using VectorCAST/Manage
and Jenkins tool together.
• The objective of this document is to standardize the test case
design and test procedure development by using the
VectorCAST/Manage and Jenkins framework for the Unit level
testing, Integration Testing and Hardware Software
Integration Testing (Known as System Testing) as per ISO
26262.
3

4. Vector CAST/Manage integration with Jenkins tool to achieve Requirement Based
Testing
4

5. 5
Automotive Software Development and ISO – 26262:Automotive industries follow the V -
model for software development life cycle to meeting the objectives of ISO 26262 for
software safety standard compliance and quality as shown.
V-model of software development life cycle with reference to ISO 26262

6. What is VectorCAST/Manage ?
• VectorCAST/Manage is a tool for automating the process associated with different
levels of testing such as Unit level, Integration level and System level as shown. It is
used for code-coverage analysis by identifying which areas of an application have
been exercised by test executions.
6
Vector CAST/Manage tool to achieve Requirement Based testing

7. What is Jenkins ?
Jenkins is the leading open-source continuous integration server. It uses
the command to compile that stimulates a complete simulation
environment for an automotive system testing. Jenkins enables
continuous testing each time a source code change is made. it provides a
distributed testing infrastructure which allows a list of ‘nodes’ .nodes
indicates the types of job that can be run and reports on job status when
complete.
7

8. VectorCAST/Manage integration with Jenkins
provides a test automation platform through
continuous Integration testing :
• Continuous integration (CI) testing ensure software to be developed rapidly,
reliably, and repeatedly with minimal human intervention.
• Most automotive Industry (OEM and supplier) focusing on Continuous Integration
(CI) process to reduce build times.
• VectorCAST/Manage Integration with Jenkins provides a platform for continuous
Integration testing by Creating Single jobs, Multiple Jobs and Diagnostic Job.
• A Continuous Integration workflow approach provides incremental and parallel
source code compilation resulting in significant reduced application build times.
• The Continuous Integration allow compile the changed files only, and those
compilation jobs are spread over multiple cores on a single server, or multiple
physical servers.
• Continuous Testing applies these same techniques for software testing, thus
reducing test cycle times by selecting the sub-set of tests that are affected by a set
of code changes, and thus running those tests in parallel.
8

9. Benefits of Continuous Integration testing:
VectorCAST/Manage with Jenkins provides the Continuous Testing (CI) to identify a
defect (majorly at integration level) and provide instant feedback so that it can be
detected as soon as possible. Using automated tests, locating and fixing of bugs
becomes easy because any bug in the process is quickly identified by relating it to
the commit. This increases the frequency of software releases, time to market and
ensures more reliable applications.
.
9
VectorCAST/Manage automation increase the speed of testing by reducing time factor

10. 10
VectorCAST/Manage work together with Jenkins to reduce the time factor:
Let’s assume that each test suite contains 10,000 individual test cases, each of which takes 5
minutes to run. if we perform testing without Jenkins, the testing would be done in 50,000
minutes or 5 weeks.
By configuring multiple “nodes” to execute tests in Jenkins, we can divided the testing work
across many machines, and shorten the total time to test. Taken to an extreme, we could have
10,000 Jenkins Slave virtual machines running, and our testing would be completed in 5
minutes.
This approach Speeds overall project testing time and reduces late-in-the-project side effects.
VectorCAST/Manage with Jenkins reduces testing time with a huge margin

11. .
11
VectorCAST/Manage with Jenkins provides the
following features:
• Ability to organize thousands of test cases into logical groups that conform
to the structure of your application.
• Ability to run the same test against multiple configurations of the
application.
• Ability to determine the pass/fail status of each test case.
• Ability to determine code coverage per function Allows tests to be easily
run on every source change.
• Ability to define a list of “nodes” which are physical or virtual machines.
• Ability to e-mail job status once jobs are complete.
• Perfect for large projects with many users and a lot of tests.
• Overnight or complete application test execution can be reduced from days
to hours.
• Impact of Change analysis (Regression testing) can be performed on the
master project, greatly reducing the time it takes to identify regression errors.

12. Types of Requirement based testing performed by
VectorCAST/Manage :
VectorCAST/Manage during Hardware/software integration testing: To verify correct
operation of the software in the target computer environment.
VectorCAST/Manage during Software integration testing: To verify the
interrelationships between software requirements and components and to verify
the implementation of the software requirements and software components
within the software architecture.
VectorCAST/Manage during Low-level testing (Unit Testing): To verify the
implementation of software low-level requirements.
12
SOFTWARE TESTING ACTIVITIES

13. VectorCAST/Manage during Software Unit
/integration Testing:
Requirements-Based Test: Requirement-based test is a method where a
software/System requirement, such as functionality is verified. The
generation of requirement-based tests is greatly facilitated within
VectorCAST/Manage through the use of a graphical user interface (GUI)
where the input and expected values of parameters, return values, global
variables and even stubs (a piece of code that replaces dependencies) can
be specified.
Interface Test: This can be done by using boundary values of the interface,
illegal values and median values. The main objective is to ensure that the
interface is robust so as to prevent errors from occurring.
13

14. 14
Fault injection Test: : ISO26262 uses fault injection tests to increase the test
coverage of safety requirements as shown :

15. • Resource Usage Test: Resource usage testing can be automated by using the
VectorCAST/Manage tool.
• Back-to-Back Test between Model and Code: VectorCAST/Manage provides
integrations with design and modelling tools causing automatic code generation,
which can be input directly into VectorCAST for testing.
Design Methods for Deriving test cases in VectorCAST/Manage for Software
Unit/integration Testing:
15
ASIL
A B C D
1a Analysis of requirements ++ ++ ++ ++
1b Generation and analysis of equivalence classes + ++ ++ ++
1c Analysis of boundary values + ++ ++ ++
1d Error guessing + + + +

16. 16
• Analysis of requirements:Requirement analysis is done from Requirement
specification document such as Software Requirement Specification (SRS) and
Software Design document (SDD).
• Test case sample for requirement based testing:

17. 17
Requirement based testing can be designed in
VectorCAST/Manage by following methods:
Normal Range Test case design: To verify the normal behaviour of intended software
test case should be designed:
(1)within the data range
(2)Using valid equivalence class and boundary values
Robustness Test case design: To verify the abnormal behaviour of intended software
test case should be designed:
(1)Outside the data range.
(2) Using Invalid equivalence class.
Example : If the Range of parameter is 1 to 200.
Normal range test case should be developed from 1 to 200.
Abnormal range test case should be developed from <1 and >200.

18. Below are the following Requirement based testing techniques performed through
VectorCAST/Manage:
1) Equivalence classes
2) Boundary Value Analysis
Generation and analysis of equivalence classes in VectorCAST/Manage: In this method,
the data is divided into set of valid and invalid class in vector cast GUI.
Example : If the Range of parameter is 1 to 200.
18
Equivalence Class Partitioning in VectorCAST/Manage tool

19. Analysis of boundary values in VectorCAST/Manage: Boundary value analysis’ testing
technique is used to identify errors at boundaries rather than finding those exist in
centre of input domain.
Example : If the Range of parameter is 1 to 200.
19
Boundary Value Analysis in VectorCAST/Manage

20. Error Guessing Test
Error guessing is a testing technique that
makes use of a tester's skill, intuition and
experience in testing similar applications to
identify defects that may not be easy to
capture by more formal techniques. It is
usually done after most of the formal
techniques are completed.
20

21. Coverage analysis through VectorCAST/Manage
• The ISO 26262 divides ASIL levels as follows:
• ASIL A
• ASIL B
• ASIL C
• ASIL D
• Functional safety standard of ISO 26262, Part 6, Section 9.4.5, Table-12 which is
shown in figure 14,
• ASIL levels are used as per the criticality of the software. The ASIL A level only
covers statements, ASIL B/C level covers statement and branch and ASIL level D
covers statements, branch and MCDC (Modified Condition Decision control).
21

22. 22
Method ASIL Level
A B C D
Statement Coverage ++ ++ + +
Branch Coverage + ++ ++ ++
MC/DC Coverage + + + ++
Automotive Safety Integrity Levels

23. Coverage analysis through VectorCAST/Manage
23

24. Coverage trend through VectorCAST/Manage
24

25. References & Appendix
https://www.vectorcast.com
https://en.wikipedia.org/wiki/ISO_26262
https://en.wikipedia.org/wiki/Jenkins_ (software)

26. 26
Thank You!!!
Logo of your
organization