Embedded control software is growing exponentially in mechanical systems, which forces test methods to evolve even faster. This presentation was part of the Rational Quality Manager enlightenment series describing how National Instruments and IBM provide end-to-end traceability and test component reuse for superior system quality and validation by enabling consistent testing, results analysis, and traceability throughout the development process.

1. Embedded Software Validation Best
Practices with National Instruments
Kyle Perkuhn
Platform Marketing Engineer: Real-Time Test
National Instruments
kyle.perkuhn@ni.com
Paul Urban
Business Development Manager
IBM
purban@us.ibm.com
Scott Fairbrother
Senior Software Engineer
ISV Technical Enablement
Rational Ready for Rational Software
scottf@us.ibm.com

2. 10-20 defects produced per 1,000 lines of code*
6.5 Mil
Commercial
Aircraft
10 Mil
Luxury
Automobile
100k
Smart Washing
Machine
1k – 2k 65k - 130k 100k – 200kDefects
Lines of Code
* The Economists.May 16th, 2010. Tech.View:
Cars and software bugs. www.economist.com
Embedded Software Quality Challenge

3. 10-20 defects produced per 1,000 lines of code*
6.5 Mil
Commercial
Aircraft
10 Mil
Luxury
Automobile
100k
Smart Washing
Machine
1k – 2k 65k - 130k 100k – 200kDefects
Lines of Code
* The Economists.May 16th, 2010. Tech.View:
Cars and software bugs. www.economist.com
Embedded Software Quality Challenge
How can you ensure that a possible
malfunction will not harm anyone?
A “safety” approach is needed

4. Industry safety standards examples
Avionics/aerospace
DO-178B/C / ED-12B (RTCA/EUROCAE)
DO-254
Medical
FDA 510(k), IEC 62304, IEC 60601
Automotive
ISO 26262
Functional safety in process industry
IEC 61508
Nuclear power plants
IEC 60880
Railway Systems
EN50128 and EN50129
4

5. 5
Standards often initially increase project costs
Example: DO-178B
Typical
DO-178B
Project
Successful
DO-178B
Project
Technical
Project without
DO-178B
• Added 60% -
100% Cost
• Added 25% - 40%
Cost for Initial
Development
• Solid processes
• Experienced
Team
+25-
40%
+60 –
100%
Common issues
Inadequate formal plans or not following
them
Inadequate level of detail and process for
Requirements
Inadequate or non-automated Requirements
Mgmt and Traceability Mgmt
Lack of automated testing
Improper Tool Qualification (too much or too
little)
Weak process and checklist management

6. 6
IBM helps you turn product development into a
competitive advantage through our Rational Systems
Engineering approach
• Improve systems
engineering to tackle
growing product
complexity
• Improve software
development to deliver
innovation faster
• With an open, integrated
systems approach that
enables access to all
engineering information
across the engineering
disciplines.

7. 7
ECU Design/Dev
via AUTOSAR
Functional Safety
for Automotive
via ISO 26262
Functional Safety
for Aerospace
via DO-178B
Functional Safety
for Medical Devices
via IEC 62304 (future)
Defense
Architectures
via DoDAF
F u n c t i o n a l S a f e t y
Automotive
HW-SW Co-Design
Aerospace Electronics
Focused on Industry safety standards

8. IBM Automotive Solution for ISO 26262
Content
DashboardsPractice library
Auto generation of
practice work items
Starting templates
Tool mentors

9. Introduction to National
Instruments

10. Diversity of Applications
No Industry >15% of Revenue in 2012
SemiconductorsEnergy
Big Physics
Life Sciences
Consumer Electronics
Mobile Devices
AutomotiveAdvanced Research
Defense/Aerospace
Academic

11. NI CompactRIOPXI PC NI CompactDAQ
Graphical
Programming
Analysis and
Reporting
Test Automation
Real-Time Testing
Enterprise
Connectivity
Tools for Next Generation Development
Modeling
Multi-Chassis Systems

12. NI TestStand—Test Management Software
• Graphical sequence editor
environment
• Automate tests written in any
language
• Multithreaded sequence execution
• ASCII, HTML/Web, XML, and ATML
report generation
• Access, Oracle, SQL Server database
connectivity

13. Connecting Design and Test

14. © 2013 IBM Corporation
Accelerating Product and Service Innovation
4/9/2014
Execution Records, Defect
Management, Test Plan
Documentation
Test Plans
Test Cases
Test Schedules
Execution Records
Requirement Links
Quality Dashboards
IBM Rational Quality
Manager
Quality Engineer
Plan tests and link to
requirements
Schedule tests
Submit defect
Execute tests
Implement
test, analysis, and
reporting
Generate results
Test Engineer
Manually Manage Results

15. © 2013 IBM Corporation
Accelerating Product and Service Innovation
4/9/2014
National Instruments Test Components
(.seq, .vi, .nivs, …)
Requirements Documents
(DOORS, .docx, .xlsx, …)
Execution Records, Defect
Management, Test Plan
Documentation
Test Plans
Test Cases
Test Schedules
Execution Records
Requirement Links
Quality Dashboards
IBM Rational Quality
Manager
Test Results
(.html, .pdf, .tdms, .jpg, …)
Sequences
Parameter Files
Code Modules
Execution Records
Requirement Links
Quality Dashboards
NI TestStand

16. © 2013 IBM Corporation
Accelerating Product and Service Innovation
4/9/2014
National Instruments Test Components
(.seq, .vi, .nivs, …)
Requirements Documents
(DOORS, .docx, .xlsx, …)
Execution Records, Defect
Management, Test Plan
Documentation
Test Plans
Test Cases
Test Schedules
Execution Records
Requirement Links
Quality Dashboards
IBM Rational Quality
Manager
Test Results
(.html, .pdf, .tdms, .jpg, …)
Sequences
Parameter Files
Code Modules
Execution Records
Requirement Links
Quality Dashboards
NI TestStand
National Instruments
Test Integration
Adapter
for
IBM Rational
Quality Manager

17. © 2013 IBM Corporation
Accelerating Product and Service Innovation
4/9/2014
Next Generation Development Process
Systems Engineer Development Quality Engineer
DOORS Rhapsody
Team
Concert
Quality
Manager
Project Manager
Modify
requirements
and evaluate
impact
System Design
& Development
Submit defectAssess progress
Create change
request
Derive system
requirements
Plan tests and
link to
requirements
Test Engineer
Schedule tests Execute tests
Implement
test, analysis,
and reporting
Generate
results
NI LabVIEW | VeriStandNI TestStand
NI
DIAdem

18. Test Components
Models Analysis Reports
User Interfaces Stimulus Requirements

19. Consistency and Efficiency throughout the
Process
Data
Logging
Stimulus
Profile
Requirements
Physical
Testing
Design
Deployment
Prototyping
HIL
Validation

20. Increasing Development Quality and Efficiency
with Test Component Re-use
Re-use of test components reduces cost and risk by enabling correlation of test
procedures and results across all development phases
Continuity of test components reduces test application development and
maintenance costs
Data
Logging
Stimulus
Profile
Requirements
Data
Logging
Stimulu
s Profile
Data
Logging
Stimulus
Profile
Data
Logging
Stimulus
Profile
Physical
Testing
Design
Deployment
Prototyping
HIL
Validation
Data
Logging
Stimulus
Profile

22. Test Asset Management and Automation
Test Plans
Test Cases
Test Schedules
Execution Records
Requirement Links
Quality Dashboards
IBM Rational Quality Manager
MIL Test
System #1
MIL Test
System #2
MIL Test
System #3
HIL Test
System #1
HIL Test
System #2
HIL Test
System #3
Integration
System #1
Integration
System #2
Integration
System #3
Test Cell
System #1
Test Cell
System #2
Test Cell
System #3
Field Test
System #1
Field Test
System #2
Field Test
System #3
Busy Available Offline

23. Test Plans
Test Cases
Test Schedules
Execution Records
Requirement Links
Quality Dashboards
IBM Rational Quality Manager
Defect Results and
Verification Items
Defect Resolution
Collaboration
Defect Management

24. Project
Dashboards
Test
Results
Test Plans
Test Cases
Test Schedules
Execution Records
Requirement Links
Quality Dashboards
IBM Rational Quality Manager
Project Documentation and Tracking

25. • Traceability all the way to test and back
• Collaboration between ALL teams, including test
• All test results available to all teams and linked
to test cases and requirements
• Quality and test considered from the outset, not
an afterthought, reducing cost and risk of
identifying and correcting defects
• Test component re-use throughout project
phases and between projects providing operational
efficiency and accuracy
IBM Rational and National Instruments are teaming
together to provide an end-to-end quality
management solution

26. Learn More:
Web pages:
IBM.com: IBM Rational and National Instruments
NI.com: Integrating Test Into Development Cycle
NI Test Integration Adapter Product page
References:
Solution Brief: Break down engineering silos to improve quality
Article: To Build a Smarter Product, Stop Separating Design & Test Ian Cannings, Danfoss
Power Electronics
Blog: Design News blog
Media:
Webinar: Integrating National Instruments testing with RQM (demo starts at min 24)
Webcast:: Embedded System Quality: Top Competitive Techniques Revealed
Video: Interview Innovate 2013- Chris Washington, National Instruments...
Video: Innovate 2013 Interview and Demo at NI Booth
Video: NI Week 2012 Keynote: Embedded Software Quality Management
Webcast: Taming the Complexities of Software Driven Innovation to Reduce Project Cost and
Risk
Podcast: Integrate testing into the development lifecycle

27. © Copyright IBM Corporation 2013. All rights reserved.
The information contained in these materials is provided for
informational purposes only, and is provided AS IS without
warranty of any kind, express or implied. IBM shall not be
responsible for any damages arising out of the use of, or
otherwise related to, these materials. Nothing contained in
these materials is intended to, nor shall have the effect of,
creating any warranties or representations from IBM or its
suppliers or licensors, or altering the terms and conditions of
the applicable license agreement governing the use of IBM
software. References in these materials to IBM products,
programs, or services do not imply that they will be available
in all countries in which IBM operates. Product release dates
and/or capabilities referenced in these materials may change
at any time at IBM’s sole discretion based on market
opportunities or other factors, and are not intended to be a
commitment to future product or feature availability in any
way. IBM, the IBM logo, Rational, and other IBM products
and services are trademarks of the International Business
Machines Corporation, in the United States, other countries or
both. Other company, product, or service names may be
trademarks or service marks of others.
www.ibm.com/software/rational

28. Backup

29. DOORS Template for ISO 26262
• Capture Severity, Probability and
Controllability attributes
• Automatically determines ASIL
• Working on matching these attributes to
attributes in Rhapsody
• Developing requirements module
structure to capture relationships
between
• Stakeholder (Item Definition) Requirements
• Functional Safety Requirements
• Technical Safety Requirements
• System Safety Requirements
• HW and SW safety requirements
• Automatic propagation through Safety
Requirement Hierarchy of ASIL
• Delivered as a DOORS project archive

30. 30
Rhapsody Kit for ISO 26262 and IEC 61508
• Overview Doc: describes the contents of the
Rhapsody kit
• Rhapsody Reference workflow : provides an
exemplary workflow for modelling, code
generation and verification in safety critical
• Rhapsody TestConductor Add On Workflow:
describes testing activities and objectives
• Rhapsody TestConductor Safety Manual:
provides additional information for using
TestConductor in safety related applications
• TÜV SÜD Certificate for Rhapsody
TestConductor Add On
• TÜV SÜD Report on Certificate for ISO 26262
and IEC 61508
• Rhapsody TestConductor Add On Validation
Suite: separately available test suite for
Rhapsody TestConductor to help in
qualification efforts
• Certification kits for the SXF (C++) and SMXF
(C) frameworks