Embedded software engineering has become a much bigger and more complex domain than we could have imagined. As devices are expected to communicate with other devices and embedded subsystems, a much larger surface area has emerged for defects that threaten the safety, security, and reliability of the software. For example, the connected car not only introduces software safety and security concerns within the car as a system, interactions with environmental components, such as communicating with 'smart traffic lights' and vehicle-to-vehicle communication, potentially expose additional risk. Additionally, as car makers develop and merge functionality into 'the autopilot' mode, driver-assist technologies have become safety-critical technologies. Embedded software organizations have always taken a 'shift-left' approach to software quality, rigorously applying defect prevention techniques early in the lifecycle. The demand for IoT requires a new testing paradigm that more closely resembles the challenges that Enterprise IT have faced for decades. As enterprise IT struggles to 'shift-left', embedded systems are struggling to 'shift-right' by testing more componentized and distributed architectures.

1. What Embedded Software
Engineering Can Learn from
Enterprise IT Testing
Techniques

2. Agenda
• Trends in embedded software engineering
• Why to learn from IT?
• Lessons from IT
• Summary
• Questions and answers

3. Industry Trends
• Increasing complexity of embedded systems
• Connectivity is a game changer
• Safety, security, quality
• New trends emerge in automotive, medical, …
• Automotive software market = $10.1B by 2020
• Software = 21% of vehicle value

4. Why is Enterprise IT like Embedded/IoT?
• Functionalities spanning
across many layers
• Open Architecture
• 24x7 Connectivity
• Constant Security threat

5. Why to learn from IT?
• Enterprise IT solved the problems
…
• Connected embedded systems
resemble architectures known in IT
• Complexity stemming from
interconnections shifts the gravity
in testing
• Embedded Left-focus strategy no
longer optimal

6. What is missing?
Embedded development
• Focus on defect prevention and
early detection
• Unit Testing, Software Coding
Guidelines
• Avoid testing fully assembled
integrated systems
• Avoid debugging of integrated
systems
Enterprise IT development
• Focus on rapid development
• Volume of code too large to widely
apply unit testing
• Automated component/API level
testing
• Testing and debugging integrated
solutions

7. Lessons from IT
• The right granularity of Unit/
Component/API tests
• Solidifying automation at the
message layer
• Gaining access to fresh, realistic,
and credible test data
• Accessing the distributed
environment in order to test
continuously

8. Strategies
1. Focus ROI of testing with code and change traceability throughout the
Pyramid
2. Automated system level tests validated integration with Automated API
Testing
3. Test anytime, anywhere with a Virtual Test Bed for entire system
4. Test for security vulnerabilities at every stage

9. Combine API testing with coverage
• Assess the quality of functionalities
deployed in multiple layers
• Automated API tests executed
against each layer
• Reasonable structure of
requirements
• Mapping tests to requirements
• Understanding tests results in
context of requirements

10. Combine API testing with coverage
• Combining code coverage with
structured tests
• Using code coverage to assess the
testing process
• Avoiding weak links in the
functionality
• Coverage tools supporting multiple
technologies
• Assess impact of change in the
code base

11. Aggregated test coverage

12. Granular individual test coverage

13. Traceability back to requirements

14. From Automated to Continuous
• Accelerating Agile requires Continuous Testing … remove the bottlenecks
• Automated = no human intervention when executing
• Continuous = no manual setup, no scheduling … test anytime
• Continuous Testing critical to DevOps workflow
• But unstable/unknown/unpredictable environment
• … need to isolate the complexity and instability of the test environment …
© 2017 Parasoft Corporation
Perfecting Software
14

15. Virtual Test Bed
• Service Virtualization
• Simulated dev / test environment
• Eliminate testing bottlenecks
• Test real-world behavior beyond mocks and
stubs
• Test anytime, anywhere
• Full control of the behavior; functional, data,
performance, network
• Locally on the desktop + shared server
environments
• Dynamically reconfigure via thin-client +
automation workflows
• Leverage Azure/AWS for on-demand scaling
15
© 2017 Parasoft Corporation
Perfecting Software

16. Test at scale with no hardware
• Performance problems found late in the cycle costly to fix
• Time between defect introduction and detection = too great
• Non-functional requirement … not the focus of a ‘sprint’
• Challenge
• Disconnect between functional and performance tests
• Infrastructure unavailable or not scalable
• Solution
• Reuse functional tests
• Service virtualize the ‘performance characteristics’
• Continuously validate the non-functional requirement

17. 1
5
10
20
50
150
Req
Design
Code
UAT
SIT
Ops
Shift left security testing
• Penetration Testing vs Security Policy
• Late cycle, disconnected from dev, costly, cannot test ’every combination’
• .. Security needs to be integrated into earlier stages of the SDLC
• Leverage functional tests to drive to security vulnerability testing
• Reduce the cost of late cycle detection
• Trace to the business cases that are impacted
• Trace to the underlying code and identify the root causes
• Move from ‘detection’ to ‘prevention’ = policy
© 2017 Parasoft Corporation
Perfecting Software
17

18. Summary
• IT industry well proven testing techniques, such as:
• Automated API Testing – stable, maintainable
• Service Virtualization – virtual test bed
• Automated Security Testing – ‘detection’ drives ‘prevention’
• Test, Requirement and Code Traceability – know what gets impacted by change
• Apply the same strategies to tame the complexity explosion of embedded
systems

19. Thank you
Q&A
Parasoft - Hall 4 stand 4/416