2019 ibm io t exchange - practical use case of AI in engineeringv2

IoT PlatformTRIRIGA EngineeringMaximo

Watson IoT: Practical Use Case of AI in Engineering
—
Galina Mishiev
World Wide AI Sales Leader
M. Kevin McHugh
Senior Managing Consultant
Practice Lead – AI and Compliance

IBM Watson IoT / © 2019 IBM Corporation
The information we need to know is expanding exponentially
… exceeding our capacity to learn
… to close that gap, we apply AI
Rate
Time
Data increasing
Capacity and
ability to learn
and adapt
Knowledge and
augmentation
AI

IBM Watson IoT / © 2019 IBM CorporationIBM Watson IoT / © 2019 IBM Corporation
Power of AI in Engineering
What is your “ETA”?

AI and Engineering
Inject quality, automation, and experience
Electrical /
Electronics
Design
Mechanical
Design
Lean Software
Engineering
Requirements
Quality Analysis
Automatic Model
Generation from
Requirements
Traceability and
Impact Analysis
Test Case
Generation from
Requirements
IBM
Engineering
Lifecycle
Management
5

• Evaluate – drives quality
• the primary purpose of an Evaluative AI application is to assess the quality of an artifact
• Transform – promotes automation
• The primary purpose of a Transformative AI application is to create or update an artifact
based on existing information
• Assist – assists with experience
• The primary purpose of an Assistive AI application is to extend or expand the knowledge,
skills, experience of an engineer when actively working with artifacts
Types of AI Application

Types of AI Application
What is your “ETA”?

• Evaluate
• Requirements Quality Assistant Product Offering - GA
• Impact Analysis – Link Assessment Services Offering – seeking beta clients - $0 cost
• Transform
• Smart Test Case Generator Services Offering – GA
• Assist
• Smart Editors Service Offering – seeking beta clients
Watson IoT Offerings

IBM Requirements Quality Assistant

IBM Watson IoT / © 2019 IBM Corporation 10
Introducing Requirements Quality Assistant
New Watson capability embedded inside DNG …
• Removes risk and ambiguity in the requirements authoring
phase out-of-the-box by using AI (Watson Natural
Language Understanding)
• Pre-trained to detect key quality indicators designed to be
consistent with the INCOSE Guidelines for Writing Good
Requirements
• Authors receive coaching from Watson to improve the
quality of the requirement as it is being written
Enterprise benefits (400 engineers example)
• Reduce the cost of defects by 60% to save $3.9M
• Reduce cost of manual reviews by 25%
• Retain engineering expertise for junior engineers

IBM Requirements Quality Assistant
• Grades requirements against a criteria that was designed
to be consistent with the INCOSE Guidelines for Writing
Good Requirements
• Pre-trained to detect 10 quality issues
– Unclear actor or user
– Compound requirement
– Negative requirements
– Escape clause
– Missing units
– Missing tolerances
– Ambiguity
– Passive
– Incomplete requirements
– Unspecific quantities
• Add to the list of quality issues or do deeper training through a
3 week services engagement with IBM services team
11

IBM Impact Analysis

13
Impact Analysis
Quality of your trace matrix
• Should all the linked artifacts actually be linked?
• Are there missing links?
• Do you have duplicates Requirements?
• Do you have duplicate Test Cases?

Scores show the correspondence between Requirements text and Test Case text
14
These Test Cases are linked to Requirements… should they be?

Are you re-executing Test Cases that don’t need to
be re-executed?
Are you executing too many Test Cases?
Are you maintaining too many Test Cases?
Are there impacts beyond what your traceability
shows?
Analyze all the Requirements and all the Test Case
21 3

800 total assessments
Let’s look at the Scope of Analysis

283 have a non-zero correlation, out of which:

24 links > 70% correlation

24 proposed missing links!

16 are indeed valid missing links!
• better trace, better test, better product
Human Analysis of the 24 Proposed Links Indicates
… if you made a change to the product, you would have failed
to test HALF of the things you needed to test

6 indicate Test Cases should be combined
• reduce test artifact maintenance.
… you are running and maintaining 6 test case that you do
NOT need to spend time on

2 do not provide useful information to the engineers

24
Business Drivers:
Organizations need to improve quality, comply
with standards, and get to market sooner – all
at less cost! Improving the accuracy of trace
links between requirements and test cases can
help achieve these objectives.
Reduce costs. Time is money. Reduce the time
spent evaluating trace, maintaining duplicate
test artifacts, running duplicate tests.
Improve Time to Market. Realize income from
your development efforts more quickly.
Increase market share by offering features
earlier than your competitors.
IBM Impact Analysis – Zero cost Beta
Offering Benefits:
We provide a capability that helps you deliver
better products, more quickly and cost-effectively.
Identify probably missing and/or incorrect links
Improve test coverage correctness, find defects
earlier
Reduce maintenance costs and reduce excess test
execution time
Offering delivers:
Score matrix between linked Requirements and
Test Cases
Score matrix between unlinked Requirements and
Test Cases.

IBM Smart Test Case Generator

A Natural Language Processing solution
Reads Requirements from DNG (now renamed ERM)
Writes Test Case sections to RQM (now renamed ETM)
Precondition
Expected Results
Test Case Design

Lean method to automatically achieve high correspondence

There’s more!
Test Case text shows a poorly written requirement.
Let’s rewrite for Testability.
…and re-generate…

Better requirements…better Test Cases… Better Product
Original Improved

Impact
- Test Cases:
- 30-60 minutes of Test Engineer time / Test Case
- Test Scripts:
- 30-60 minutes of Test Engineer time / Test Script
- Test artifact reviews:
- 10 minutes / artifact / review board member
- Requirements review:
- 10 minutes / artifact / review board member
- Requirements are improved through the cycle with the tester. Therefore the reviews are quicker
- Aligns Test Case more closely to Requirements
- Fewer operational defects. Higher quality = lower costs
- Saves expensive rework due to ambiguously written
requirements
- This improves the requirements themselves. Therefore the Architecture, Design, and Implementation are
more true. Therefore the ‘as built’ product meets requirements and expectations more closely
- Improved ASPICE / ISO 26262 ratings

32
Business Drivers
Discover more defects, reduce costly
rework.
Better Test Cases means better product.
Reduce costs. Time is money. Reduce the
time spent manually writing Test Cases.
Improve Time to Market. Realize income from
your development efforts more quickly.
Increase market share by offering features
earlier than your competitors.
Smart Test Case Generator for IBM Engineering - Quick Start
Value Proposition
Offering Benefits
Improve compliance score by raising
correspondence between Requirements and
Test Case content
Better requirements…Better Test
Cases…Better Product.
Lean your test authoring process.

IBM Smart Editors

The Smart Editor Highlights
Find the system component relevant
See the requirement completed with the AI assist
ensuring consistency
• Terminology becomes
consistent across all artifacts
• Reduces variations such as: "Adaptive Cruise
Control", "ACC", and "Adaptive CC”.
• Multiple variations of names for the same
entity, can lead readers (Testers, Developers)
to think they are different entities.
• New team members come up to
speed faster when an entity has
only one name.
• The Smart Editor also reduces
typos for complex names.

Smart Editor for ETM Test Cases
ETM Test Case
sections available
as well

Smart Editors increase alignment and consistency in terminology
between team members during product development. Leading to…
Higher productivity
Higher quality
New team members spin up faster
Smart Editor – Value Proposition

• Evaluate – drives quality
• Transform – promotes automation
• Assist – assists with experience
Summary: Types of AI Application
What is your
“ETA”?

Use case: Evolving Solution Architecture
RQA
DNG or DOORS
TC Text
(RQM)
Req Text
(DNG)
Test Case
Quality Assistant
Smart Test Case
Generator
Client Data
Evaluative
Requirements
Smart Editor
Test Case
Smart Editor
Impact Analysis
Transformative
Assistive

Where are you taking AI next?

Thank you
M. Kevin McHugh
Practice Lead for Watson IoT Lab Services AI and Compliance
—
mchughm@us.ibm.com
+1-719-331-0886
ibm.com
Twitter: mkmchu
LinkedIn: www.linkedin.com/in/m-kevin-mchugh-65048315
Blog: https://mkevinmchugh.wordpress.com/

2019 ibm io t exchange - practical use case of AI in engineeringv2

More Related Content

Similar to 2019 ibm io t exchange - practical use case of AI in engineeringv2

More from M Kevin McHugh

Recently uploaded

2019 ibm io t exchange - practical use case of AI in engineeringv2