ODD + Project Control 0.9

Obstacle Driven Development +
Project Control 0.9

Testing First?
08/11/2016 ©odd.enterprises 2
Answering a
question before
its asked is like
creating a
solution before
a test.

ODD Motivation
Life will test every
aspect of a products
development stages,
levels and elements.
So,
• Why do engineering
differently?

Background
Ideas of Obstacle Driven Development (ODD) are based on
numerous development processes including:
• Control Theory
• Scientific Method
• ISO V-model
• Test Driven Development
• ISO specifications
• Requirements analysis spiral
• Agile principles
• SOLID principles

Control Theory
Control theory applies
feedback and
feedforwards control.
• Error between
reference and plant
output compared
• Extensive
engineering,
mathematic and
scientific uses

Project Control
Control theory applied to
creating a solution gives a
model for control of
projects.
• Negative feedback from
testing solution
• Feedforward control is
creating tests
• Feedback control is
solving tests

Traditional Engineering without Tests
Traditional engineering
relies heavily on
assumptions of
engineers.
• Good engineers
make good
assumptions
• Good and/or bad
engineers make bad
assumptions

Traditional Engineering + Testing
Testing allows us to
identify and correct
errors.
• Creating tests after a
solution means we
design tests according
to a solution
• More tests mean we
can identify more errors

Assumptions and Debugging 1
If we don’t have efficient
testing then debugging is
inefficient.
• Assumptions can
easily lead to bugs
• Without thorough
testing we can’t
effectively find bugs
• Automated testing
improves efficiency

Assumptions and Debugging 2
Without thorough
testing we may have
solutions full of bugs.
• May create
exponentially more
bugs
• Bugs exponentially
more difficult and
expensive to fix
• What is “Works as
Expected”?

ODD Testing
Creating tests first
allows design of
solution with closed
loop control.
• Feedforwards testing
allows creation of a
negative feedback
loop
• Feedback used to
determine project
progress and control
development

ODD Proof 1
𝑂 𝑑 , 𝐸 𝑑 𝑎𝑛𝑑 𝑌 𝑑
are Obstacle (input),
Error and Output.
• 𝑑 is 2*n bits for
status of tests and
solutions
• Use bitwise
addition and
multiplication

ODD Proof 2
𝑂 𝑑 , 𝐸 𝑑 𝑎𝑛𝑑 𝑌 𝑑
and outputs are
replaced by 2 bits.
• 11 is reference for
test and solution
to be created
• First loop round
gives 𝐸 𝑑 =11
and test created

ODD Proof 3
Complete one loop
to create a test
before solution.
• 𝐸 𝑑 = 01 as test
has been created
without solution
• We develop a
solution for the
test

ODD Proof 4
Complete two loops
to create a test and
solution.
• 𝐸 𝑑 = 00 as test
has been created
with solution
• Continue until
𝑛 obstacles are
tested and solved

Project Control
Control theory applied to
give feedforward and
feedback control.
• Feedforwards control is
creating a test
• Feedback control is
solving a test
• Multiple feedback
loops to Analysis

Obstacle Driven Development
Obstacle Driven
Development allows
testing of all stages.
• Fully testable stages,
abstraction levels and
elements
• Identify, correct and
prevent failure at
earliest opportunity

Why Use Obstacle Driven Development?
Traditional engineering tests solutions
once they are created.
• Solution tested against often
incorrect requirements
• Requirements fixed and not testable
• Errors detected very late
• Errors propagating through stages
cost 10x to fix per stage

Scientific Method
Scientific method says we must
create tests before solutions.
• Therefore creating solutions
before tests is unscientific
• Traditional engineering creates
solutions before tests, so is
unscientific
• Is a scientific method to
engineering required?

Scientific & Engineering Method
Comparing scientific and
engineering methods show
differences.
• Depends on experience
and talent of engineers
• No way of testing
requirements
• Customers discover their
requirements differ from
products

ODD Scientific Method
Scientific method is
created through
adapting model.
• More testing and
feedback than
traditional scientific
method
• Allows repetition to
refine theories
• Learning from failure

ODD Engineering Method
Engineering version of a
scientific model.
• Stages, checkpoints
and control of ODD
shown
• Verification provided
by creating tests
• Validation provided by
solving tests

ODD Scientific & Engineering Methods
Comparing methods we see structure
is identical and wording equivalent.
• Conclusion is the method is
scientific
• Each method repeatable to
continue refinement
• We can combine the methods for
R&D

Extreme Programming
Original inspiration for ODD came
from Extreme Programming (XP).
• XP is a software development
methodology
• Improves software quality and
responsiveness to requirements
• Unit testing of all code
• Avoids programming features
until they are needed

Extreme Engineering
From Extreme
Programming we built a
method for engineering.
• XP ideas applied to
stages of software,
hardware and
embedded
• XP adapted to
engineering gives
Extreme Engineering

Overkill? 1
Testing every stage, level
and element of a
development is difficult
and expensive.
• Overkill for simple
problems
• More efficient the
longer and more
complex development

Overkill? 2
There are 2 main uses
for the method.
• Ideal for complex
projects
• Ideal for long lasting
projects
• Best for complex and
long lasting projects

ODD Principles
Principles inspired by Agile
manifesto.
• Over substituted so both can
support development
• Stages adapted and models
created help facilitate aims
• Focus on linking obstacles to
solutions
• Processes and tools which
encourage individuals and
interactions
• Working software through
comprehensive documentation
• Contract negotiation through
customer collaboration
• Following a plan which responds
to change

Principle 1
Processes and tools which
encourage individuals and
interactions.
• Identification and solving of
problems encouraged
• Individuals empowered to use
full ability of process and tools
• Interactions simplified with tests
helping communication

Principle 2
Working software through
comprehensive documentation.
• Describing products and systems
helps create tests and solutions
• Working software designed
according to tests
• Comprehensive documentation
prevents basic errors

Principle 3
Contract negotiation through
customer collaboration.
• Specification helps meet
customer requirements
• Behaviours easy to describe and
understand
• Creating a specification is cheap,
testable and adaptable

Principle 4
Following a plan which responds to
change.
• Stages and elements subject to
change as required
• No stages frozen and adapted to
requirements
• Failing tests may result in
modifying previous stages

Test Driven Development 1
Test Driven Development is
software development where tests
are written before code.
• Creating tests first ensures code
tested and testable
• Unit tests ran through test suites
for automated testing
• Improved code through early
error detection
Write
Test
Write
Code
Refactor

Tests are written before code.
1. Test is written and ran to
observe a fail
2. Simplest code is written to pass
the test
3. Code is refactored according to
SOLID principles
4. Repeat until coding complete
Write
Test
Write
Code
Refactor

Development of ODD began with a
question when using TDD:
• Where do tests come from?
Write
Test
Write
Code
Refactor

Behaviour Driven Development
Described as “Test Driven
Development done right”.
• Behaviours implemented for
testing and design
• Additional stage of thinking
about tests
• Increased efficiency over TDD
Write
Test
Write
Code
Refactor
Think

ODD Behaviour Driven Development
Adapting sequence gives one similar
to traffic lights.
1. Red light for behaviour to be
coded
2. Amber light when tests are
created
3. Green light when code written
and tested
4. Blue light when tests are passed
5. Repeat until specification is coded

ODD Process
Process becomes generic to create
a new development model.
• Applications to hardware,
software and embedded
• Links obstacles with tests for
verification and validation
• Refactoring included in Validate
Solution

Obstacle Driven Development 1
Process adapted and repeated
between four stages in sequence.
• Analysis
• Specification
• Solution
• Production
Specification
Solution
Production
Analysis

Obstacle Driven Development 2
Verification and validation link
stages and provide feedback.
• Verification ensures a product is
built in the right way
• Validation ensures a product is
built right
• Creating and solving tests give
verification and validation

ODD Process

ODD for Embedded
designed for software, hardware
and embedded.
• Stages defined to help create
physical products
• Benefits of software, hardware
and embedded principles
• Suitable for safety critical
applications

V-model
V-model engineering for
requirements, designs, testing of
problem solutions.
• V-models give framework of
• Development for safety critical
design processes
• Described by international
standards

International Organisation for Standardisation
Standards and processes provided
for requirements, specifications,
guidelines or characteristics.
• Ensures products, services and
components are fit for purpose
• Sets standards required for state
of the art and markets
• ODD designed to be compatible
with international standards

ISO V-model
V-model to apply ISO standards for
software development.
• Model links various levels of
development
• Testing for levels from software
up to vehicle tests
• Testing for feedback and ensures
each level is verified

Test Driven Development V-model 1
V-models are combined with Test
Driven Development.
• Help create tests and integrate
systems
• System created according to
designs through tests
• Tests help integrate low level
code to high level

Test Driven Development V-model 2
We extend and invert V-models so
development stages are separated.
• Models consist of V-models with
TDD and unit testing
• Inverted V-models allow for
separation of stages
• Inverted V-models link with V-
models creating further models

Inverted V-models
V-models inverted to link stages
and help decomposition.
• V-models help integrate a
solution
• Inverted V-models to decompose
a creation
• Testing is adapted for
decomposition of elements

Extending a Specification 1
• Traditional Problem
and Solution Space
– Specification a small
interface
• Extended
Specification
– Specification a
separate stage
49©odd.enterprises08/11/2016

Extending a Specification 2
Extending Specifications for
creating new models combined
with TDD.
• V-model compatible with Safety
Integrity Levels
• TDD processes for V&V of
Specification
• Specification to create tests

ODD Verification and Validation 1
Provides full verification
and validation through
creating and solving tests.
• Verification (“is it built
right”) provided by
creating tests
• Validation (“built
right”) provided by
solving tests

ODD Verification and Validation 2
Verification and validation links
stages.
• Specification
– Verification and validation
• Solution
– Testing and design
• Production
– Quality assurance and
control
• Analysis
– Utilisation and elicitation

Feedforward Processes
Verification is a
feedfoward process
where tests are
created for next stage.
• Verification
• Testing
• Quality assurance
• Utilisation

Feedback Processes
Validation is a
feedback process
where tests are solved.
• Validation
• Design
• Quality control
• Elicitation

M-Model Comparison
Model links four separate stages of
problem domain with unit testing.
• Verification and validation
appropriate to each stage
• Extends traditional problem and
solution domain
• Unit testing processes links
stages

X-model Form
Alternative form is X-model and we
concurrently integrate stages.
• Operates same as M-model
except all stages are integrated
• Structure is 4 V-models around a
central axis
• For developments where time is
shorter

Abstraction Levels
Element describes stage and level
from material to product and
analysis to production.
A Product consists of:
• Systems; which consist of
• Subsystems; which consist of
• Components; which consist of
• Materials

Abstraction Levels
Abstraction levels allow divisions to
a stage to organise testing,
integration and decomposition.
• Abstraction levels from material
to product
• Extendable to company and
customer levels
• Integration and decomposition of
stages through SRP

System Divisions
We divide stages into abstraction
levels and system divisions.
• Linked to equivalent obstacle in
previous stage
• Linked to equivalent solution in
next stage
• Each single block is an element

Linking Tests 1
Tests link
behaviours with
solutions through
testing and design.
• Solutions designed
according to tests from
behaviours
• Each solution is a single
element of a product
• Unit testing is applied
• Test suite created and
ran when changes
occur

Linking Tests 2
Testing and design
of solutions from
behaviours of
specification.
• Solution implements
1 or more behaviours
• Tests suite ran for any
changes or additions
• Created as with Test
Driven Development

ODD Test Suites
Test suites maintain
solutions for
software and
identify errors.
• Test suites contain
individual and combined
unit tests
• Test suites are intended to
be implemented between
all stages
• TDD process applied
throughout development
to create ODD

Linking Solution to Production 1
Solution produced
with consistent
quality often for
large quantities.
• Solution assures and
controls quality of
related production
• Tests are created for
quality assurance
• Tests passed give
measure for quality
control

Linking Solution to Production 2
Elements of solution
create quality
assurance tests for
production.
• Basic assurance and
control from failure
rates
• Control ensures failure
rate is acceptable
• Assurance will
determine an
acceptable failure rate

Linking Production to Analysis 1
Linking Production to
Analysis ensures
product features
utilised and elicited.
• Features of product
utilised by customers
• Elicitation after
utilisation of feature to
find requirements
• Verifies current
solutions and identifies
new obstacles

Linking Production to Analysis 2
Production & Analysis
linked with features a
customer sees in a
product.
• New feature may
cover requirement
but create another
• Elements and level
investigated
independently

Requirements Analysis 1
Requirements analysis is an
essential stage where we determine
what a product should do.
• Requirements analysis is
performed in numerous ways:
– Spiral model
– Use case analysis
– Safety integrity Levels (SILs)
• Requirements analysis spiral is
adapted and integrated

Requirements Analysis 2

Requirements Analysis Adaptions 1
Spiral model is superimposed onto
an M-model and adapted.
• Agreed Behaviours substitutes
Agreed Requirements
• Quality Assurance equivalent to
Testing
• Negotiation similar to Verification
– Continues on next slide

Requirements Analysis Adaptions 2
Spiral model is superimposed
ontoan M-model and adapted.
• Verification substituted for
Negotiation
• Validation substitutes Evaluation
• Testing substitutes Quality
Assurance

ODD Analysis
Requirements analysis combines with
Specification in an inverted V-model.
• Tree diagram approach creates
situations from events
• Safety Integrity Levels measure and
process situations into hazards
• Checkpoints for Product,
Consolidated Requirements and
Documents

ODD Solution
Solution created from specification
through behaviour tests
• Red light for behaviour contained
in specification
• Amber light when test for
behaviour is created
• Green light when solution
designed to pass test

Integration of Solution
Solutions integrated from materials
into components, components into
subsystems etc.
• Solutions created with bottom-
up process
• Each level is integrated into a
higher level
• Higher level tests provide
integration testing

Solution Control
Using a Specification
we create closed loop
control to develop
solution.
• Creating tests from
Specification is
feedforwards
control
• Solving tests allows
a feedback loop
with control of
creating solution

Testing Spiral for Solution Stage
Solution stage tests Materials first
and integrates until Product.
• Continue in circle until all tests
passed for abstraction level
• Once tested we spiral outwards
to next abstraction level
• Once completed we have a fully
tested Solution to a Specification

ODD Production
Production created from solution
through quality assurance and
control.
• Red light for solution with no
quality assurance and control
• Amber light when test to assure
quality is created
• Green light when quality control
passed

Decomposition of Production
Production decomposed from high
level to assure efficiency and high
quality levels.
• Production created with a top-
down process
• Each level decomposed for
efficiency of lower levels
• Efficiency through testing of
production quality

Production Control
Using a Solution we
create closed loop
control to develop
Production.
• Creating tests
from Solution is
feedforwards
control
• Solving tests
allows a feedback
loop with control
of Production

Testing Spiral for Production Stage
Production stage tests Product first
and decomposes until Materials.
• Continue in circle until all tests
are passed for abstraction level
• Effective distribution and control
of Production from high level

ODD Analysis
Analysis performed on production
features through tests.
• Red light for no utilisation or
elicitation of feature
• Amber light when customer
utilisation test created
• Green light when customer
elicitation test solved

Integration of Analysis
Analysis integrated from events in a
situation tree to give complex
situations.
• Situations identified from
bottom-up
• Situations analysed to identify
hazards
• Hazards processed into Safety
Integrity Levels (SILs)
• Requirements found from SILs

Analysis Control
With Production we
create closed loop
control to develop
Analysis.
• Creating tests
from Production is
feedforwards
control
• Solving tests
allows a feedback
loop with control
of Analysis

Testing Spiral for Analysis Stage
Analysis stage elicits information on
Materials first and integrates to
Product.
• Continue in a circle until all tests
solved for abstraction level
• Efficient elicitation of customers
to all elements and levels

SIL Tree Diagram 1
SIL tree diagram creates situations
from events and processes
requirements.
• Severity and Controllability
added to each event
• Requirements found from SIL
ratings using branches
• Allows unit testing approach for
situations and requirements

SIL Tree Diagram 2
Adding SILs to a Probability Tree
allows requirement identification
and processing.
• Branching tree with SIL ratings
for events
• SILs found by multiplying along
branches of SIL tree
• Each situation given SIL rating to
determine requirements

SIL Tree Diagram 3
Processing SIL tree diagram is very
similar to a probability tree.
• SIL ratings processed by
multiplying probability, severity
and controllability
• SIL rating multiplied along
branches for each situation
• SIL result between 1 and 4, with
4 being best
Component 1
Pass
P(P1)*S(P1)*C(P1)
Component 2
Pass
P(P2)*S(P2)*C(P2)
Component 2
Fail
P(F2)*S(F2)*C(F2)
Component 1
Fail
P(F1)*S(F1)*C(F1)
Component 2
Pass
P(P2)*S(P2)*C(P2)
Component 2
Fail
P(F2)*S(F2)*C(F2)

SIL Tree Diagram 4
Using a SIL tree diagram gives
numerous advantages.
• Branches ensure all expected
situations identified
• Extendible and adaptable to new
situations by adding events
• Each branch is a situation
• New situations found from
adding new events
Component 1
Pass
P(P1)*S(P1)*C(P1)
Component 2
Pass
P(P2)*S(P2)*C(P2)
Component 2
Fail
P(F2)*S(F2)*C(F2)
Component 1
Fail
P(F1)*S(F1)*C(F1)
Component 2
Pass
P(P2)*S(P2)*C(P2)
Component 2
Fail
P(F2)*S(F2)*C(F2)

ODD Specification
Specification created from analysis
through requirement tests.
• Red light for a requirement
• Amber light when verification
test is created
• Green light when test passed and
behaviour validated

Decomposition of Specification
Specification decomposed from
high level behaviours to ensure high
level performance.
• Specifications created with top-
down process
• Level decomposed until all levels
specified
• Enables integration testing of a
solution

Linking Behaviours to Situations 1
Situations links to
Specification by
creating and solving
tests.
• Situations described
by a decision tree
• Diagram capable of
failure mode effects
and analysis
• Linked to a behaviour
which covers the
situation

Linking Behaviours to Situations 2
Branches of SIL tree
are situations to be
covered by
Specification.
• Each situation is tested
to ensure coverage by
specification
• Ensures situations are
covered before
creation of solution
• All expected situations
should have an
associated behaviour

Testing Spiral for Specification Stage
Specification stage tests Product
first and decomposes elements
until Materials achieved.
• Continue in a circle until all tests
are passed for abstraction level
to the next abstraction level
• Decomposition of Specification
from ensures high level features

Obstacle Driven Evolution 1
Spiral model adaptable to model an
evolutionary process.
• Dashed arrows represent failure
• Dashed arrows extending
outwards for too ambitious
• Dashed arrows contracting for
too complacent
• Failure in quadrant indicate
invention has failed stage
Note: Electric
car invented in
1835

Using the model we can plot why a
product or invention failed.
• Was analysis insufficient?
• Was Specification too ambitious
or complacent?
• Was Solution created too
expensively or too poorly?
• Was Production too high quality
or too little?
Note: Electric
car invented in
1835

ODD 3D Pyramid Model
Concurrent development of
multiple systems requires we
layer M-models of the systems.
• Pyramid shaped for
increasing number of
elements at lower levels
• Observable development
progress measurement

ODD Continuous
Process
Here we show 2 models which
have been linked into a cyclic
model.
• Process repeats for continuous
improvement
• Further stages may be added
• Proceed clockwise through
each stage

ODD Systems Engineering 1
Further stages added as necessary
to complete hardware system
development linked through tests.
• Safety Integrity Levels and
Situation Analysis provide
information on hazards
• Supply and Assemble give
hardware capabilities

We can divide SIL tress into
separate stages for additional error
checks.
• Situation Analysis identifies
situations
• Safety Integrity Levels process
situations into hazards
• Requirements analysis becomes
similar to Specification

For hardware we need to ensure
that physical elements are supplied
and assembled correctly.
• Supply stage handles supply of
physical hardware
• Assemble stage handles
assembly of physical hardware
• Completed through creating and
solving tests

Steps to creating and maintaining a
product are converted into an ODD
stage with some thought.
• Stages added should ideally
come in pairs
• Pairs of stages allow integration
and decomposition
• We can add, combine or reorder
the stages as required

Further divisions are added to the
ODE model so we can demonstrate
success or failure of that sector.
• Branches may be added for other
vehicle types e.g. trucks, buses
• Ability to investigate past failures
and plan future success
• Extendable to other scientific
disciplines

ODD OODA
Col. John Boyd’s OODA
method has been
combined with ODD to
produce new models.
• OODA = Observe,
Orient, Decide, Act
• Used for military
strategy for decades
• Similarity to OODA
effectively proves ODD

ODD OODA
OODA methods have been
used for many purposes.
• Originally for developing
military strategy
• Adapted to Artificial
Intelligence, business
strategy and legislation
• Adapted to be
compatible with ODD

ODDSAP OODA
Further stages are added to
provide steps for hardware
development.
• Feedback from all stages
to Analysis
• Decision included to
determine progression
• Outside Information and
Unfolding Circumstances
are also analysed

Generic Model OBSA
Comparing ODD with
OODA resulted in a
generic method to
describe many things.
• OBSA – Obstacle,
Behaviour, Solution,
Action

Other Uses
Generic model may be
used to create further
models.
• Shown is a model
for healthcare
adapted from
ODDSAP OODA
• Other methods for
legislation & AI
• Further methods to
be identified

ODD Materials
ODD is explained in further
presentations.
• Obstacle Driven
Development
• ODD: Requirements Analysis
• ODD: Extending a
Specification
• ODD: Extending TDD
• ODD: Extending V-models
• ODD Is Not Agile or Waterfall
ODD Is Not
Agile or
Waterfall
Obstacle Driven
Development
ODD:
Requirements
Analysis
ODD: Extending
a Specification
ODD: Extending
V-models
ODD:
Extending TDD

Further Information and Questions
Website
Presentations
Facebook
Twitter
Email

Legal Stuff
References
Test Driven Development for Embedded C
James Grenning, 2011
Test Driven Development
http://en.wikipedia.org/wiki/Test-driven development
Behaviour Driven Development
http://en.wikipedia.org/wiki/Behavior-driven development
Unit Testing
http://en.wikipedia.org/wiki/Unit testing
Disclaimer
The ODD M-model and associated processes are provided by odd.enterprises and may be
used for any purpose whatsoever.
The names odd.enterprises and associated logos should not be used in any representation,
advertising, publicity or other manner whatsoever to endorse or promote any entity that
adopts or uses the model and/or associated processes.
odd.enterprises does not guarantee to provide support, consulting, training or assistance of
any kind with regards to the use of the model and/or processes including any updates.
You agree to indemnify odd.enterprises and its affiliates, officers, agents and employees
against any claim or demand including reasonable solicitors fees, related to your use,
reliance or adoption of the model and/or processes for any purpose whatsoever.
The model is provided by odd.enterprises “as is” and any express or implied warranties,
included but not limited to the implied warranties of merchantability and fitness for a
particular purpose are expressly disclaimed.
In no event shall odd.enterprises be liable for any damages whatsoever, including but not
limited to claims associated with the loss of data or profits, which may result from any
action in contract, negligence or other tortious claim that arises out of or in connection with
the use or performance of the model.

ODD + Project Control 0.9

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (7)

Similar to ODD + Project Control 0.9

Similar to ODD + Project Control 0.9 (20)

More from Jonathan Herring

More from Jonathan Herring (12)

Recently uploaded

Recently uploaded (20)

ODD + Project Control 0.9