Slides of my 2nd version talk 'Unembedding embedded systems with TDD: Benefits of going beyond the Make-It-Work phase', delivered at Embo++2022 Conference on March 2022.
Little explanation and link to the talk here:
https://francliment.com/embedded-tdd/unembedding-embedded-systems-with-tdd-at-embo2022-conference/
11. “Software is this thing that can have a long
useful life, but firmware will become
obsolete as hardware evolves”
Updated definition:
.francliment.com
(Re)Definition of Firmware
Software engineer and author. Co-author of the Agile
Manifesto. Author of ‘Test Driven Development for
Embedded C’.
― James W. Greening, Clean Architecture
[Robert C. Martin]
12. “Software is this thing that can have a long
useful life, but firmware will become
obsolete as hardware evolves”
Updated definition:
.francliment.com
(Re)Definition of Firmware
Software engineer and author. Co-author of the Agile
Manifesto. Author of ‘Test Driven Development for
Embedded C’.
“Firmware is firmware because of what it
depends on and how hard is to change
those dependencies”.
― James W. Greening, Clean Architecture
[Robert C. Martin]
13. “Software is this thing that can have a long
useful life, but firmware will become
obsolete as hardware evolves”
Updated definition:
.francliment.com
(Re)Definition of Firmware
Software engineer and author. Co-author of the Agile
Manifesto. Author of ‘Test Driven Development for
Embedded C’.
“Firmware is firmware because of what it
depends on and how hard is to change
those dependencies”.
― James W. Greening, Clean Architecture
[Robert C. Martin]
14. “Software is this thing that can have a long
useful life, but firmware will become
obsolete as hardware evolves”
Updated definition:
.francliment.com
(Re)Definition of Firmware
Software engineer and author. Co-author of the Agile
Manifesto. Author of ‘Test Driven Development for
Embedded C’.
― James W. Greening, Clean Architecture
[Robert C. Martin]
“Firmware is firmware because of what it
depends on and how hard is to change
those dependencies”.
“… Non-embedded developers also write
firmware whenever they bury SQL in their code
or when they spread platform dependencies
throughout their code and business logic …”
15. 1. Make It Work
.francliment.com
Development as 3 strategic steps
3. Make It Fast
2. Make It Right
https://www.eetindia.co.in/engineers-in-the-covid-19-era/
https://www.ee,tindia.co.in/engineers-in-the-covid-19-era/
― Kent Beck, Test-Driven Development By Example, 2003
― Butler W. Lampson, ’Hints for Computer System Design’, 1983
― Stephen C. Johnson & Brian W. Kernighan, ’The C Language and Models for Systems Programming’, 1983
16. 1. Make It Work
.francliment.com
Development as 3 strategic steps
3. Make It Fast
2. Make It Right
https://www.eetindia.co.in/engineers-in-the-covid-19-era/
https://www.ee,tindia.co.in/engineers-in-the-covid-19-era/
― Kent Beck, Test-Driven Development By Example, 2003
― Butler W. Lampson, ’Hints for Computer System Design’, 1983
― Stephen C. Johnson & Brian W. Kernighan, ’The C Language and Models for Systems Programming’, 1983
Focused in understanding how to solve the problem, not about
engineering the code right
17. 1. Make It Work
.francliment.com
Development as 3 strategic steps
3. Make It Fast
2. Make It Right
https://www.eetindia.co.in/engineers-in-the-covid-19-era/
https://www.ee,tindia.co.in/engineers-in-the-covid-19-era/
― Kent Beck, Test-Driven Development By Example, 2003
― Butler W. Lampson, ’Hints for Computer System Design’, 1983
― Stephen C. Johnson & Brian W. Kernighan, ’The C Language and Models for Systems Programming’, 1983
Focused in understanding how to solve the problem, not about
engineering the code right
Engineer and craft the code right to make it easy to
understand, maintain and evolve
18. 1. Make It Work
.francliment.com
Development as 3 strategic steps
3. Make It Fast
2. Make It Right
https://www.eetindia.co.in/engineers-in-the-covid-19-era/
https://www.ee,tindia.co.in/engineers-in-the-covid-19-era/
― Kent Beck, Test-Driven Development By Example, 2003
― Butler W. Lampson, ’Hints for Computer System Design’, 1983
― Stephen C. Johnson & Brian W. Kernighan, ’The C Language and Models for Systems Programming’, 1983
Focused in understanding how to solve the problem, not about
engineering the code right
Engineer and craft the code right to make it easy to
understand, maintain and evolve
Fulfill system timing requirements (if any)
41. .francliment.com
• No long life SW (Domain, Application or Business Entities?)
• Nothing is modular or reusable
• Impossible to test any part in isolation
• Manual testing/debugging.
• Expensive and hardly repeatable ‘process’
Making It Work: MCU & HW PLATFORM FIRMWARE
PROBLEMS (common):
ALL EVIL SEEDS:
• Fragile
• Immobile
• Needless complexity
• Needless repetition
• Opaque
• Rigid
42. .francliment.com
PROBLEMS (Ditto due to being embedded):
• Expensive & scarce powering, input generation and output measurement
equipment is needed
Source: https://www.appliedinnovationalliance.com/wp-content/uploads/2017/12/Bottleneck-1.png
• Compiles only with the selected cross-compiler
• Executes only under an specific MCU family (MCUx)
• Runs only under an specific hardware platform (PCB assembly)
Making It Work: MCU & HW PLATFORM FIRMWARE
45. HW
GROUP
FW
GROUP
When new code is tested directly on Hardware (Debug on Hardware) and it does not
work as expected…
Where is the problem located?,
Is it HW?, is it FW?, both?
FW ERROR HW ERROR
.francliment.com
EMBEDDED TEAM?
Hardware Bottleneck problems: Defect localization
58. .francliment.com
Vendor’s IDEs, frameworks, utilities, toolchains,
libraries… MUST BE low level details properly
hidden & decoupled from our application
Embedded Traps: Vendor’s partnership
59. .francliment.com
The vendor is NOT our partner
Vendor’s IDEs, frameworks, utilities, toolchains,
libraries… MUST BE low level details properly
hidden & decoupled from our application
Embedded Traps: Vendor’s partnership
60. .francliment.com
The vendor is NOT our partner
Vendor’s IDEs, frameworks, utilities, toolchains,
libraries… MUST BE low level details properly
hidden & decoupled from our application
Embedded Traps: Vendor’s partnership
The running processor is a low level DETAIL
68. .francliment.com
• ISO 26262 for automotive electric/electronic systems.
• EN 50128 for railway applications.
• IEC 62304 for medical devices.
• IEC 62061 for machinery systems.
IEC 61508
Functional safety of electrical/electronic/programmable
electronic safety-related systems
Embedded Traps: Compliance Mindset
87. .francliment.com
Takeaways: Better protect yourself
Nobody but the organization itself
is going to take care about its
reputation, development,
aftersales and health costs
90. 1. Make It Work
.francliment.com
https://www.eetindia.co.in/engineers-in-the-covid-19-era/
Development as 3 strategic steps
Red
Green
Refactor
Focused in understanding how to solve the problem, not about
engineering the code right
91. 1. Make It Work
.francliment.com
https://www.eetindia.co.in/engineers-in-the-covid-19-era/
Development as 3 strategic steps
Red
Green
Refactor
Make It Work
Focused in understanding how to solve the problem, not about
engineering the code right
92. 1. Make It Work
.francliment.com
https://www.eetindia.co.in/engineers-in-the-covid-19-era/
Development as 3 strategic steps
Red
Green
Refactor
Make It Work
KISS: Keep It Simple, Stupid
YAGNI: You Aren’t Gonna Need It
Focused in understanding how to solve the problem, not about
engineering the code right
93. 1. Make It Work
.francliment.com
https://www.eetindia.co.in/engineers-in-the-covid-19-era/
Development as 3 strategic steps
Red
Green
Refactor
Make It Work
KISS: Keep It Simple, Stupid
YAGNI: You Aren’t Gonna Need It
“All sins are allowed”
Focused in understanding how to solve the problem, not about
engineering the code right
94. Red
Green
Refactor
2. Make It Right
.francliment.com
https://www.eetindia.co.in/engineers-in-the-covid-19-era/
Development as 3 strategic steps
Engineer and craft the code right to make it easy to
understand, maintain and evolve
Make It Work
KISS: Keep It Simple, Stupid
YAGNI: You Aren’t Gonna Need It
“All sins are allowed”
95. Red
Green
Refactor
2. Make It Right
.francliment.com
Development as 3 strategic steps
Engineer and craft the code right to make it easy to
understand, maintain and evolve
Make It Work
Make It Right
KISS: Keep It Simple, Stupid
YAGNI: You Aren’t Gonna Need It
“All sins are allowed”
Test code only
DAMP : Descriptive And
Maintainable Procedures
2
KISS : Keep it Simple, Stupid
1
DRY : Don’t Repeat Yourself
1
2
Production code only
1
96. Red
Green
Refactor
2. Make It Right
.francliment.com
https://www.eetindia.co.in/engineers-in-the-covid-19-era/
Development as 3 strategic steps
Engineer and craft the code right to make it easy to
understand, maintain and evolve
Make It Work
Make It Right
KISS: Keep It Simple, Stupid
YAGNI: You Aren’t Gonna Need It
“All sins are allowed”
Test code only
DAMP : Descriptive And
Maintainable Procedures
2
KISS : Keep it Simple, Stupid
1
DRY : Don’t Repeat Yourself
1
2
Production code only
1
Simple Design,
SOLID,
Design Patterns…
105. .francliment.com
“We can get the same benefits
following a Test-Later approach”
Fallacies & Mantras: Test-Later vs Test-First
106. .francliment.com
“We can get the same benefits
following a Test-Later approach”
Test-Later are testing techniques focused on
External Quality only
Fallacies & Mantras: Test-Later vs Test-First
107. .francliment.com
Takeaways: Quality is not coffe sugar
― Harold F. Dodge
“Inspection does not improve the quality,
nor guarantee quality.
Inspection is too late.
The quality, good or bad, is already in the product”
109. 3. Make It Fast
.francliment.com
https://www.eetindia.co.in/engineers-in-the-covid-19-era/
Development as 3 strategic steps
Make It Right
Fulfill timing requirements
111. .francliment.com
“Even if you know exactly what is going on in your system,
measure performance, don’t speculate.
You will learn something, and 9 times of 10, it won’t
be that you were right!”
― Ron Jeffries, Refactoring: Improving the Design of
Existing Code [Fowler]
Embedded Trap: Nedless performance
112. 3. Make It Fast (enough)
.francliment.com
https://www.eetindia.co.in/engineers-in-the-covid-19-era/
Development as 3 strategic steps
Make It Work
Make It Right
Fulfill timing requirements (only when & where are really
needed).
113. 3. Make It Fast (enough)
.francliment.com
https://www.eetindia.co.in/engineers-in-the-covid-19-era/
Development as 3 strategic steps
Red
Green
Refactor
Make It Work
Make It Right
Fulfill timing requirements (only when & where are really
needed).
1. Add the timing spec
114. 3. Make It Fast (enough)
.francliment.com
https://www.eetindia.co.in/engineers-in-the-covid-19-era/
Development as 3 strategic steps
Red
Green
Refactor
2. Make It Work
Make It Right
Fulfill timing requirements (only when & where are really
needed).
1. Add the timing spec
KISS: Keep It Simple, Stupid
YAGNI: You Aren’t Gonna Need It
“All sins are allowed”
115. 3. Make It Fast (enough)
.francliment.com
https://www.eetindia.co.in/engineers-in-the-covid-19-era/
Development as 3 strategic steps
Red
Green
Refactor
2. Make It Work
Make It Right
Fulfill timing requirements (only when & where are really
needed).
1. Add the timing spec
KISS: Keep It Simple, Stupid
YAGNI: You Aren’t Gonna Need It
“All sins are allowed”
Test code only
DAMP : Descriptive And
Maintainable Procedures
2
KISS : Keep it Simple, Stupid
1
DRY : Don’t Repeat Yourself
1
2
Production code only
1
Simple Design,
SOLID,
Design Patterns…
116. .francliment.com
Takeaways: Right tunable software
“The secret of fast software is to write tunable
software first and then tune it for
sufficient speed”
― Martin Fowler, Refactoring: Improving the Design of
Existing Code
118. .francliment.com
Common Traps: Knowledge & Skills
“The competent programmer is fully aware of the limited size of his
own skull. He therefore approaches his task with full humility, and
avoids clever tricks like the plague.”
— Edsger W. Dijkstra
121. Embedded TDD: Dual-Target work flow
.francliment.com
HOST DEVELOPMENT ENVIRONMENT
HOST BASED FLOW
Embedded TDD cycle
Source: “Test-Driven Development for Embedded C” book, James Greening
122. .francliment.com
HOST DEVELOPMENT ENVIRONMENT
HOST BASED FLOW
Generate
self-checking
specs
Embedded TDD cycle
Embedded TDD: Dual-Target work flow
Source: “Test-Driven Development for Embedded C” book, James Greening
123. Embedded TDD cycle
.francliment.com
HOST DEVELOPMENT ENVIRONMENT
HOST BASED FLOW
Generate
self-checking
specs
Embedded TDD: Dual-Target work flow
Fulfill the
specs
Source: “Test-Driven Development for Embedded C” book, James Greening
124. .francliment.com
HOST DEVELOPMENT ENVIRONMENT
HOST BASED FLOW
Generate
self-checking
specs
Fulfill the
specs
Improve the
solution
Embedded TDD cycle
Embedded TDD: Dual-Target work flow
Source: “Test-Driven Development for Embedded C” book, James Greening
125. .francliment.com
HOST DEVELOPMENT ENVIRONMENT
HOST BASED FLOW
Generate
self-checking
specs
Review results and metrics
Embedded TDD cycle
Embedded TDD: Dual-Target work flow
Fulfill the
specs
Improve the
solution
Source: “Test-Driven Development for Embedded C” book, James Greening
126. .francliment.com
HOST DEVELOPMENT ENVIRONMENT
HOST BASED FLOW
Generate
self-checking
specs
Review results and metrics
Embedded TDD cycle
Cadence: 1 min
Embedded TDD: Dual-Target work flow
Fulfill the
specs
Improve the
solution
Source: “Test-Driven Development for Embedded C” book, James Greening
127. Embedded TDD cycle
.francliment.com
HOST DEVELOPMENT ENVIRONMENT
HOST BASED FLOW
TARGET BASED FLOW
Generate
self-checking
specs
Review results and metrics
Embedded TDD: Dual-Target work flow
Fulfill the
specs
Improve the
solution
Source: “Test-Driven Development for Embedded C” book, James Greening
128. Embedded TDD cycle
.francliment.com
HOST DEVELOPMENT ENVIRONMENT
HOST BASED FLOW
TARGET BASED FLOW
Generate
self-checking
specs
Cross
compile
Review results and metrics
Embedded TDD: Dual-Target work flow
Fulfill the
specs
Improve the
solution
Source: “Test-Driven Development for Embedded C” book, James Greening
129. Embedded TDD cycle
.francliment.com
HOST DEVELOPMENT ENVIRONMENT
HOST BASED FLOW
TARGET BASED FLOW
Generate
self-checking
specs
Cross
compile
Eval Hardware
Review results and metrics
Embedded TDD: Dual-Target work flow
Fulfill the
specs
Improve the
solution
Source: “Test-Driven Development for Embedded C” book, James Greening
130. Embedded TDD cycle
.francliment.com
HOST DEVELOPMENT ENVIRONMENT
HOST BASED FLOW
TARGET BASED FLOW
Generate
self-checking
specs
Cross
compile
Eval Hardware
Review results and metrics
Embedded TDD: Dual-Target work flow
Fulfill the
specs
Improve the
solution
Source: “Test-Driven Development for Embedded C” book, James Greening
131. Embedded TDD cycle
.francliment.com
HOST DEVELOPMENT ENVIRONMENT
HOST BASED FLOW
TARGET BASED FLOW
Generate
self-checking
specs
Cross
compile
MCU Simulator
Review results and metrics
Embedded TDD: Dual-Target work flow
Fulfill the
specs
Improve the
solution
Source: “Test-Driven Development for Embedded C” book, James Greening
132. Embedded TDD cycle
.francliment.com
Source: “Test-Driven Development for Embedded C” book, James Greening
HOST DEVELOPMENT ENVIRONMENT
HOST BASED FLOW
TARGET BASED FLOW
Generate
self-checking
specs
Cross
compile
MCU Simulator
Review results and metrics
Cadence: 5 min
Embedded TDD: Dual-Target work flow
Fulfill the
specs
Improve the
solution
133. .francliment.com
NO FEAR
Dual-Target Embedded TDD: Main Benefits
OUTSIDE
Embedded
limits
• Incrementally developed
• Comprehensively tested at least on two different platforms
• Fast feedback loop: courage to try & learn fast
134. .francliment.com
NO FEAR
Dual-Target Embedded TDD: Main Benefits
OUTSIDE
Embedded
limits
NO LIMITS
✓ Static analysis, rule checkers …
✓ Dynamic análisis: xUnit, mutation testing
✓ Continuous Integration / Continuous Testing
New tools finally avilable:
Working on
99% code base
136. TARGET
- Dynamic analysis:
- Unit testing (Ceedling + XC + PIC (mdb) simulator)
- Integration testing (Ceedling + XC + PIC (mdb) simulator)
- Metrics / Trends:
- Target compilation warnings (Ceedling + XC, at project level).
- ROM & RAM used (Ceedling + XC, at project level).
- STACK used (Ceedling + XC, at project level).
XC
.francliment.com
Dual-Target Continuous Integration / Continuous Testing
137. .francliment.com
XC
Build result ‘#17- 44-modbus_controller’: UNSTABLE
Artifacts associated to build ‘#17- 44-modbus_controller
Changes associated to the push originating the build
‘#17- 44-modbus_controller
Coverage statistics for build
‘#17- 44-modbus_controller
function is_Synt_lock
@fraclipe
Tests results for build ‘#17- 44-modbus_controller
Warnings detected for build ‘#17- 44-modbus_controller
Warnings’ changes from last build
Identified probles for build
‘#17- 44-modbus_controller
Build Log
XC Warnings
44-modbus_controller #17 - 44-modbus_controller
modbus controller
Dual-Target Continuous Integration / Continuous Testing
138. .francliment.com
NO FEAR
Dual-Target Embedded TDD: Main Benefits
OUTSIDE
Embedded
limits
NO LIMITS
✓ Static analysis, rule checkers …
✓ Dynamic análisis: xUnit, mutation testing
✓ Continuous Integration / Continuous Testing
New tools finally avilable:
Working on
99% code base
Increased competitiviness:
✓ Fewer defects
✓ Development & lead time reduced
✓ Sustainable pace
✓ Anybody, anywhere, at any moment
✓ Improved talent acquisition and retention
✓ Better market adaptability through portable solutions
139. .francliment.com
NO FEAR
Dual-Target Embedded TDD: Main Benefits
OUTSIDE
Embedded
limits
NO LIMITS
NO EXCUSES
Complete portable solutions
Reusable modules
Domain Driven Embedded
Software
Make It Right
149. .francliment.com
Takeaways: Reusable IC Drivers test suite goals
IC Drivers test suite GOALS
Capture our IC datasheet interpretation, allowing:
▪ Implement the IC Driver accordingly to it without introducing errors
▪ Hardware Bottleneck related uncertainties and delays removal
150. Making It Right: Reusable Drivers implementation (Dual-Targetted)
.francliment.com
154. PAL production code example: ADC reading independent of MCU
.francliment.com
155. .francliment.com
Takeaways: MCU PAL test suite goals
MCU PAL test suite GOALS
Capture our MCU datasheet interpretation, allowing:
▪ Implement its PAL accordingly to it without introducing errors.
▪ Use the MCU in our solution.
▪ Hardware Bottleneck related uncertainties and delays removal.
176. .francliment.com
Takeaways: Application test suite goals
Application test suite GOALS
Capture product specifications, allowing:
▪ Implement them without introducing errors.
▪ Implement them safely and incrementally.
▪ Specify them for different system or product configurations.
▪ Hardware Bottleneck related uncertainties and delays removal.
▪ Any type: functional, load, performance…
187. .francliment.com
“Debugging is part of development”
Debugging is a SHAME!
While debugging we are not creating anything
new: we are adding zero value to our
organization
Fallacies & Mantras: Debugging
194. We can’t be Agile if
our code sucks!
.francliment.com
Takeaways: Beware of Flacid Agile
195. We can’t improve
our code if we fear
to refactor it
.francliment.com
Takeaways: Acquired Incompetence
196. We can’t fearless refactor it
without a comprehensive
suite of automated tests
.francliment.com
Takeaways: Comprehensive safety net
197. “Raise your quality standards as high as you
can live with, avoid wasting your time on routine problems, and always try
to work as closely as possible at the boundary of your abilities. Do this, because
it is the only way of discovering how that
boundary should be moved forward”
.francliment.com
Takeaways: Continuous improvement
― Edsger W. Dijkstra