The document discusses the importance of software testing, emphasizing that all software developers make mistakes and therefore require robust testing strategies. It outlines various types of tests, methodologies, and the significance of prioritizing tests based on project needs, highlighting the benefits of tests in documentation, project management, and reducing development time. Additionally, it explores the challenges and pitfalls of testing, advocating for effective test-driven development and behavioral tests to ensure product quality.

1. Software testing
for people the who make mistakes – i.e. everyone but you ;-)
by Nico Heidtke

2. Tests?
• “There’s no time to write tests, we’re already late!”
• “Software testing proves the existence of bugs not their absence.”
• “If you don’t like testing your product, most likely your customers
won’t like to test it either.”
• “Software testers always go to heaven; they’ve already had their
fair share of hell.”
• “Code that cannot be tested is flawed.”
• “Why do we never have time to do it right, but always have time to
do it over?”

3. What can be
tested?
Unit / Component
Integration
Performance
Robustness
Security
Usability
Deployment
System
Acceptance

4. How can be
tested?
smoke
stress
regression
functional
negative
monkey
happy path
alpha / beta
manual

5. Your project
influences
your tests
• Audiences
• Types
• Phases
• Architectures
• Teams
Different projects …
• Test-Driven-Develop
• Acceptance tests
• Behavioral tests
• Manual QA test
• Continuous testing
• A/B testing
... mean different test strategies and methodologies

6. Not all projects are the same

7. Prioritize the
right tests
Don’t assume you will be given the time to write all
tests that are possible or required
Identify the most important aspects for project
Identify stable requirements and behavior
• Unit tests for algorithms with well defined and somewhat stable
• Integration tests for components with stable interfaces and life-
time
• Performance tests in cases of strict hardware limitations
• Acceptance / Behavioral tests for customer requirements that
are unlikely to change
• Stress and monkey tests if stability is paramount
• Security tests if you are exposed on the internet and a likely
hacking target

8. Why have
tests?
•Tests increase your confidence in your ability to deliver a working
product after changes. E.g. new features or refactoring
•Tests reduce the time spent in the “test phase” and speed up the
release process
Tests as a productivity tool
•Tests can serve as documentation for the development team and the
project management
•Tests can have legal utility for customer and supplier
Tests a communication device
•Test coverage and success can be used a metric for project progress
•Tests can be used to measure performance of developers
Tests a management utility
•Tests make you worry less about everything you are testing.
•Testing done manually is boring and expensive and makes people quit
Tests as anti-stress medicine

9. How can tests
go wrong?
Tests are a lot of work and often surpass the amount
of product code
Useless, redundant, obsolete and useless tests can
slow you down
Wrong tests could cause wrong code go by
unnoticed. Tests can have bugs too!
Plenty but bad tests might make you feel TOO safe
for your own good
Testing too much/early limits your creative and
productive space
Writing good tests is and testable code is anything
but easy.
Some aspects (e.g. UI) are difficult to test and might
not be worth the effort

10. Example:
std::physics
A C++ framework implementing SI physical units,
operators, conversions and formulas.
One explicit goal is to propose this library for an
upcoming C++ standard.
Open-source on github but with key contributors
like C++ developers and physicists working
for/sponsored by GOOGLE, NASA and CERN.
Development time is: “When it’s done”

11. Example:
EcoFleet
Show-case
A prototype for autonomous electric car fleet
management. Demonstrate possibilities efficient
charge cycling and routing of electric cars.
Sales requests great visuals and usability to
impress executives in the automotive industry.
Small team within a larger software company (4
developers + PM).
A first version is expected within only 2 month.
Continuous rapid improvement based on
feedback is expected.

12. Example:
PropMan
2020
Cloud-based custom development for a huge
property management firm. Large and diverse
amount of entities, relationships and users.
Distributed and diverse set of users and 3rd party
systems. A lot evolution of requirements and legal
regulations are expected during development and
maintenance.
Implemented by several groups (40+people) at a
large consulting firm. Partially executed on-site at the
customers offices.
Estimated 3-year development. New releases are
planned every 6 months. 12 years of maintenance
agreement.

13. Example: Bug
Hunt 2 –
Termite-
Nuclear War
A competitive multiplayer mobile game
for iPhone and Android.
In-app micro-transactions. Personal data
being collected. Target number of users:
500,000 within the first 3 month.
Initial development is expected to last
only 1 year and support for as long as the
game is profitable.
1 of 2 projects of a medium-sized game
company – about 8 developers in a single
scrum team.

14. Example:
AccessTool.exe
User-friendly and convenient centralized access
management tool to grant/revoke users access to
resource likes Dropbox, NAS, DBs, chats, wiki etc.
Planned for gradual adoption in-house only within
a retail company of 5000 people.
Started as a side project of a project manager and
3 developers one week ago.
No hard dead-line but expected to finish within 6
months and being updated as needed.

15. Test driven
development
• Write tests of whatever you are about to implement BEFORE
you start the implementation
The Idea:
• You translate requirements into tests directly thereby
documenting them
• You know when you are finished – when the tests work
• You have to start thinking before you start working on the
feature. Designing with testability in mind.
• You can’t forget to add tests later or be coerced to skip them
Pros:
• Your project could be exploratory and you maybe are not be
able to write good tests already
• You might easily create too many and redundant tests
• You hesitate to change your architecture if you have a high
amount of tests
Cons:

16. Behavioral
Testing
•Test system behavior as a “user” would do in a less abstract way.
Often tests are written in human-like language to make not-developers
able to understand or even write the tests
The Idea:
•Using common scenarios what is most close to the customer/users to
describe tests – thereby implicitly testing the most important aspects
•When the tests are working your product is guaranteed satisfy at the
fundamental functionality and user experience
•Bad user experiences become visible in the tests
•Behavioral tests are intuitive to communicate even for non-technical
people and can have legal utility
Pros:
•Non-development people increase communication overhead and can
seriously block the autonomy of development team.
•Behavioral tests can a lot of work to create and maintain
•Structuring and writing good behavioral tests is open to a lot of
interpretation
•Behavioral tests often require complicated/expensive 3rd party frameworks
for e.g. UI tests
Cons:

17. Behavioral test example (Gherkin)
GIVEN “Bob” is logged into the mobile client as “facility manager”
AND a “Replace broken floor light” job was reported to him by “Monika”
WHEN he opens the “Open Jobs” Panel
AND then selects a “Replace broken floor light” job
AND then clicks “Done” on the job
THEN “Bob” is shown a “Thank you” confirmation dialog
AND “Monika” is notified by email of the job completion
AND the job details are written to the long-term job archive

18. Don’t test bugs
out, DESIGN them out.
• Use the right language/tools - let the compiler/analyzers work for you
• Be safe where you can – e.g. type-safe, thread-safe, memory-safe
• Prefer well tested 3rd-party libraries and software
• Refactor to reduce the amount of code. Cleanup behind yourself
• Implement minimal, unambiguous and clear interfaces everywhere
• Follow best practices (e.g. SOLID principles for OOP)
• Make running, writing and reading your code and tests easy from the
very beginning
Lets be honest: We prefer writing production code over
test code. So let’s make sure we do not have to write as
much test code!