Cash to concept. Because when you already have an application that is in the hands of your users, delivering value, it can be quite a challenge to get back in control of quality. So while most examples start from nothing and show how to build up to a high-quality app, in this workshop we start with an existing app and reverse engineer it. We’ll go back to defining the high-level product map and show how to fill in all the unknowns to get a complete overview of functionality, generate some test metrics and write solid functional tests. To know what we test, we need to have a detailed overview of the functionality of our application. Then we can decide where our risks are, and start creating the tests that will get us to safety. In this workshop we create a functional overview of our application, use Story Mapping to probe and prioritize our functionality, and Example Mapping to generate our tests. Then we show how to use the results to get control using first manual testing scripts and how to decide what and how to automate so you can get in that coveted continuous delivery mode. In our practice, we often encounter organisations that have existing applications without any defined, structured testing in place, let alone a strategy for automated testing. Going from that situation to one where we can move towards more frequent releases requires going back towards product definition, gaining a top-down overview, before moving back down into detail to create first a manual regression test and then automate at different levels to get to something that can suitably be called agile testing. This workshop aims to be an extract of the steps we take to gain control in those situations. Since the aim is to not just get to a test strategy, but also to get the POs, testers and developers to learn practices they can use in the continued developing of the product, we stay close to the standard tools for agile requirement and test definition, using BDD as our base. The workshop is run in a ‘mob programming’ type setup: small teams (~6 people), around each table, building in short cycles where we move direction and execution through the team to keep everyone engaged. Each table works through one story map (a selected user flow/journey of an existing application) and we then let the teams select specific slices through that flow as suitable for manual end-to-end tests, initial outside-in automation tests and then prioritise the more detailed story and business rule based tests that come later.

1. @wouterla #lascot @karelboekhout
Ca$h to conc€pt
Bringing a legacy system under test

2. @wouterla #lascot @karelboekhout
Who are we?
Karel Boekhout
@karelboekhout
Wouter Lagerweij
@wouterla
2

3. @wouterla #lascot @karelboekhout
You know this one...
3
Source: Jeff Patton

4. @wouterla #lascot @karelboekhout
Let’s create this one!
4
Why?
● Usually not greenfields, but a big ball of legacy application
● No tests available
● Practical: you can start today

5. @wouterla #lascot @karelboekhout 5
What
does this
%^)#$
application do?
Let’s start with a
Mental Breakdown
What do we have?
What does it do?
Will it keep
working?
Does it do
what you
think it
does?
Do you care?

6. @wouterla #lascot @karelboekhout
Our example
6
Let’s start with a
Functional Breakdown

7. @wouterla #lascot @karelboekhout
Spotify: Home
7
Home screen
<suggestions>
More of what you like
Made for ….
Keep the vibe going
Recently played
Mood
Your heavy rotation

8. @wouterla #lascot @karelboekhout
Spotify: Search
8
Search
Browse all
Your top genres
Search bar

9. @wouterla #lascot @karelboekhout
Spotify: Library
9
Your Library
Songs
Stations
Playlists
Made for You
Albums
Artists
Podcasts
Settings
Recently Played
Profile

10. @wouterla #lascot @karelboekhout
Spotify: Play
10
Player
Manage devices
Artist
Player controls
TitleAdd to my library
Share to socials
Cover
Playlist
Extra actions
Back to Home
Behind the lyrics

11. @wouterla #lascot @karelboekhout
Start a ‘Heatmap’
The Functional Breakdown
Colour coded for the testing we have available
The Heatmap
● Our overview for the level of control over our system:
○ What are we testing?
○ How are we testing it?
11

12. @wouterla #lascot @karelboekhout
Example of advanced Heatmap
12

13. @wouterla #lascot @karelboekhout
Heatmap
13
Home
Search
Library
Play
These are the created tests.
We trust they are well executed. 😇

14. @wouterla #lascot @karelboekhout
Choose or lose: Priorities
Recent: new features, new areas of code are more vulnerable
Core: essential functions must continue to work
Risk: some areas of an application pose more risk
Configuration sensitive: code dependent on settings can be vulnerable
Repaired: bug fixes can introduce new issues
Chronic: some areas in an application may be perpetually sensitive to breaking
RCRCRC by Karen Johnson, via Matt Heusser
14
We pick Search

15. @wouterla #lascot @karelboekhout
Back to the Heatmap
15
Search
Home
Library
Playlist
Manage Playlist
Play

16. @wouterla #lascot @karelboekhout
What is a Story Map?
16
● Follow the flow through the application (user activities)
● Specific User Stories in that flow (detailed actions)
● Slices (things that go well together)
● Priority (what do we do/test first?)
● Difference with building new functionality:
○ Priorities
○ Testing effort instead of implementation effort
○ Dependencies
○ Granularity
○ We know more!
User Story Mapping by Jeff Patton

17. @wouterla #lascot @karelboekhout
Our Story Map of Manage Playlist
17
Create and play
Find and edit
Sort
Share
Later

18. @wouterla #lascot @karelboekhout
Back to the Heatmap
18
Search
Home
Create and play
Library
Playlist
Manage Playlist
Play
Find and edit
Sort
Share

19. @wouterla #lascot @karelboekhout
How to make a Story Map
● We start with flow horizontally (orange)
● Each step in the flow can have multiple stories (yellow)
● Use the vertical to create different levels of priority (slices) (pink)
● Make each horizontal ‘slice’ something you can give a name to
● We do this together (three amigos)
19

20. @wouterla #lascot @karelboekhout
Story Mapping of Spotify Search
20
Search
Browse all
Your top genres
Search bar
Now you try!
Recent
Core
Risk
Configuration sensitive
Repaired
Chronic

21. @wouterla #lascot @karelboekhout
Story Mapping: let’s discuss!
● Do you have slices?
● Can you name each slice?
● Why in this order?
21

22. @wouterla #lascot @karelboekhout
Using the outcomes of Story Map
● Each slice can be named as a separate box in the functional overview
● Each separate box can be an end-to-end test
● Actually creating those tests turns the heatmap yellow
● Stories generated can be starting points for more detailed exploration
22
Validation
Automated E2E
Low Level
Unknown
Manual

23. @wouterla #lascot @karelboekhout
Back to the Heatmap
23
Validation
Automated E2E
Low Level
Unknown
Manual
Search
Home
Create and play
Library
Playlist
Manage Playlist
Play
Find and edit
Sort
Share

24. @wouterla #lascot @karelboekhout
Validation
Each slice from the Story Map can be a test
Search for a Song
<assumes app opened, user logged in>
Go to Search
Search for (part of) a Song’s name
Tap the Song to start it playing
24

25. @wouterla #lascot @karelboekhout
Validation
Each slice from the Story Map can be a test
Recent Searches
Go to Search
Search for any term
Go back to Home
Go back to Search
See term appear on top of Recent Searches
Delete term from Recent Searches
Check by restarting Search
Delete all items from Recent Searches
Check by restarting Search
25

26. @wouterla #lascot @karelboekhout
Automate?
Wait.
We’re finding out what functionality there is to be tested
Let humans test, don’t restrict them more than necessary
Once you’ve discovered what is interesting about the functionality, maybe,
automate
Or, better, go into detail with an example map to get more high resolution tests
26

27. @wouterla #lascot @karelboekhout
Back to the Heatmap
27
Validation
Automated E2E
Low Level
Unknown
Manual
Search
Home
Create and play
Library
Playlist
Manage Playlist
Play
Find and edit
Sort
Share

28. @wouterla #lascot @karelboekhout
Validation
● High-level, low resolution, human end-to-end testing
● High-level, low resolution, automated end-to-end testing
● Low-level, high resolution, automated example based tests
28
Validation
Automated E2E
Low Level
Unknown
Manual

29. @wouterla #lascot @karelboekhout
Validation: Example Mapping
For a story:
- List the business rules
- Provide examples that illustrate the rule
- Capture open questions
Centered around business rules
Examples can be formalised (in Gherkin, or otherwise) in the form of a test:
Input -> Action -> Outcome (Given/When/Then, Arrange/Act/Assert)
29
Example Mapping by Matt Wynne

30. @wouterla #lascot @karelboekhout 30
Validation: Example Mapping

31. @wouterla #lascot @karelboekhout
Examples of Examples
31

32. @wouterla #lascot @karelboekhout
Example Mapping
32
Now you try!

33. @wouterla #lascot @karelboekhout
Example Mapping
Let’s discuss!
33

34. @wouterla #lascot @karelboekhout
Examples can be formalised into Scenarios
Scenario: Playlist without name is auto-numbered
Given existing Playlists
| My Playlist #1 |
| My Playlist #2 |
When a new playlist without name is added
Then that new playlist is given the name ‘My Playlist #3’
Scenario: Playlist without name gets next available slot
Given existing Playlists
| My Playlist #1 |
| My Playlist #3 |
When a new playlist without name is added
Then that new playlist is given the name ‘My Playlist #2’
34

35. @wouterla #lascot @karelboekhout
pseudocode
Scenarios are implemented at same level the
functionality lives
Scenario: Playlist without name is auto-numbered
Given existing Playlists
| My Playlist #1 |
| My Playlist #2 |
When a new playlist without name is added
Then that new playlist is given the name ‘My Playlist #3’
function testPlaylistNameAutoNumber() {
playlists.addAll(‘My Playlist #1’, ‘My Playlists #2’)
playlists.add(‘’)
expect(playlists.getAll()).toContain(‘My Playlist #3’)
}
35

36. @wouterla #lascot @karelboekhout
Back to the Heatmap
36
Validation
Automated E2E
Low Level
Unknown
Manual
Search
Home
Create and play
Library
Playlist
Manage Playlist
Play
Find and edit
Sort
Share

37. @wouterla #lascot @karelboekhout
Future?
Test Result
Failed
Attention
😄
37
Validation
Automated E2E
Low Level
Unknown
Manual
Search
Home
Create and play
Library
Playlist
Manage Playlist
Play
Find and edit
Sort
Share

38. @wouterla #lascot @karelboekhout
Takeaways: Get back in control
○ What does it do? (functional breakdown)
○ Is it tested? (heatmap)
○ Go broad for overview (Story Map, manual/e2e tests)
○ Dive deep where necessary (Example Map, detailed tests)
○ Iterate!
38

39. @wouterla #lascot @karelboekhout 39
Thank you!
And...
If you have a problem
If no one else can help
This is how you find us:
wouter@lagerweij.com
karel@hedgefields.com