a talk by Alexander Grosse (soundcloud) & Alexander Schwartz (Nokia) for the Agile Testing Days 2011 (http://www.agiletstingdays.com)

1. Automated testing
of complex service
oriented
architectures
ALEXANDER GROSSE
ALEXANDER SCHWARTZ

2. Who are we?
• ALEXANDER SCHWARTZ (NOKIA)
• ALEXANDER GROSSE (SOUNDCLOUD)

3. SOUNDCLOUD
• World’s leading sound sharing
platform
• HQ in Berlin with 70 employees
• Guess how many QA ROLES?

4. PROBLEM

5. PROBLEM
• Service oriented architecture (REST)
• Evolve services independently from
each other(loose coupling)
• Release often

6. COMMON SOLUTION
• Chaos
• Full regression tests
• Hide behind client
• Release in big intervals

7. WHAT IS THE
PROBLEM WITH FULL
REGRESSION TESTS?
• Users might only use Part of the API
• results in unnecessary version
increases

8. PROVIDER
CONTRACTS
• Provider Contracts bind the consumer to
the entirety of the provider’s functionality
• Irrespective of how much of this
functionality it really needs
• Think WSDL + XML
• As the name indicates: Provider Centric!
• loose coupling?

9. WHAT IS ACTUALLY
USED?
obj1
/ Resp
obj2
/
obj3
/
obj4
/
obj5
/
obj6
/

10. CDC’S
• contracts are becoming consumer centric
not provider centric
• means the consumer writes the contract
• provider still has to accept it

11. HOW IS A CONTRACT
MADE?
• Consumer team develops the usage of the
api
• Writes an executable test case
• discusses that test case with the provider
team
• Provider accepts by integrating into CI

12. WELCOMED SIDE
EFFECTS
• communication
• better use of API’s
• Responsibility for failed releases changes
• SOA is not RPC!

13. BUT...
• API’s are also used by externals
• the exact same procedure cannot be
repeated with external consumers

14. WHAT TO DO?
• Choose lead customers
• Analyze log files to get the used resources
(but not how responses are being used)

15. ALEX S STARTS!
;)

16. Alex Schwartz
•Background: Programming
•20+ years in IT
•Scrum Master, 7+ years agile
•Sr. Release and Test Specialist
•Head of a Delivery Team
-DevOps
-Kanban
-Pipelines
-Continuous Delivery

17. Nokia gate5 Berlin ng!
h iri
We a re
•500+ people
•from 40+ countries
•very international
•provides maps for web and devices,
and other applications
•100+ open positions

19. Places Backend Services - Architecture
2 © 2011 Nokia Company Confidential v1.0 2011-06-16 Alexander Schwartz CDCs@PlacesBackend

20. Different Change Frequencies
- frozen
- multiple versions
- frozen, legacy
- one versions
- frequent changed
- one version
(during rollout: two versions)
2 © 2011 Nokia Company Confidential v1.0 2011-06-16 Alexander Schwartz CDCs@PlacesBackend

21. CDCs - How to verify coverage?
Talk to the client team!
Expose each bug with a test.
Double check:
• Find all the requests: analyze the access log
• which part of the response is used?
• analyze the code
7

22. Services and Pipelines
Places
Backend
API
<<REST services>>

23. Services and Pipelines
Places
Registry
<<REST>>
Places
Search
<<REST>> Backend
API
Recommendations <<REST services>>
<<REST>>

24. Services and Pipelines
Places
Registry
<<REST>>
Account
Places
Service Search
<<REST>> <<REST>> Backend
API
Recommendations <<REST services>>
<<REST>>

25. Services and Pipelines
City Pages
<<webapp>>
Places
Registry Maps app for
<<REST>>
Windows 7
<<device app>>
Account
Places
Service Search
<<REST>> <<REST>> Backend Maps Web
<<webapp>>
API
Recommendations <<REST services>>
PBAPI client
<<REST>>
API
<<client API>>

26. Services and Pipelines
City Pages
<<webapp>>
Places
Registry Maps app for
<<REST>>
Windows 7
<<device app>>
Account
Places
Service Search
<<REST>> <<REST>> Backend Maps Web
<<webapp>>
API
Recommendations <<REST services>>
PBAPI client
<<REST>>
API
<<client API>>
Build Pipeline Order

27. Services and Pipelines
City Pages
<<webapp>>
Places
Registry Maps app for
<<REST>>
Windows 7
<<device app>>
Account
Places
Service Search
<<REST>> <<REST>> Backend Maps Web
<<webapp>>
API
Recommendations <<REST services>>
PBAPI client
<<REST>>
API
<<client API>>
Upstrea Downstream
Build Pipeline Order

28. CDC Example
Use a builder
pattern.
6

29. CDC Example
Use a builder
pattern.
The provider provides a library to
simplify CDC suites.
6

30. How we manage the CDCs?
Provider Consumer Side
Library
lib-cdc
CDC S60-to-
Provider app code repo
vvv
- Test suite CDCs
7

31. CDC Example w/ cucumber
6

32. CDCs - Test Data
• best practice: every single test controls its test
data
• options:
• create data
• use existing data
• use a controlled data set
• depends on context of the test system
7

33. CDCs - How to verify coverage?
Talk to the client team!
Expose each bug with a test.
Double check:
• Find all the requests: analyze the access log
• which part of the response is used?
• analyze the code
7

34. Q&A
7