This document discusses challenges with test automation and best practices for sustainable test automation. It recommends addressing manual testing efforts, long feedback times, and operational blindness. Test automation should be treated as an activity that is part of the definition of done and not everything needs to be automated. Risk management is important and the Pareto principle applies. Test cases need maintenance over time. Data management techniques include independent test data, external storage, and microservices. Embrace microservices, learn about data virtualization, and make test automation part of the CI/CD pipeline. BDD, page objects, and separating framework, page and test layers are advised.

1. SUSTAINABLE TEST
AUTOMATION
COLLABORATE
WITHIN TEAM
Berk DUlger
SQA Days 2017, St. Petersburg

3. BusinessKnow-How Asymmetry
Changing Requirements
ImproperRisk Management
Tool Dependence
Insufficient Time
UntestableCode
Test AutomationAnti-Patterns
Problematic Data Management
Monolithic Applications
Technical Debt
Over Expectations
Unclear Duties
Challenges
in
Test Automation

4. Any Test Automation ACTIVITY should address
Manual Testing Efforts
Long feedback times
Operatıonal Blındness
Error-Prone Actıvıtıes

5. We need to make test automation
SUSTAINABLE

6. Sustainable success
requires both
bottom-up practices
and top-down
management support

7. Handle Test automatıon As an Actıvıty
Make part of Definition of Done

8. It’s not Reasonable to automate everything
Manualtesting is strongly advised andALSO
required for SOMETest Level/types

9. Learn RISK MANAGEment
IDENTIFY, ASSESS and MITIGATE RISKS
Be Friend of P-I Tables

10. %80 of Customer Usage arises from
%20 Functionality, Pareto Prınciple
No need to automateAll of Your Test CaseS

11. More cases you develop, More Maıntenance
effort you face
Soon or Later, Logistics curve will be here
Get rid of old test cases

12. Continuous Testing E-commerce Case Study
InflectionPoint
2-3TestCasesperMan/Day
NearlyNoMaintanceEffort
3-5TestCasesperMan/Day
LessMaintance Effort (%20)
2-3TestCasesperMan/Day
ModerateMaintanceEffort (%70)
3 TestCasesperMan/Day
ModerateMaintanceEffort (%50)
~1TestCasesper Man/Day
HeavyMaintanceEffort (%90)
Basedon metricsfrom 14 consultancyprojects

13. Test Automation BEST Practices
Software Testıng Ice-Cone Antı-Pattern
vs
Ideal Software testıng pyramıd
IMAGE, www.thoughtworks.com/insights/blog/architecting-continuous-delivery

14. Four Design Techniques for Successful
Test Automation Data Management
A typical maturity level of data management for test
automationprocess is outlinedhere;
FullyIntegrated Test Data
PartiallyIndependentTest Data
StoringTest Datainan External Source
DynamicTest Data Management (MicroServices, GUI ?)

15. EMBRACE MICROServıces
At Least Services
ımage https://martinfowler.com/articles/microservices

16. Learn more about Data Virtualization
It makes test data management easier
An added advantage here is thatif the data in any environmentis
corrupted – let’s say after a round of testing- it is very easy to
revert the data back to original state.

17. Make Test Automation, Backbone of the CI/CD Pipeline

18. Test Automation,
like other CI/CD
activities,is
extremely
CROSS-DISCIPLINARY
And require
different
skill and mind-set

19. MAKEEveryoneintheTeam
InvolvedintheProcess,
3-Amigos
Analyst
Developer
Tester

20. BDDfocuses on obtaining
a clear understandingof
desiredsoftware
behaviorthrough
discussion with
stakeholders.
Martin Fowler’s Business Readable, Domain Specific
Language Blog Post

21. Another crucial point is to provide an
abstraction layer between automation
tool and test sUite.
Selenium, Sahi, UFT, Coded UI
whatever tool underlying
should not be concern
for automator

22. Three LayerArchitecture andBDD

23. A Test Architect is responsiblefor
FRAMEWORK LAYER, whichprovides
utility functionsto upper layers
It is heavily one-timeeffort, and not
addressedin the project iterations

24. DEVELOPERS within
projects/sprints is
responsible for PAGE
LAYER, provide
keywords for
upper/bdd layer.

25. Testers are responsiblefor TEST LAYER, which is
Better To BE BDD, to create business value to each
stakeholder
Analystsare responsible for detailedand
correct acceptance-criteria thatBDD Scripts
base on

26. ALSO %50 of THE MAINTENANCEEFFORTCOMES
FROM CHANgING OBJECT IDENTIFIERS
LET’S MAKEIT EVERYONE’SJOB IN THE TEAM
NOT ONLY SOMEONe

27. Спасибо
berk.dulger@keytorc.com
berkdulger.github.io
www.linkedin.com/in/berkdulger