В кратком докладе рассматривается пример "неудачной" автоматизации, которая приводит к демотивации команды, нестабильным тестам, бессмысленным затратам времени и неудовлетворённому заказчику. Стек автоматизации: API (Exchange, REST, Azure), DB, WindowsPowerShell, TFS, MsTest.

1. Gop-stop automated testing
Tsimafei Avilin

2. Meaning…
Gop-stop (slang) – robbery; an attack aimed at theft of
property of a victim committed with the use of violence,
or with the threat of violence…
Translation: гоп-стоп - грабёж; нападение с целью хищения имущества потерпевшего,
совершённое с применением насилия, либо с угрозой применения насилия

3. What's stealing?
• Time
• Money
• Mental energy (so called - demotivation)
• People
• Projects
• Your cases …. ?
SPRING COMAQA CONFERENCE 2017

4. Why does it happen…sometimes?
- Inexperienced managers
- Inexperienced workers
- Few controlled circumstances
SPRING COMAQA CONFERENCE 2017

5. How can identify that?
Usual phrases sound like:
- “We have to see 100% parity and coverage!”
- “We have to automate 100% test cases!”
- “You promised to complete this test set last week!”
- “Why our tests are still failing?”
- “You promised to complete that at 8 P.M.!”
- “Why this test case is still failing???”
- “We are doing code cleaning…”
SPRING COMAQA CONFERENCE 2017

6. Stop this demagogy,
give me the real example!!!
SPRING COMAQA CONFERENCE 2017

7. Framework issues…
SPRING COMAQA CONFERENCE 2017
Test automation framework side:
- too complicated framework with many spikes -> hard
maintainability,
- automated test case is longer then two listings;
- no stable test cases set -> CI builds time to time fails;
- extensive programming -> do as many code lines as possible
w/out encapsulation;
- test data is given by different "providers": hard-coded, as files in
a project, from api provider (e.g.TFS), etc.

8. Framework issues…
SPRING COMAQA CONFERENCE 2017
Real-life example:
- developed framework with complicated architecture;
- about 15 bad-designed test cases (each test case is about 150-
200 lines of code) and all these tests got failed both in CI and
even locally have been developed;
- test data is given by different "providers": hard-coded, as files in
a project, from api provider (e.g.TFS)

9. CI results for such framework…
SPRING COMAQA CONFERENCE 2017
0
50
100
150
200
250
300
Numberoftestcases
Date
Failed test cases Passed test cases
WTF???

10. Process issues…
SPRING COMAQA CONFERENCE 2017
Testing and development process side:
- it takes much time newcomer to start working on a project;
- much time and effort is going to fix some unique situations
instead of working on main functionality/important bugs;
- switching between the projects each few days;
- complicated relationships between the customer and the vendor
-- solve problem - not people!
- the customer's PMs cannot make a final decision and put it on
the team shoulders;

11. Process issues…
SPRING COMAQA CONFERENCE 2017

12. Potential solutions…
SPRING COMAQA CONFERENCE 2017
Common recommendations:
-- dive deeply into designed architecture;
- focus on specific pieces of an application instead of long E2E
testing;
- try to not automate 100% if the tests set will be unstable;
Audited framework recommendations:
- define strategy of preparing test data (in the framework all
business entities are created in the test case directly, which
takes sometimes about 50 lines of code or more…);
- stop developing new automated test cases and encapsulate
members to improve and simplify test cases design;
- improve quality of functional test cases;

13. SPRING COMAQA CONFERENCE 2017