Environment Manager allows QA teams to easily and quickly execute and report on TestComplete tests across more than 500 test environments. Right from inside TestComplete, without setup or configuration, you get an access to a real device lab for executing GUI tests across a range of browsers, operating systems, and resolutions.

1. Test Design + Environment Management:
Scaling UI Automation for Agile

2. 2
Quick Intro
Nikhil Kaul

3. 3
Broad automation & test management tools
GUI Layer
API Layer
Developer QA Engineer Manual Tester
TestCompleteTestLeft
TestServer Ready! API
Desktop Mobile Packaged AppsWeb
CrossBroswerTesting
Selenium/
Appium
Target Apps
QAComplete
Test
Management
for risk-based
analysis
Environment
Management

4. 4
On our plate today
2
Flaky tests & ways to overcome these problems
3
How to overcome challenges due to frameworks
1
The typical automation process
4
Test Environment management practices

5. 5
The process looks something like
Create tests
Run tests
Report & Metrics
Find bugs
1
2
3
4

6. 6
But as your tests grow you realize
Random Is it a bug?? Start ignoring Limited collaboration

7. 7
Shift the focus on quality rather than quantity
Quality
Quantity

8. 8
Common reasons for flakiness
Expensive to maintainNot Stable
Flaky Tests Flaky Framework

9. 9
And to do that set up a process that
1
2
3
Number
Time

10. 10
Common reasons for flakiness
Expensive to maintainNot Stable
Flaky Tests Flaky Framework

11. 11
Flaky tests result from
Expensive to maintainNot Stable
Sleeps
Stable identifiers
lacking
Rely on flaky
services
Interdependent
Unreliable tests

12. 12
Flaky tests result from
Expensive to maintainNot Stable
Sleeps
Stable identifiers
lacking
Rely on flaky
services
Interdependent
Unreliable tests

13. 13
Things to look for identifiers
‐ ID
‐ Classes
‐ CSS
‐ Xpath
– Depends on page structure
– Won’t change in future
– Speed
– Exclusive

14. 14
Flaky tests result from
Expensive to maintainNot Stable
Sleeps
Stable identifiers
lacking
Rely on flaky
services
Interdependent
Unreliable tests

15. 15
Hardcoded sleeps or delays
Expensive to maintainNot Stable
Delay (ms)
Sleep (ms)
Smarter wait mechanism
Wait Untill, jQuery.active == 0

16. 16
Flaky tests result from
Expensive to maintainNot Stable
Sleeps
Stable identifiers
lacking
Rely on flaky
services
Interdependent
Unreliable tests

17. 17
Tests need to be independent
Expensive to maintainNot Stable
1
Open the login page
2
Enter username and password
3
4
Multiply two numbers
Check the result
Two Challenges
Not independent
Longer than
necessary

18. 18
Flaky tests result from
Expensive to maintainNot Stable
Sleeps
Stable identifiers
lacking
Rely on flaky
services
Interdependent
Unreliable tests

19. 19
Relying on flaky services
Our App
System2
System3

20. 20
Common reasons for flakiness
Expensive to maintainNot Stable
Flaky Tests Flaky Framework

21. 21
Not much thought into framework
Expensive to maintainNot Stable
Login Forgot Password

22. 22
Instead separate actions and control identifiers
Expensive to maintainNot Stable
Object Class Test
Abstract control identification
Abstract control actions
Test Steps

23. 23
So now my test looks like..
Expensive to maintainNot Stable
Abstract identifiers Abstracting actions Test Step

24. 24
Infact, don't try to do everything on the UI
UI
API
Unit
$$
$

25. 25
Catch bugs earlier and faster
Web
Mobile
Tablet
1
2
3
4

26. 26
You can even reuse these tests across teams
API functional test
API load test
GUI functional test
API security test
Monitoring
Time reduced
by 80%

27. 27
On our plate today
2
Flaky tests & framework
3
How to overcome these challenges
1
The typical automation process
4
Test Environment management practices

28. 28
Now we are good at the first half
Record &
Replay
Script
QA Engineer/ Manual
Tester
OSBrowsers Resolutions

29. 29
Environment setup often needs manual efforts
VMs
Speed
Redundancy
Cost
Discrepancies
Advanced testing features
missing
Device LabsLocal Machines
1
2
3
UI Testing
Manual Efforts Necessitated
5
4

30. 30
For effective environment management
Easy to install
Easy to enhance
Easy to share
Easy to tear down
1
2
3
4

31. 31
Cloud test environments come handy
Example:
800*600,
1680*1050
1920*1440
Example:
IE 9, 10,
Chrome 50, 55
Firefox 49
Operating System Browser Resolution
Example:
Windows 10
Windows 8
Windows 7
Cloud Test Environments

32. 32
So to recap
1
Follow steps to overcome flaky test and flaky framework
2
Start small and build a modular test design
3
Don’t limit yourself to test automation
3
Integrated GUI and API testing is key
Learn more about
Environment Manager

33. Questions?
Learn more about
Environment
Manager