This document discusses high performance functional testing based on Selenium. It begins with an introduction of the speaker and outlines a testing pyramid with unit, integration and functional/UI tests. It then demonstrates how to automate functional tests for web applications using Selenium and discusses techniques for decreasing test execution time, including using Selenium Grid, PhantomJS, HTTP requests instead of Selenium, and parallel test execution. The document emphasizes that faster tests allow for faster development and provides approaches and tools to reduce test bottlenecks and optimize test execution time.

1. High performance
Functional tests
based on Selenium
Andrii Soldatenko
27-28 February 2015
@a_soldatenko

2. About me:
• Lead QA Automation Engineer at
• Backend Python Developer at
• Speaker at PyCon Ukraine 2014
• Speaker at PyCon Belarus 2015
• @a_soldatenko

3. "Tests are
the Programmer’s
stone,
transmuting fear
into boredom."

4. We don’t have time
for tests

5. Test Automation Pyramid
from
http://martinfowler.com/bliki/TestPyramid.html

6. Unit tests

7. Integration tests

8. Functional or UI tests

9. How to automate
Functional tests for Web?

10. Demo

11. What is
Test Execution Time?
• Faster Tests Mean Faster Development
• Slow Tests Don’t Get Run as Often

12. How to calculate test
execution time

13. How to decrease test
execution time?
“Man is a tool-using animal.
Without tools he is nothing, with
tools he is all.“ - Thomas Carlyle

14. xUnit in Python
parallel
execution
plugins
native
test
coverage
unittest NO NO NO
YES YES YES
YES YES NO

15. Parallel
test execution

16. Demo

17. Use Selenium Grid
Node
Node
Node
Hub

18. How to setup
Selenium Grid locally

19. Add Selenium Grid
support to tests

20. Selenium Grid +
Concurrent test execution
Node
Node
Node
Hub
OR

21. Demo

22. Selenium Grid in Cloud

23. Example

24. Use Selenium with
PhantomJS
Ghost
Driver
https://github.com/ariya/phantomjs/

25. What is GhostDriver?
https://github.com/detro/ghostdriver

26. PhantomJS:
advantages

27. PhantomJS:
disadvantages

28. Demo

29. Using http Requests
instead of Selenium

30. Demo

31. Approach
Execution
Time
%
num
processes
Selenium WebDriver 22.76 sec 100% 1
Selenium + py.test 9.24 sec 40% 4
Selenium + PhantomJS 15.55 sec 68% 1
Selenium + PhantomJS 9.98 sec 43% 4
Selenium + PhantomJS +
Requests
10.9 sec 47% 1
Selenium + PhantomJS +
Requests
6.1 sec 26% 4

32. “Premature optimization is the
root of all evil” - Donald Knuth

33. How to find bottlenecks
in functional tests?

34. Final Thoughts
• Don’t test constants
• Avoid double-testing functionality
• Try to test behavior rather than implementation
• Use Continuous Integration
• Slow tests kill productivity
• Use explicit rather than implicit waits and wait pattern

35. Questions
?

36. Thank You
a_soldatenko@wargaming.net
@a_soldatenko
http://asoldatenko.com

37. We are hiring
a_soldatenko@wargaming.net