Successfully reported this slideshow.
Your SlideShare is downloading. ×

QA Fest 2017. Игорь Любин. Примеры ускорения автотестов

QA Fest 2017. Игорь Любин. Примеры ускорения автотестов

Нас часто спрашивают: Сколько нужно времени на тестирование? Правильный ответ должен быть таким - минуты. Именно минуты, а не десятки минут, часов или дней… Такого результата от нас ждёт бизнес и разработчики. В своём докладе я хочу рассказать о практических приёмах, как я ускорял тесты.

Нас часто спрашивают: Сколько нужно времени на тестирование? Правильный ответ должен быть таким - минуты. Именно минуты, а не десятки минут, часов или дней… Такого результата от нас ждёт бизнес и разработчики. В своём докладе я хочу рассказать о практических приёмах, как я ускорял тесты.

More Related Content

More from QAFest

Related Audiobooks

Free with a 30 day trial from Scribd

See all

QA Fest 2017. Игорь Любин. Примеры ускорения автотестов

  1. 1. Automation tests accelerating examples Indonesia PhilippinesMalaysia ThailandSingapore Vietnam Igor Lyubin 1 Automation tests accelerating examples Indonesia PhilippinesMalaysia ThailandSingapore Vietnam Igor Lyubin 1
  2. 2. About me 2 # Igor Lyubin # PhD in Physics # In QA since 2007 # Senior QA in Lazada # http://blog.auto-testing.ru/post-550
  3. 3. Lazada 3 # Founded in 2012 # Biggest e-commerce in Asia # 6 countries # 650M users # 40M products # 1,2M products per day
  4. 4. Lazada 4 # 4 TechHub: VN, SG, TH, RU # 600 IT engineers # GoLang, PHP, Python # Docker, Kubernetes # https://www.slideshare.net/VLDCORP/kubernetes-76514226
  5. 5. My project 5 # Order API # 1,5K tests # Releases every day # 2 showroom -> 6 staging -> live # Testing should be fast < 10 minutes
  6. 6. Tests accelerating 6 # Precondition # Prepare your stuff before # Speed vs Coverage # Get more resources # Make slow fast # UI Testing # Hand Helper # https://www.slideshare.net/alimenkou/how-to-make-your-functional-tests-really-quick # https://www.slideshare.net/VLDCORP/sqa15 - Badoo
  7. 7. Precondition 7
  8. 8. Set the goal! 8 # Stay in plank while regression testing go
  9. 9. Fix flaky tests 9 # https://wiki.jenkins.io/display/JENKINS/Flaky+Test+Handler+Plugin # https://www.joecolantonio.com/2015/12/21/top-10-reasons-for-flaky-automated-tests py.test –-count=100 tests/
  10. 10. Prepare your stuff before 10
  11. 11. ? 11 def test_ok_filter_return_orders_by_ra_number(self): return_order = db.get_random_return_orders() ra_number = return_order['ra_number'] ... def test_ok_filter_return_orders_with_return_date(self): return_order = db.get_random_return_orders() date = return_order['return_date'] ...
  12. 12. Cache data via session scope 12 def test_ok_filter_return_orders_by_ra_number(self): return_order = db.get_random_return_orders() ra_number = return_order['ra_number'] ... @pytest.fixture(scope='session') def return_orders(): return db.get_return_orders(100) @pytest.fixture def random_return_order(return_orders): return random.choice(return_orders) def test_ok_filter_return_orders_by_ra_number(self, random_return_order): ra_number = random_return_order['ra_number'] ...
  13. 13. Define fixtures for readonly tests 13 # For readonly tests: statistics, filters, get @pytest.fixture(scope='session') def session_order(): return create_order() # For action tests: return order, cancel order @pytest.fixture def order(): return create_order()
  14. 14. QA API 14
  15. 15. Coverage vs Speed 15
  16. 16. ? @pytest.mark.parametrize('status', ['active', 'inactive', 'deleted']) def test_ok_get_payment_method_by_status(self, status): response = api.get_payment_method_by_status(status) ... 16
  17. 17. List to random @pytest.mark.parametrize('status', ['active', 'inactive', 'deleted']) def test_ok_get_payment_method_by_status(self, status): response = api.get_payment_method_by_status(status) ... statuses = ['active', 'inactive', 'deleted'] def test_ok_get_payment_method_by_status(self): status = random.choice(statuses) response = api.get_payment_method_by_status(status) ... 17
  18. 18. Split tests on groups 18 # 1 priority – killer feature # 2 priority - good coverage # 3 priority – work and ok
  19. 19. ? 19 class TestGetCancelOrderQuestions: def test_ok_get_questions(self): api.get_cancel_order_questions() class TestCreateCancelOrderQuestion: def test_ok_create_question(self, request): question = api.set_cancel_order_question(request) questions = api.get_cancel_order_questions() assert_that(questions[-1], equal_to(request)) ...
  20. 20. Combine getter + setter as feature 20 class TestGetCancelOrderQuestions: def test_ok_list_of_questions(self): api.get_cancel_order_questions() class TestCreateCancelOrderQuestion: def test_ok_create_question(self, request): question = api.set_cancel_order_question(request) questions = api.get_cancel_order_questions() assert_that(questions[-1], equal_to(request)) ...
  21. 21. ? 21 def test_ok_create_return_order(self, request): response = api.create_return_order(request) data = api.get_return_order_by_id(response.id_return_order) assert_that(data, equal_to(request)) def test_400_create_return_order_twice(self, request): api.create_return_order(request) response = api.create_return_order(request) assert_that(response.status_code, equal_to(400))
  22. 22. Combine ok + twice 22 def test_ok_create_return_order(self, request): response = api.create_return_order(request) data = api.get_return_order_by_id(response.id_return_order) assert_that(data, equal_to(request)) def test_400_create_return_order_twice(self, request): api.create_return_order(request) response = api.create_return_order(request) assert_that(response.status_code, equal_to(400))
  23. 23. Get more resources 23
  24. 24. Parallelism by classes 24 # Nunit: [assembly: Parallelizable(ParallelScope.Fixtures)] # pytest-xdist 1.20: # https://github.com/pytest-dev/pytest-xdist/pull/191 py.test -v --dist=loadscope
  25. 25. Parallelization on CI 25
  26. 26. Make slow fast 26
  27. 27. Mock 27 @pytest.fixture(scope='session') def hello_pay_mock(): create_hello_pay_mock() use_hello_pay_mock() def test_ok_make_payment_with_hello_pay(self, hello_pay_mock): ... # http://www.mbtest.org - mountebank # https://github.com/aholyoke/mountebank-python
  28. 28. Response time SLA 28 @sla(300) def order_items_by_id(json): return post('order-items/by-id', json) @sla(500) def order_statistics_by_ip(json): return post('order-statistics/by-ip', json) # utils/measure_time.py def sla(milliseconds): def func_decorator(func): def func_wrapper(*args): r = measure_time(lambda: func(*args)) assert_that(r.time, less_than(milliseconds) return r.result return func_wrapper return func_decorator
  29. 29. DB queries vs API requests 29 def test_ok_create_cancel_order_reason(self, request): response = api.create_cancel_order_reason(request) data = db.get_cancel_order_reason_by_id(response.id) assert_that(response, equal_to(data)) def test_ok_create_cancel_order_reason(self, request): response = api.create_cancel_order_reason(request) data = api.get_cancel_order_reason_by_id(response.id) assert_that(response, equal_to(data))
  30. 30. UI Testing 30
  31. 31. Only functional tests 31 # Create # Read # Update # Delete
  32. 32. Direct navigation 32 def test_ok_create_cancel_order_reason(self, reason_data): app.create_cancel_order_reason(reason_data) reason = app.get_cancel_order_reason_by_name(reason_data.name) assert_that(reason, equal_to(reason_data)) def create_create_cancel_order_reason(reason_data): open_cancel_order_reason_page() ...
  33. 33. Direct asserts in api 33 def test_ok_create_cancel_order_reason(self, reason_data): app.create_cancel_order_reason(reason_data) reason = app.get_cancel_order_reason_by_name(reason_data.name) assert_that(reason, equal_to(reason_data)) def test_ok_create_cancel_order_reason(self, reason_data): app.create_cancel_order_reason(reason_data) reason = api.get_cancel_order_reason_by_name(reason_data.name) assert_that(reason, equal_to(reason_data))
  34. 34. ? 34 driver.find_element_by_xpath('//*[contains(@class, "header__links-bar")]')
  35. 35. ? 35 driver.find_element_by_xpath('//*[contains(@class, "header__links-bar")]') driver.find_element_by_xpath('//div[contains(@class, "header__links-bar")]')
  36. 36. Do not use x-path locators 36 driver.find_element_by_xpath('//*[contains(@class, "header__links-bar")]') driver.find_element_by_xpath('//div[contains(@class, "header__links-bar")]') driver.find_element_by_id('links-bar-id')
  37. 37. Hand Helper 37
  38. 38. HandHelper 38# https://www.youtube.com/watch?v=Pbe5fK8ksEI
  39. 39. igor.lyubin@lazada.com https://goo.gl/p17WQF 39

×