Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

QA Fest 2017. Игорь Любин. Примеры ускорения автотестов

216 views

Published on

Нас часто спрашивают: Сколько нужно времени на тестирование? Правильный ответ должен быть таким - минуты. Именно минуты, а не десятки минут, часов или дней… Такого результата от нас ждёт бизнес и разработчики. В своём докладе я хочу рассказать о практических приёмах, как я ускорял тесты.

Published in: Education
  • Be the first to comment

  • Be the first to like this

QA Fest 2017. Игорь Любин. Примеры ускорения автотестов

  1. 1. Automation tests accelerating examples Indonesia PhilippinesMalaysia ThailandSingapore Vietnam Igor Lyubin 1 Automation tests accelerating examples Indonesia PhilippinesMalaysia ThailandSingapore Vietnam Igor Lyubin 1
  2. 2. About me 2 # Igor Lyubin # PhD in Physics # In QA since 2007 # Senior QA in Lazada # http://blog.auto-testing.ru/post-550
  3. 3. Lazada 3 # Founded in 2012 # Biggest e-commerce in Asia # 6 countries # 650M users # 40M products # 1,2M products per day
  4. 4. Lazada 4 # 4 TechHub: VN, SG, TH, RU # 600 IT engineers # GoLang, PHP, Python # Docker, Kubernetes # https://www.slideshare.net/VLDCORP/kubernetes-76514226
  5. 5. My project 5 # Order API # 1,5K tests # Releases every day # 2 showroom -> 6 staging -> live # Testing should be fast < 10 minutes
  6. 6. Tests accelerating 6 # Precondition # Prepare your stuff before # Speed vs Coverage # Get more resources # Make slow fast # UI Testing # Hand Helper # https://www.slideshare.net/alimenkou/how-to-make-your-functional-tests-really-quick # https://www.slideshare.net/VLDCORP/sqa15 - Badoo
  7. 7. Precondition 7
  8. 8. Set the goal! 8 # Stay in plank while regression testing go
  9. 9. Fix flaky tests 9 # https://wiki.jenkins.io/display/JENKINS/Flaky+Test+Handler+Plugin # https://www.joecolantonio.com/2015/12/21/top-10-reasons-for-flaky-automated-tests py.test –-count=100 tests/
  10. 10. Prepare your stuff before 10
  11. 11. ? 11 def test_ok_filter_return_orders_by_ra_number(self): return_order = db.get_random_return_orders() ra_number = return_order['ra_number'] ... def test_ok_filter_return_orders_with_return_date(self): return_order = db.get_random_return_orders() date = return_order['return_date'] ...
  12. 12. Cache data via session scope 12 def test_ok_filter_return_orders_by_ra_number(self): return_order = db.get_random_return_orders() ra_number = return_order['ra_number'] ... @pytest.fixture(scope='session') def return_orders(): return db.get_return_orders(100) @pytest.fixture def random_return_order(return_orders): return random.choice(return_orders) def test_ok_filter_return_orders_by_ra_number(self, random_return_order): ra_number = random_return_order['ra_number'] ...
  13. 13. Define fixtures for readonly tests 13 # For readonly tests: statistics, filters, get @pytest.fixture(scope='session') def session_order(): return create_order() # For action tests: return order, cancel order @pytest.fixture def order(): return create_order()
  14. 14. QA API 14
  15. 15. Coverage vs Speed 15
  16. 16. ? @pytest.mark.parametrize('status', ['active', 'inactive', 'deleted']) def test_ok_get_payment_method_by_status(self, status): response = api.get_payment_method_by_status(status) ... 16
  17. 17. List to random @pytest.mark.parametrize('status', ['active', 'inactive', 'deleted']) def test_ok_get_payment_method_by_status(self, status): response = api.get_payment_method_by_status(status) ... statuses = ['active', 'inactive', 'deleted'] def test_ok_get_payment_method_by_status(self): status = random.choice(statuses) response = api.get_payment_method_by_status(status) ... 17
  18. 18. Split tests on groups 18 # 1 priority – killer feature # 2 priority - good coverage # 3 priority – work and ok
  19. 19. ? 19 class TestGetCancelOrderQuestions: def test_ok_get_questions(self): api.get_cancel_order_questions() class TestCreateCancelOrderQuestion: def test_ok_create_question(self, request): question = api.set_cancel_order_question(request) questions = api.get_cancel_order_questions() assert_that(questions[-1], equal_to(request)) ...
  20. 20. Combine getter + setter as feature 20 class TestGetCancelOrderQuestions: def test_ok_list_of_questions(self): api.get_cancel_order_questions() class TestCreateCancelOrderQuestion: def test_ok_create_question(self, request): question = api.set_cancel_order_question(request) questions = api.get_cancel_order_questions() assert_that(questions[-1], equal_to(request)) ...
  21. 21. ? 21 def test_ok_create_return_order(self, request): response = api.create_return_order(request) data = api.get_return_order_by_id(response.id_return_order) assert_that(data, equal_to(request)) def test_400_create_return_order_twice(self, request): api.create_return_order(request) response = api.create_return_order(request) assert_that(response.status_code, equal_to(400))
  22. 22. Combine ok + twice 22 def test_ok_create_return_order(self, request): response = api.create_return_order(request) data = api.get_return_order_by_id(response.id_return_order) assert_that(data, equal_to(request)) def test_400_create_return_order_twice(self, request): api.create_return_order(request) response = api.create_return_order(request) assert_that(response.status_code, equal_to(400))
  23. 23. Get more resources 23
  24. 24. Parallelism by classes 24 # Nunit: [assembly: Parallelizable(ParallelScope.Fixtures)] # pytest-xdist 1.20: # https://github.com/pytest-dev/pytest-xdist/pull/191 py.test -v --dist=loadscope
  25. 25. Parallelization on CI 25
  26. 26. Make slow fast 26
  27. 27. Mock 27 @pytest.fixture(scope='session') def hello_pay_mock(): create_hello_pay_mock() use_hello_pay_mock() def test_ok_make_payment_with_hello_pay(self, hello_pay_mock): ... # http://www.mbtest.org - mountebank # https://github.com/aholyoke/mountebank-python
  28. 28. Response time SLA 28 @sla(300) def order_items_by_id(json): return post('order-items/by-id', json) @sla(500) def order_statistics_by_ip(json): return post('order-statistics/by-ip', json) # utils/measure_time.py def sla(milliseconds): def func_decorator(func): def func_wrapper(*args): r = measure_time(lambda: func(*args)) assert_that(r.time, less_than(milliseconds) return r.result return func_wrapper return func_decorator
  29. 29. DB queries vs API requests 29 def test_ok_create_cancel_order_reason(self, request): response = api.create_cancel_order_reason(request) data = db.get_cancel_order_reason_by_id(response.id) assert_that(response, equal_to(data)) def test_ok_create_cancel_order_reason(self, request): response = api.create_cancel_order_reason(request) data = api.get_cancel_order_reason_by_id(response.id) assert_that(response, equal_to(data))
  30. 30. UI Testing 30
  31. 31. Only functional tests 31 # Create # Read # Update # Delete
  32. 32. Direct navigation 32 def test_ok_create_cancel_order_reason(self, reason_data): app.create_cancel_order_reason(reason_data) reason = app.get_cancel_order_reason_by_name(reason_data.name) assert_that(reason, equal_to(reason_data)) def create_create_cancel_order_reason(reason_data): open_cancel_order_reason_page() ...
  33. 33. Direct asserts in api 33 def test_ok_create_cancel_order_reason(self, reason_data): app.create_cancel_order_reason(reason_data) reason = app.get_cancel_order_reason_by_name(reason_data.name) assert_that(reason, equal_to(reason_data)) def test_ok_create_cancel_order_reason(self, reason_data): app.create_cancel_order_reason(reason_data) reason = api.get_cancel_order_reason_by_name(reason_data.name) assert_that(reason, equal_to(reason_data))
  34. 34. ? 34 driver.find_element_by_xpath('//*[contains(@class, "header__links-bar")]')
  35. 35. ? 35 driver.find_element_by_xpath('//*[contains(@class, "header__links-bar")]') driver.find_element_by_xpath('//div[contains(@class, "header__links-bar")]')
  36. 36. Do not use x-path locators 36 driver.find_element_by_xpath('//*[contains(@class, "header__links-bar")]') driver.find_element_by_xpath('//div[contains(@class, "header__links-bar")]') driver.find_element_by_id('links-bar-id')
  37. 37. Hand Helper 37
  38. 38. HandHelper 38# https://www.youtube.com/watch?v=Pbe5fK8ksEI
  39. 39. igor.lyubin@lazada.com https://goo.gl/p17WQF 39

×