Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Mobile automatic testing: done by Readdle
Sergey Hizhnyak
About Readdle
Readdle creates popular productivity apps such Spark
email, Documents, Scanner Pro and PDF Expert.
The team ...
About myself
• Started to work in QA more than 10 years ago
• Worked as automation QA, performance QA,
white box tester
• ...
Setup
• WebDriver
• Appium
• Java
• Cucumber JVM
• Jenkins
Why?
• Why not Allure?
• Why Cucumber and BDD?
• Why managers read our reports?
Because
• Single development language
• Shared API and utilities
• Easy to add new platforms (Android, Windows)
• Easy to ...
Testing process
• Tests in TestRail with unique ID
• Link Cucumber scenario
• Populate TestRail with execution status
• Vi...
Parallel execution: issue
• Execution time exceeded 4 hours
• Cucumber is not designed to run in
parallel 
Parallel execution: solution
• Why we do not use Grid?
• Why we do not use some open-source
solution?
• Web service that b...
Solution bonus
• Online execution tracking
• Current queue visualization
• Running scenario status
• Execution statuses an...
Tests maintenance: issue
• Tests fail a lot
• 1000+ tests report analysis
• Wasted time!
Tests maintenance: solution
• Sorting of failed tests
• Development
• Stable: passed 5 times in a row
• Verification
• Not...
• 70% of test cases are automated
• Manual QA are “developing” Cucumber
scenarios
• Manual testing of all non-automated
fu...
Fail #1: Simulators
• Memory management and data protection
• Why do we run on simulator?
• Conclusion: test manually
Fail #2: TestRail<->Cucumber mess
• Test was moved/removed
• Link with wrong ID
• Copy-Paste
Fixing the mess
• Compare Cucumber with TestRail
• Multiple IDs
• Incorrect names
• Non-existing TestRail tests
• Etc.
Is it available somewhere?
• All code is open source (read-only)
• It is not very clean, but you could get some
ideas
• ht...
Conclusion
• Develop your own solution
• Automate surroundings
• Check manually
• Involve everyone in developing your auto...
Q&A
Upcoming SlideShare
Loading in …5
×

QA Fest 2017. Сергей Хижняк. Автоматизация тестирования мобильных приложений: опыт продуктовой компании

175 views

Published on

Поговорим о проблемах автоматизации для iOS и Web, о том как синхронизировать работу manual и automation QA, о проблемах которые возникают когда люди практически без опыта программирования начинают писать автоматические тесты и о том как мы эти проблемы решили.
Затронем тему распараллеливания выполнения тестов. Поговорим о том, как выглядит процесс тестирования продукта, в котором 70-80% функционала автоматизированно.
Цель доклада показать как можно построить эффективный процесс автоматизации и как manual QA могут помочь в этом.Хоть мы будем говорить о WebDriver и Appium, в целом доклад на них не фокусируется (так же не требуется знание какого-то конкретного языка программирования).

Published in: Education
  • Be the first to comment

  • Be the first to like this

QA Fest 2017. Сергей Хижняк. Автоматизация тестирования мобильных приложений: опыт продуктовой компании

  1. 1. Mobile automatic testing: done by Readdle Sergey Hizhnyak
  2. 2. About Readdle Readdle creates popular productivity apps such Spark email, Documents, Scanner Pro and PDF Expert. The team is 110 people and we managed to build a sustainable business, never taking VC money or external funding. We are the pioneers of iOS App Store and one of the first companies to create file management and scanning apps. Available on the App Store from the day one, our apps have been downloaded more than 40 million times.
  3. 3. About myself • Started to work in QA more than 10 years ago • Worked as automation QA, performance QA, white box tester • Currently QA manager for Fluix department in Readdle
  4. 4. Setup • WebDriver • Appium • Java • Cucumber JVM • Jenkins
  5. 5. Why? • Why not Allure? • Why Cucumber and BDD? • Why managers read our reports?
  6. 6. Because • Single development language • Shared API and utilities • Easy to add new platforms (Android, Windows) • Easy to adopt by non-programmers
  7. 7. Testing process • Tests in TestRail with unique ID • Link Cucumber scenario • Populate TestRail with execution status • Visible coverage and testing progress • We do not have a concept of passed/failed for the whole automation
  8. 8. Parallel execution: issue • Execution time exceeded 4 hours • Cucumber is not designed to run in parallel 
  9. 9. Parallel execution: solution • Why we do not use Grid? • Why we do not use some open-source solution? • Web service that builds queue from Cucumber scenarios
  10. 10. Solution bonus • Online execution tracking • Current queue visualization • Running scenario status • Execution statuses and statistics
  11. 11. Tests maintenance: issue • Tests fail a lot • 1000+ tests report analysis • Wasted time!
  12. 12. Tests maintenance: solution • Sorting of failed tests • Development • Stable: passed 5 times in a row • Verification • Note: this is purely for our convenience
  13. 13. • 70% of test cases are automated • Manual QA are “developing” Cucumber scenarios • Manual testing of all non-automated functionality • Manual retest for all failed/unstable automation • 6-8 months of stable releases • And we failed  Our “perfect” testing process
  14. 14. Fail #1: Simulators • Memory management and data protection • Why do we run on simulator? • Conclusion: test manually
  15. 15. Fail #2: TestRail<->Cucumber mess • Test was moved/removed • Link with wrong ID • Copy-Paste
  16. 16. Fixing the mess • Compare Cucumber with TestRail • Multiple IDs • Incorrect names • Non-existing TestRail tests • Etc.
  17. 17. Is it available somewhere? • All code is open source (read-only) • It is not very clean, but you could get some ideas • https://github.com/k-g-aleksandrov/cucumber- queue-runner
  18. 18. Conclusion • Develop your own solution • Automate surroundings • Check manually • Involve everyone in developing your automation • Have fun!
  19. 19. Q&A

×