Recommended
Recommended
More Related Content
More from QAFest
More from QAFest (20)
Recently uploaded
Recently uploaded (20)
QA Fest 2017. Сергей Хижняк. Автоматизация тестирования мобильных приложений: опыт продуктовой компании
- 1. Mobile automatic testing: done by Readdle Sergey Hizhnyak
- 2. About Readdle Readdle creates popular productivity apps such Spark email, Documents, Scanner Pro and PDF Expert. The team is 110 people and we managed to build a sustainable business, never taking VC money or external funding. We are the pioneers of iOS App Store and one of the first companies to create file management and scanning apps. Available on the App Store from the day one, our apps have been downloaded more than 40 million times.
- 3. About myself • Started to work in QA more than 10 years ago • Worked as automation QA, performance QA, white box tester • Currently QA manager for Fluix department in Readdle
- 4. Setup • WebDriver • Appium • Java • Cucumber JVM • Jenkins
- 5. Why? • Why not Allure? • Why Cucumber and BDD? • Why managers read our reports?
- 6. Because • Single development language • Shared API and utilities • Easy to add new platforms (Android, Windows) • Easy to adopt by non-programmers
- 7. Testing process • Tests in TestRail with unique ID • Link Cucumber scenario • Populate TestRail with execution status • Visible coverage and testing progress • We do not have a concept of passed/failed for the whole automation
- 8. Parallel execution: issue • Execution time exceeded 4 hours • Cucumber is not designed to run in parallel
- 9. Parallel execution: solution • Why we do not use Grid? • Why we do not use some open-source solution? • Web service that builds queue from Cucumber scenarios
- 10. Solution bonus • Online execution tracking • Current queue visualization • Running scenario status • Execution statuses and statistics
- 11. Tests maintenance: issue • Tests fail a lot • 1000+ tests report analysis • Wasted time!
- 12. Tests maintenance: solution • Sorting of failed tests • Development • Stable: passed 5 times in a row • Verification • Note: this is purely for our convenience
- 13. • 70% of test cases are automated • Manual QA are “developing” Cucumber scenarios • Manual testing of all non-automated functionality • Manual retest for all failed/unstable automation • 6-8 months of stable releases • And we failed Our “perfect” testing process
- 14. Fail #1: Simulators • Memory management and data protection • Why do we run on simulator? • Conclusion: test manually
- 15. Fail #2: TestRail<->Cucumber mess • Test was moved/removed • Link with wrong ID • Copy-Paste
- 16. Fixing the mess • Compare Cucumber with TestRail • Multiple IDs • Incorrect names • Non-existing TestRail tests • Etc.
- 17. Is it available somewhere? • All code is open source (read-only) • It is not very clean, but you could get some ideas • https://github.com/k-g-aleksandrov/cucumber- queue-runner
- 18. Conclusion • Develop your own solution • Automate surroundings • Check manually • Involve everyone in developing your automation • Have fun!
- 19. Q&A
Editor's Notes
- 1
- 2
- Manager joke
- 4
- 5
- 6
- 7
- 9
- 10
- 11
- 14
- 15
- 18