This document discusses the concept of "shifting left" mobile application testing. Shifting left means starting test processes as early in the software development lifecycle as possible to provide early feedback. This helps reduce the cost of fixing bugs. The document outlines challenges for shifting left testing in mobile, such as tool and infrastructure maturity. It then provides recommendations for implementing a shift left approach, including measuring unit test coverage, setting up continuous integration, replacing some UI tests with unit tests, implementing partial integration testing, and rearchitecting end-to-end tests to be more modular and parallelizable.

1. Shift left Mobile ApplicationTesting
Satyajit Malugu

2. Urbanspoon

3. Specification Design Development Maintenance

4. Domain Finder

5. Specification Design Development Maintenance

6. Still had lot of problems
• Testing unable to fit into agile dev cycle
• UI automation unable to keep up with constant changes
• A very low coverage after weeks of playing catchup

7. Test pyramid reversed
http://adventuresinqa.com/2015/10/19/mobile-test-pyramid/#more-3194

8. Godaddy Investor

9. Specification Design Development Maintenance
What is “Shift left”
• Start test processes as early in SDLC as possible
• Continue to provide early feedback

10. Why Shift left
• Cost of fixing bugs exponential across SDLC
Specification Design Development Maintainance

11. Reclaiming the pyramid

12. Enabling Agile
• Agile is all about adaptability to change
• Enabled through fast feedback loops
• Test early and often

13. Challenges for shift left in mobile
• Tool support not mature
• Device fragmentation
• Culture of native development teams

14. Tool support is getting better
• Companies dedicated to mobile testing
• Real device clouds
• Network variations
• Sensor simulation

15. Android tool support
• Unit tests – Junit
• Code coverage – Jacoco
• Continuous integration – Jenkins
• Functional tests –Roboelectric
• E2E testing – Appium, Espresso
• Continuous deployment – Alpha, beta releases in play store
• Device emulation – adb, genymotion
• Performance testing – rendering profiler, traceview

16. iOS tool support
• Unit tests – XCTest,
• Code coverage – Xcode server with bots
• Continuous integration – Jenkins, Xcode server
• Functional tests – EarlGrey
• E2E testing – Appium, KIF, Calabash, XCUITest
• Continuous deployment –Testflight alpha, beta
• Device emulation – Xcode
• Performance testing – XCTest, instruments

17. Step I – Measure unit test code coverage
• What you can’t measure, can’t be improved
• Get the team to agree on a good coverage %
• Send coverage report everyday

18. Slack bots reporting daily coverage

19. Step II– Create a CI infrastructure
• For running unit, functional, UI tests
• Build various environments
• Code coverage reports

20. Pull Request triggers unit and UI tests

21. Jenkins UI test run sample

22. Step III– Replace some UI tests with unit tests
• As a principle prefer lower level test over a higher one
• Don’t repeat a test - DRY

23. Step IV- Partial integration testing
• Create test hooks for navigating into app directly
• Deep links, shortcuts
• Level – 1 verifications
• Not everything need to be verified through UI
Setup Action Verification

24. StepV- Re-architect E2E tests
• Should be atomic not interdependent
• Enables running in parallel
• Design tests so that they can be triggered in multiple ways
• Different language
• Different screensize
• Different OS
• Should be minimal
• Run them continuously
• On a cloud provider

25. Specification Design Development Maintenance
1. Continuous testing
2. Crash Monitoring
3. Performance Monitoring
4. App store reviews monitoring
5. Engagement monitoring
1. Functional tests
2. UITests
3. Unit tests
1. Testability hooks
2. Accessibility ids
3. API level testing
4. Unit test
coverage%
1. Test infrastructure
2. CI/CD setup
3. Device lab
Summary

26. On the horizon – React Native
• Bringing web like deployment capabilities to native world
• Learn once, write anywhere
• Very high importance to testing

27. Questions