Satyajit Malugu presented on shifting mobile application testing left to earlier phases of the development lifecycle. This approach involves starting testing processes alongside development processes to find and fix bugs earlier. Challenges include immature tooling, device fragmentation, and cultural barriers. Strategies discussed to shift left included measuring unit test coverage, creating a continuous integration infrastructure, replacing some UI tests with lower level unit tests, partial integration testing with test hooks, and rearchitecting end-to-end tests to be more modular and parallelizable. Upcoming tools like React Native may further enable this shift left approach for mobile.

1. W14
Track Session
4/20/2016 2:00 PM
"Shift Left Mobile Application
Testing"
Presented by:
Satyajit Malugu
GoDaddy
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ info@techwell.com ∙ www.techwell.com

2. Satyajit Malugu
GoDaddy
A senior SDET at GoDaddy, Satyajit Malugu is focused on mobile testing. His
work involves automation of native and hybrid apps, and providing test
perspective to his team that recently converted from waterfall to agile. As a
testing leader in a company that is deploying a suite of native apps, Satyajit is
involved with strategizing, executing, and mentoring other testers and
stakeholders on best practices for native mobile testing. Previously, Satyajit
worked at Urbanspoon and gained SDET knowledge at Microsoft. He frequently
presents on mobile topics. Read his blog and follow him on Twitter at @malugu.

3. Shift left Mobile Application Testing
Satyajit Malugu
Urbanspoon

4. Specification Design Development Maintenance
Domain Finder

5. Specification Design Development Maintenance
Still had lot of problems
• Testing unable to fit into agile dev cycle
• UI automation unable to keepup with constantchanges
• A very low coverageafter weeks of playingcatchup

6. Test pyramid reversed
http://adventuresinqa.com/2015/10/19/mobile-test-pyramid/#more-3194
What is shift left
• Testing processesstart alongwith developmentprocesses

7. Godaddy Investor
Specification Design Development Maintenance

8. Why Shift left
• Cost of fixingbugs exponentialacrossSDLC
Specification Design Development Maintainance
Enabling Agile
• Agile is all about adaptabilityto change
• Enabled through fast feedbackloops
• Test earlyand often

9. Challenges for shift left in mobile
• Tool support not mature
• Device fragmentation
• Culture of native development teams
Tool support is getting better
• Companiesdedicated to mobiletesting
• Real device clouds
• Network variations
• Sensor simulation

10. Android tool support
• Unit tests– JUNIT
• Code coverage –Jacaco
• Continuous integration –Jenkins
• Functionaltests – Robotium,roboelectric,Espresso etc
• Continuous deployment –Alpha, betareleases in play store
• Device emulation – adb,genymotion
• Performance testing– rendering profiler, traceview
iOS tool support
• Unit tests–XCTest,
• Code coverage –Xcodeserver with bots
• Continuous integration –Jenkins/Xcodeserver
• Functionaltests – EarlGrey
• Continuous deployment –Testflight alpha,beta
• Device emulation –Xcode
• Performance testing–XCTest,instruments

11. Reclaiming the pyramid
Step I – Measure unit test code coverage
• What youcan’t measure, can’t be improved
• Get the team to agree on a goodcoverage %
• Send coveragereport everyday

12. Slack bots reporting daily coverage
Step II– Create a CI infrastructure
• For runningunit, functional,UI tests
• Build various environments
• Code coveragereports

13. Pull Request triggers unit and UI tests
Jenkins UI test run sample

14. Step III– Replace some UI tests with unit tests
• As a principleprefer lower level test over a higher one
• Don’t repeat a test - DRY
Step IV- Partial integration testing
• Create test hooks for navigatinginto app directly
• Deep links,shortcuts
• Level – 1 verifications
• Not everything need to be verified throughUI

15. StepV- Re-architech E2E tests
• Should be minimal
• Run them continuously
• On a cloud provider
• Should be atomic not interdependent
• Enables running in parallel
• A single test can be triggered in multiple ways
• Different language
• Different screensize
• Different OS
Specification Design Development Maintenance
1. Continuous testing
2. Crash Monitoring
3. Performance Monitoring
4. App store reviews monitoring
5. Engagement monitoring
1. Functional tests
2. UITests
1. Testability hooks
2. Accessibility ids
3. API level testing
1. Test infrastructure
2. CI/CD setup
3. Device lab
Summary

16. On the horizon – React Native
• Bringingweb likedeploymentcapabilitiesto native world
• Learn once, write anywhere
• Very high importanceto testing