For more info visit: www.zentestlabs.com www.blog.zentestlabs.com --Mobile Application Testing --Mobile Configuration Optimization --Mobile Test Automation --Mobile Performance Testing

1. Zen Test Labs
Mobile Application Testing

2. ABOUT ZEN TEST LABS
Domains
Technology, Tools and Frameworks
Development & Testing Expertise
Product Focus
ISO 27001 & ISO 9001 certified
processes
Domain accelerators
ROI driven engagement models
Thought leadership- STAR
conferences, etc.
Geographic Spread
USA operations based out of
Chicago and Maine
Offshore testing lab in India
Ability to deploy onsite resources
for key phases of projects
Ability to dip into resource pool to
fill onsite gaps
Customer base that spans US, EU,
APAC and India
Partner network that spans US, EU,
MEA and APAC
2
Mobile
Gaming
Cloud
Banking & Financial Services
Banking & other ISVs
eLearning

3. TYPICALCHALLENGES FACEDAND THEBUSINESS IMPACT
3
Common Challenges
•OS & Device fragmentation
• Rapid rollout schedule
• Dealing with a wide range of connectivity modes
• Security
• Data Usage
• Installation and launch
• App Performance – Memory and BW consumption,
behaviour under low bandwidth & memory
Business Impact
• Having an app run on some devices & not others is a
major hindrance to widespread adoption.
• Quick rollouts can become extremely expensive for
the company’s reputation if the app doesn’t meet
the quality users expect.
• Wi-Fi, 3G, 4G, all connectivity modes can
exponentially multiply the no. of tests to run and
become costly & time intensive.
• Users expect personal data to remain personal &
expect apps to be 100% free of viruses, malware &
other threats.
• Users abandon apps that drain battery faster in
favour of more economical ones.
• 60% of mobile users abandon the app or website if it
doesn’t load within 3 seconds!

4. THE ZEN TEST LABS’ SOLUTION
4
MOBILE CONFIGURATION OPTIMIZATION
Zen Test Labs’ unique utility, provides the optimum no. of
configurations to test your app on in a few easy clicks.
MOBILE TEST AUTOMATION
Zen Test Labs’ automation methodology guarantees 60-70% reduction
in testing effort, complete coverage across devices & highly reusable
scripts.
MOBILE PERFORMANCE TESTING
Holistic approach to performance testing covering areas such as volume
testing, endurance testing, performance monitoring, soak testing and
testing under real time scenarios.

5. ZEN TEST LABS’ CONFIGURATIONOPTIMIZATIONUTILITY
5
WIDE CHOICE OF
DEVICES
• Select based on
app. Type
(Native, Web,
Hybrid)
• Select based on
device type (
Smartphone,
Tablet)
WIDE CHOICE OF
CONFIGURATIONS
• Select OS and
Versions
(Android, iOS ,
Windows Phone
etc.)
• Select device
manufacturer
(Samsung, HTC,
etc.)
OPTIMIZED DEVICE
COMBINATION
• Prioritized &
optimized
combination of
devices to test
your app.
• Predict
popularity of
devices in target
segments (geo.
specific)

6. ZEN TEST LABS’ CONFIGURATIONOPTIMIZATIONUTILITY
6
MARKET
RESEARCH
WIDE RANGE OF DEVICES
TO TEST
OPTIMIZED &
PRIORITIZED SET OF
DEVICES TO TEST
BI

7. ZENMOBILE: ZEN TEST LABS' MOBILEAUTOMATION FRAMEWORK
7
APPLICATION UNDER TEST
(PARALLEL EXECUTION ACROSS MULTIPLE
DEVICES WITH DIFFERENT OS’S)
AUTOMATION TOOL (COMMERCIAL: QTP
PLUG-IN OR OPEN SOURCE: MONKEY TALK)
ZENMOBILE AUTOMATION FRAMEWORK
TEST MANAGEMENT
TOOL
EXECUTE
BUILD
FUNCTIONS
CALL
FUNCTIONS
BUILD SCENARIOS
BUILD TEST CASES
BUILD TEST DATA
VIEW REPORTS
REPORT &
MANAGE

8. ZENMOBILE (FRAMEWORK) FEATURES
8
 Dashboard rich with features like displaying ongoing execution along with status, time required for
execution of individual business functions
 Parallel execution across multiple devices and configurations
 Development language agnostic
 Automation tool agnostic
 Runs on real mobile devices as well as emulators
 Provides performance analysis reports
 Test data reusability
 Ability to work in a continuous integration environment
 Ability to work with native, web based and hybrid apps

9. ZEN TEST LABSMOBILEPERFORMANCE TESTING
9
Test
performance
across variety
of devices and
environments
Load Testing
Stress Testing
Volume Testing
Soak Testing
Scalability Testing
Interruption Testing
Provide Metrics
Across
Resource Utilization
App. Behavior
Memory Leaks
Response Time
Test app.
performance
under varying
conditions for
bandwidth,
memory and
battery.
Gather Inputs
Establish Performance
Benchmark
Performance Goals
Critical Scenarios
Workload

10. ZEN TEST LABS’ MOBILETESTINGMETHODOLOGY
INPUT
GATHERING
• Plan to release, current
status , geography
• Define purpose of
testing.
• Classify app according to
type(Native, Web,
Hybrid)
• Establish the OS and
versions on which the
app will run
• Establish the no. of
releases planned
• Establish the budget for
the number of
configurations.
APPLICATION
DIAGNOSIS
• Explore the application
• Size the application
• Identify best
configurations for the
app based on market
research
• Identify the no. of rounds
required for testing
• Design optimized test
cases
• Identify readymade test
case repositories
• Calculate total effort
required for testing
ROADMAP &
DELIVERABLES
• Provide a roadmap for
testing the application
• Provide a list of optimum
configurations required
for testing
• Outline the total effort
required for testing
• Provide value adds for
achieving better ROI
• Provide an overview of
other non functional
tests critical to the
application
• Deliverables
• Identify if specialized
test are required
10

11. ZEN TEST LABS EXPERIENCE
 Test automation on the cloud for a US based leading e-gift card services provider
 Game testing for an iPhone based game development company
 Web testing of games and tools for a leading academic and reference books publishers
 End to end testing of a mobile data back up application for Mobibook (a mobile application
development company) across mobile platforms and social networks
 Manual testing for a web 2.0 based email marketing solution
 Web testing of games and tools for a global media conglomerate engaged in publishing, music,
magazines, TV and radio production and distribution, as well as Internet services
 End to end testing of multiple mobile and web based applications for a leading “user-centric”
software solutions company
 Web testing of games and tools for one of the largest US-based development house for learning
materials.
11

12. CASE STUDY
The Client
The client is a product based company, and they introduced an app as a new product line. The app enables the user to
read their favourite stories anywhere, anytime be it either books, magazines, blogs etc. The user can browse, search, and
download them on the mobile phone! They can re-download stories from My Bookshelf countless times and all data was
meant to be stored on Cloud. It also enables a cross platform multi-lingual chat .
Key Project Challenges
• Constant crashing of the App due to instability
• To be tested across various configurations considering wide range of devices with multiple OS versions
• Limited bandwidth of resources which hamstrung the width of testing
• Vendor issues with domestic and global ones for testing payment gateways
• Frequent releases, since agile mode
• Undocumented requirements and constant evolving of requirements
Our Solution
Zen Test Labs put together a team of 2 manual testers. In addition to this Zen Test Labs developed a mobile testing lab
consisting of the latest devices the client wanted the app to run on
• Consulted at the business Requirements stage to structure and ensure coverage of the testing process
• Performed End to End testing scenarios via user simulation testing
• Performance testing of app on cloud with 10,000 users
• Functional test automation and performance testing
• Optimized device and OS configurations to ensure maximum output in minimum time
12

13. CASE STUDY
13
• Acquired business objectives from the client
• Expectations from the stakeholders
• Challenges and methodologies from the developers
Inception
• Milestones of the project
• Aligned release dates along sprints
• Wrote test cases which were used by developers to
ensure coverage and functionality
Plan
• Executed on the basis of Agile
• Performed user simulation testing and allowed ‘Carry
the device home'
• Immediate reporting of bugs
• Simultaneous fixing of bugs which resulted in the
defects being fixed before the next release went live,
<reduce the cycles>
Execution & Reporting
• Performed Test automation and regression suits
followed by performance testing on the staging
environment which was a replica of the production
environment
• Sign off
Testing on Production
Environment
Key Results
• Logged 316 defects with
more than 25% falling
under the category which
were crashing the app
• Automated a regression
suite of 250 test cases
• Single click run facility to
run automated smoke test
cases within 30-45
minutes
• Reduced 1.5 day of
manual execution to 2
hours along with the
facility of auto reporting
• Simultaneous fixing of
bugs which resulted in the
defects being fixed before
the next release went live,
which in turn reduced the
cycles
• Reopen ratio was
drastically reduced
Methodology

14. Office No.2, 2nd Floor,
Abja Pavilion,
D.P. Road, Aundh,
Pune - 411007,
Maharashtra, India.
AUTOMATE
VALIDATE
INNOVATE
Thank You!
Blog: www.zentestlabs.com/blog
Website: www.zentestlabs.com