Now we talked about the How can they overcome this challenges
It is not necessary to have the organization to have all the four stages…
Amalgamation of BDD, parallel execution and mobile automation
parallel execution and
- Jigesh Shah
- Mradul Kapoor
2 Amalgamation of Behavior driven development, parallel execution and mobile automation All rights reserved.
Mradul Kapoor is working in one of the leading consulting firm based in Mumbai, India. His experience,
knowledge and skills include Business Analysis, Automation and Functional Testing. Mradul has worked on
various testing tools like QTP, Selenium, Oracle Oats, Coded UI, SpecFlow, SpecLog, Appium, Microsoft team
system and various test management tools. He has over 7.5 years of experience in US Healthcare, Insurance,
eLearning, Life Science and Human and Health Services
Jigesh Shah is working in one of the leading consulting firm based in Mumbai, India. His experience, knowledge
and skills include Business Automation and Functional Testing. Jigesh a Certified Java Programmer has
worked on various testing tools like QTP, Selenium, QC, Rally, JIRA , SpecFlow, SpecLog, Appium, Microsoft
team system and various test management tools. He has over 7.5 years of experience in US Healthcare,
Insurance, and eLearning
3 Amalgamation of Behavior driven development, parallel execution and mobile automation All rights reserved.
With the rise of Mobile first approaches, a substantial traffic has now shifted from the desktops to the
Mobile phones. To ensure quality in mobile apps, Testing needs to be continuous and inclusive. This makes
Mobile Test Automation a formidable business driver. Along with that there needs to be strong collaboration
between developers, testers and non-technical or business participants in a software project to maintain
application consistency across multiple platforms.
The intent of this presentation will be to provide a comprehensive mobile automation testing strategy using
Behavior Driven Development, optimized selection of target devices, and an effective combination of
manual and automated testing; use of correct tools to cover appropriate functional testing essential for
getting your mobile applications to market on time and within budget.
This presentation will also discuss on how the scenarios which are derived from the requirement can be
automated using BDD approach across Web and Mobile. Also, it will highlight on how parallel execution
can be achieved using BDD & Mobile Automation.
Target audience : Senior Leadership, Head Testing Practice, Head Quality Assurance, Delivery Head, Test Managers, Test Leads and Test Professionals
4 Amalgamation of Behavior driven development, parallel execution and mobile automation All rights reserved.
Current market analysis and trends
With the fantastic growth of mobile computing platforms there has been a dramatic increase in the value
of mobile applications for most companies.
One of the biggest challenges that one faces when developing a mobile application is how to test it.
Period Samsung Apple Lenovo Huawei Others
Q1 2015 24.60% 18.30% 5.60% 5.20% 46.30%
Q1 2014 30.70% 15.20% 6.60% 4.70% 42.80%
Q1 2013 31.50% 16.90% 4.70% 4.20% 42.70%
Q1 2012 28.90% 22.90% 5.00% 3.40% 39.80%
Period Android iOS
Q1 2015 78.00% 18.30% 2.70% 0.30% 0.70%
Q1 2014 81.20% 15.20% 2.50% 0.50% 0.70%
Q1 2013 75.50% 16.90% 3.20% 2.90% 1.50%
Q1 2012 59.20% 22.90% 2.00% 6.30% 9.50%
Source : IDC May 2015
6 Amalgamation of Behavior driven development, parallel execution and mobile automation All rights reserved.
What are the testing challenges faced by today’s organizations while
automating mobile and web automation?
Mobile automation challenges
Rapid rollout schedule
• Rapid rollouts of bug fixes
• Constant expectation to meet
• Issues with multiple Devices and
• Object Identification support issues
• Difficult to test in the field
• No VPN on old devices
• How to push updates and test them?
• Issues with understanding
• Each team has developed
their own understanding
• Identify devices to test
on to support your user
• Identify device limitations
Right tools & approach
• Applications supported
• Lack of Frameworks
• Low Automation expertise
Lack of documentation
• Documentation is
neglected at times
• Document not inline with
the code changes
• Delays in
• Lack of proper
• No correct ROI of
• High script
• Low resource moral
• No correct channel
is followed for
8 Amalgamation of Behavior driven development, parallel execution and mobile automation All rights reserved.
Description Benefits Challenges
manual on mobile
• All manual don’t automates • No additional cost for transition or
to build Mobile Automation
• No need for Mobile Lab Setup
• No additional
• Does not address underlying issues
• Risk of credibility and brand value
• Loss of revenue and business
Web and mobile
• Automate Web and Mobile
Application without Continuous
Integration and Parallel
• Efficient Testing
• Upgradation and Reusability
• User Environment Simulation
• Reduces Human Error
• May result into code redundancy for
mobile and web scripts
• Higher transition cost
• mobile technologies changes too fast –
automation need continuous support.
• May result into inconsistency
Description Benefits Challenges
• Automate Web and Mobile
Application including Customer
Centric Solution, Continuous
Integration and Parallel
• Thinking through solutions,
before starting to code
• Documentation of Assumptions
• Addresses Device Fragmentation
• Remote and cloud automation
• Test consistently. Each test is
executed in the same way
• Active involvement from Business and
• Resistance to change
• Mobile technologies changes too fast –
automation need continuous support
Here are some key options…
10 Amalgamation of Behavior driven development, parallel execution and mobile automation All rights reserved.
Architecture and design
• Captures the unique
identifier to handle the
• Stores the environment
and other global
variables to determine
the config values
• Handles the Application
under Test data and
• Reusable page/module
• Reports and Logs
• Contains the logic to
• Reusable functions for
Native and Web Apps
• Contains the actual Test
Set of Preconditions
Set of Test Condition
Mobile and web applications
Set of Expected Results
BDDWeb and mobile automation
11 Amalgamation of Behavior driven development, parallel execution and mobile automation All rights reserved.
Continuous integration for mobile and web
Create mobile project for testing :
QA creates the scripts and
submits it to the Code
Execution on node machines
Hub sends the tests to each
node and when it is executed
the result is sent back to the Hub
Run deployment scripts on hub
CI Server compiles and builds scripts
and sends it to the Hub Machine
The Automation Test Execution results
are sent to QAs and Stakeholders
Send test results
The Hub Machine sends the
consolidated result back to the CI
13 Amalgamation of Behavior driven development, parallel execution and mobile automation All rights reserved.
These elements can serve as a foundation to build an efficient solution
Business case study
• Ran the Test Suite which comprised of 20 Test Scripts on Mobile and Web Application
• Out of these 80 Test scripts(20* 4) , 96.25 % scripts passed
• Developed automation test suite comprising of 20 test scripts covering 80% of application screens
on both Mobile and Web
• Also covered 80% of the gestures supported by the Mobile Automation Tool
Effort Saving and
• Test Execution time for Manual Test Cases is around 20 Test Cases * 4 Devices * 15 min =
Approx. 1200 min/Cycle =20Hr/cycle
• Test Execution time for Automation Test Cases is around 20 Test Cases * 4 Devices * 8 min =
Approx. 640 min/Cycle =11Hr/cycle.
• Executing parallel on 4 nodes will again reduce the time by 11 / 4 = ~ 3 hours
• Defects are detected early whenever a new build is deployed
• Recovery scenarios helps the Test Cases to proceed ahead in case of any unwanted failures
• Details trace of the test is also provided for analysis which can help in concluding an issue.
• Using BDD Components, the same scenario can reused across multiple device with minimal
• The Web scripts that were used to automate the Web Application have been extended to
incorporate the code for Mobiles
15 Amalgamation of Behavior driven development, parallel execution and mobile automation All rights reserved.
• Reduced coding & maintenance efforts as the tests runs on multiple platforms
• Reaping the rewards in the future as we can add more test cases or update existing application
• The Mobile Device Lab mandates a particular process for adding devices to the lab. And also
makes the lab easily scalable. Common processes can be set for tasks such as resetting
installing/uninstalling software on the devices, software or OS upgrade, etc.
• It helps in drastically reducing the time required to get the test results
• Everyone in the chain knows exactly what test cases are being executed and the test data being
used, as the test cases are written in Gherkin (Given-When-Then) format which are easily
readable and can be understood by everyone involved in the SDLC
• QAs can just pick these tests and plug them into the framework saving the time & effort of re-
writing automation scripts.
• Testing results are more reliable since we test on devices.
• Issues can be reported in the software encountered for low bandwidth, security, performance, etc.
• Automation runs on real devices can be kicked off immediately without any manual intervention or
waiting for the QA team to start the process.
• Parallel execution of test suites can be easily configured and fired under CI tool
17 Amalgamation of Behavior driven development, parallel execution and mobile automation All rights reserved.
There are four phases to implement BDD with mobile automation
Feasibility Study and POC
• Benefits of automation in this project?
• Identifying parts that can be automated on mobile and integrate
• Implement the POC along with BDD
• Identify challenges and scope of optimization
• Implement the framework along with parallel execution
• Integrate mobile and web automation effectively
• Add BDD along with required traceability
Rollout and Optimize
• Rollout the implementation across project
• Optimize the common components based
on learning from different projects
• Rollout the updated implementation
19 Amalgamation of Behavior driven development, parallel execution and mobile automation All rights reserved.
Lessons learned and best practices
Shift focus from Tests to Behavior :
It helps to describe behaviors in a single notation which is directly accessible to domain experts, testers and
developers, so as to improve communication
Invest in Continuous Integration :
Having an automated job that will build and deploy applications on the mobile, trigger automation runs, email
notifications, etc. would really help in achieving a 24/7 service.
Invest in Mobile Labs :
Testing on real devices helps to uncover some critical bugs and failures and also helps to determine the
actual behavior of the devices in various conditions
Use Page Object Patterns :
It helps the operations and flows in the UI to be separated from verification. This concept makes the code
cleaner and easy to understand.
Develop Strong Test Recovery :
Develop agents that can restart test components on devices. Enhance test case recovery, capability to
restart test case.
20 Amalgamation of Behavior driven development, parallel execution and mobile automation All rights reserved.
For organizations facing the challenges related to merging of multiple technologies and platforms, the
customer centric solution amalgamated with parallel execution and Mobile Automation will help to address
If implanted properly it can help Organizations to achieve
• Collaboration between Business stakeholders, Business Analysts, QA team and Developers
• Multilingual Test Scenarios (35+ Languages)
• Right tools and the right approach
• Remote and Cloud Execution
• Parallel Executions
• Framework and Programming scalability
• Continuous Integration
21 Amalgamation of Behavior driven development, parallel execution and mobile automation All rights reserved.
• IDC : IDC May 2015
• Jigesh Shah & Mradul Kapoor : “Traceability and Automation using BDD”
• Cucumber : https ://cucumber.io/
• Parallel and Web Automation : http ://www.seleniumhq.org/
• Mobile Automation : http ://appium.io/