Agile automation-practices by Shadi Mousa Galil Software

1Confidential and propriety Galil Software, Ltd. 20141-Jun-14
Implementing Agile Automation
Shadi Mousa
QA Automation Director - Galil Software
1

 Automation in Agile – what’s different?
 So, where to begin?
 Developing Automation in an Agile team
 The Automation team & responsibilities in Agile
 Agile Automation in large scale organizations
 The ATDD/BDD approach
 Case Studies
Agenda

 Agile requires “fast” automation development and execution; within
sprint time frames.
 Collaboration in Agile teams ensures the developed products are
testable by automation tools.
 Stories to automate are “simple” and clear – it is easier to prioritize
automation tasks.
 Automation milestones should be aligned with product release
milestones.
 Automation code refactoring may be needed several times;
especially with immature products and when automating fast.
 ROI of automation in Agile is clearer and more measurable.
Automation in Agile – What’s different?

 Assess current available automation
 Used tools
 Developed code and available auto tests
 Required automation skills to support current automation
 Select appropriate tools and framework
 Define automation targets
 Automating procedures - continuous environments; development,
integration and deployment
 Automating installation and configuration scripts
 Automating test scripts - regression & progression
 Build an automation resources strategy
 Who will develop automation code?
 Who will develop script level?
 Who will lead the automation roadmap, integration, E2E …?
 Consult with Galil Software automation experts 
So, where to begin?

Developing automation in Agile

 Option 1 – Self contained team
 Tester skills should include the capability to create the required stories in the
automation tool format
 Product developers are responsible for implementing the code layer of the automation
tool
 All required tools and automation framework are the team’s responsibility
 Option 2 – Dedicated Automation resources
 The larger and more complex the product becomes, the more the need to establish a
dedicated team
 Automation must include automation experts, including skills with the selected tool
The Automation team & responsibilities in Agile
Role Responsible
Option 1 Option 2
ScrumTeam
Test planner Manual Tester Manual Tester
Automation developer Developer Auto Expert
Automation executer Manual tester Manual Tester/ Auto Expert
Automation Maintenance Developer Auto Expert
…
Automation
Team
Automation Road Map responsible Scrum Master/Product Owner Automation TL
Infrastructure & Tools ??? Auto Experts
Integration Between Multiple Teams ??? Automation TL

Automation in large scale organizations
 A Core Automation Team is required
 Centralized automation development and maintenance
 Responsible for integration and E2E automation
 Dynamic automation resource assignment

 ATDD – Acceptance Test-Driven Development
 BDD – Behavior-Driven Development
 The main idea behind ATDD and BDD, in the aspect of automation,
is that the test automation is built before coding, and delivering the
new code depends on the success of the automation
 Human language as a base for automation scripts:
 User story
 “As a [role] I want [feature] so that [benefit]”
 Acceptance criteria
 Given [initial context], when [event occurs], then [ensures outcome]
 Tools that implement ATDD / BDD: JBehave, Cucumber, Robot …
The ATDD/BDD Approach

 Product – Operations Management & Optimizer Service
 Operations Admin portal – Web Client
 Operations Optimization Server – ASP .NET server
 Database - MSSQL
 Team and Method
 Method: Scrum
 Team: 5 Developers, 2 QA, 1 Team Leader (Scrum Master)
 Team Responsibility
 CR, bug fixes, features and improvement requests within a production
environment
 Sprint Description:
 Sprint duration – 3 weeks
 Content - contains sub tasks of CR, feature development, design and
testing tasks…
Case Study 1

 Scrum Master requested automation:
 “I want to have a strong regression testing suite that will verify the
correctness of complicated features; this will be used during the sprint
and also in the future when working on other sprints/features.
In addition, the automation should run in the integration environment
before deployment to customer.”
 Selected feature to automate:
 New optimization option added and activated via a button from the
Administrator Portal; the data entered by the administrator is optimized
upon request.
 Automation “Starting point”:
 The customer worked with an outdated and unmaintained automation.
 The decision was to start from scratch.
Case Study 1 – Cont.

 Selected tool – Robot Framework with Selenium2Library
 Resources:
 The project started with 1 QA engineer who was part of the team
 QA engineer had the relevant programming skills
 Task assignment as part of the sprint tasks
 Defined Script:
 Admin enters “Operation” data via Admin portal
web application
 Activate the new optimization option on the added
data
 Verify the entered data in the appropriate table in
the DB
 Verify the converted data in the targeted table is
equal to the new table
 Run on different sets of data and expected results

Robot Script example
*** Test Cases ***
TestCase 1 - Login to Admin Portal
Open Browser ${AdminPortalUrl} ${BROWSER}
Maximize Browser Window
${username} DataReader.getValue ${Credentials_fileName}
AdminPortalUser
${password} DataReader.getValue ${Credentials_fileName}
AdminPortalPassword
Login Portal ${username} ${password}
TestCase 2 - Data Sets Import Verification
${DataFileList} ${length} DataReader.getFileList
: FOR ${INDEX} IN RANGE 0 ${length}
Click Link ${Import_Operation Data_link}
${XSL_fileName} DataReader.getFileName ${DataFileLis
t} ${INDEX}
Load Operation Data File ${XSL_fileName}
Press Button ${Optimize_Vacation_button}
Press Button ${Accept_Data Change_button}
${DB_connection} Connect to
Database ${DB_host} ${DB_user} ${DB_password}
${isEqual} Compare DB Table to
XSL ${DB_connection} ${Vacation_Opt_table} ${XSL_fileName}
Assert True ${isEqual} Table Data is not equal
between ${DB_host}:${Vacation_Opt_table} and ${XSL_fileName}
Close Browser
[Teardown] Close Browser

 Results
 Auto script was performed successfully, but in the following sprint.
 Various data sets were supplied, and as test data
 Script was added to the regression suite (first to be added)
 Best Practices and Conclusions
 Automation should have dedicated resources, and not rely solely on QA engineers’
“best efforts”; especially in order to align automation deliveries with sprint timeframes
 On the other hand, “best efforts” solved wasted time in QA engineer resources
 Selecting the right tools is crucial; especially if “start point” of no tools in place
 QA documentation was added directly to test scripts
 The outcome of the scrum team automation must be shared with other teams
 Next steps
 Request for dedicated resources; by increasing the QA testers with programming
capabilities and they perform manual AND automation testing.
 In parallel, create and develop automation keywords for previous developed features
 Define and execute a work plan for integration environment

 Product – ITIL System
 Phone rep client – Windows desktop application (Winforms)
 ITIL Servers – farm of ITIL servers
 Team and Method
 Method: Scrum
 Team: 4 Developers (1 remote – near shore), 3 QA (offshore), 1 Team
Leader (Scrum Master)
 Team Responsibility
 CR, bug fixes, features, and future versions of the client
 Sprint Description
 Sprint duration – 2-3 weeks
 Content - contains sub tasks of CR, feature development, design, unit tests
and testing tasks…
Case Study 2

 Scrum Master requested automation:
 “I want to have a strong regression testing suite that will verify the
correctness of complicated features.”
 “Automation will be added to the nightly build environment.”
 “I also need automation for the client as an independent component without
the need to set up the entire system.”
 Selected feature to automate:
 New field was added to the client; a value entered in the field is sent to the
server
 Server may reply with different results according to submitted value
 Connection timeout should be handled
 Automation “Starting point”:
 Client – No automation available
 Servers – Available APIs

 Selected tool – Coded UI (TFS)
 Resources:
 QA testers in the scrum team had no development skills
 Development engineers had no capacity to handle automation
 External automation expert was assigned to the scrum team
 Defined Script:
 Phone rep enters a value in the
client and submits
 Rerun over list of optional values
and expected server replies
 Simulate reply timeout

 Results
 Task took longer than expected:
 Desktop application was not ready for automation; UI controls missed IDs –
task for development was added to the sprint to reorganize and fix the UI
controls IDs
 Auto script was performed successfully, yet completed in the following
sprint
 Script was added to the regression suite
 Isolating desktop application was performed after writing the script
 Isolating desktop application was
performed by simulating server
connection and responses on local
port of the automation client

 Best Practices and Conclusions:
 Team Leader feedback:
 Adding the automation resource to the scrum team and participating in the
design made the application better designed and implemented
 Automation should have dedicated resources, with the relevant skills
 A supportive team for automation is required, since in such a large
system, self contained automation resources cannot perform all
required automation tasks
 There is a need for automating utility / configuration scripts such as
upgrading environments…
 Next steps:
 To add continuous tasks for developers to make the application testable
by exposing the appropriate control IDs, recognized by automation tool
 Define and execute a work plan for continuous integration environment

Thank You
Shadi Mousa
QA Automation Director - Galil Software

Agile automation-practices by Shadi Mousa Galil Software

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Agile automation-practices by Shadi Mousa Galil Software

Similar to Agile automation-practices by Shadi Mousa Galil Software (20)

Recently uploaded

Recently uploaded (20)

Agile automation-practices by Shadi Mousa Galil Software