Experience in apply agile practices for test automation in scrum projects

1. Software Test Automation Practices in Agile
Development Environment: An Industry
Experience Report
Eliane Collins, Dr.Vicente F. De Lucena Jr.
AST 2012 Nokia Technology Institute

2. Agenda
1. Introduction
2. Software Projects Characteristics
3. Study Cases
4. Lessons Learned
5. Conclusion
Instituto Nokia de Tecnologia
1
Nokia Internal Use Only

3. Nokia Technlogy
Introduction Intitute
INdT – Nokia Technology Institute
Independent and Nonprofit institute
committed to conducting research and
development of technological solutions
through application development, new
technologies and concepts. Manaus
Recife
Brasília
Main Areas:
• Product Creation
• Apps and Services São Paulo
• Hardware and Manufacturing
• Software Open Source and User
Interface Instituto Nokia de Tecnologia
2

4. Introduction - Scrum
• Agile development process focusing on
teamwork. Large used in the industry.
• The functionalities are listed in the
Product Backlog.
• The team defines the features which
may be accomplished within the
iteration (Sprint).
• The Software Testing process must be
agile for all projects.
Instituto Nokia de Tecnologia
3
Nokia Internal Use Only

5. Introduction – Agile Testing
• Testing with a plan to learn about it,
let the customer information guide
the testing in line with agile values
working software which responding to
change .
• Tests to prevent defects.
• Proactive Testers.
• Test Automation is the key of
successfully agile development and
the core of agile testing.
• [Crispin, L.; Gregory, J. 2009]
Instituto Nokia de Tecnologia
4
Nokia Internal Use Only

6. Software Projects Characteristics
Project 1:
Web interface to register campaigns, users and ads attaching
images, videos and texts with good performance.
Team: Scrum Master, 3 full-time developers, 1 half time Test
Leader and 2 testers.
Project 2:
Web project to attend Nokia factory. It automates procedures
to control materials and production costs.
Team: 4 developers, Scrum Master one Test Leader half-time
and two Testers full time.
Instituto Nokia de Tecnologia
5
Nokia Internal Use Only

7. Software Projects Characteristics
The Testers used tools to automate test tasks:
TestLink
Mantis Bug Tracker
Subversion
Jmeter
Selenium
Fitnesse
Java platform
Instituto Nokia de Tecnologia
6
Nokia Internal Use Only

8. Study Cases
Project 1 First Attempt:
Development Team automating unit testing and coding;
Testers automating all UI tests (rec-and-play) and non-
functional tests;
No support of developers and no continuous integration.
Problems:
Internally separation of Testers and Developers;
No Knowledge transfer between scrum team;
High learning curve in automation test tools.
Automation difficult to maintain and wasn’t efficient
because of changes in the interface;
Instituto Nokia de Tecnologia
7
Nokia Internal Use Only

9. Study Cases
Communication problems between scrum team;
No time to execute tests to agregate value to the project;
Waterfall effect;
Load and Security tests were executed in the end of project
Project Second Attempt:
Development Team automating unit testing and supporting
Testers with automated functional tests;
Testers supporting unit tests and automating functional
(rec-and-play) for tests in the sprint for stable interface.
Use of Testlink and Selenium APIs to integrate tools and
results;
Instituto Nokia de Tecnologia
8
Nokia Internal Use Only

10. Study Cases
Improvements found:
Knowledge transfer between scrum team;
Automated regression tests works well for stable interface;
Time to exploratory tests;
Communication between team improved;
Quick Feedback of tests and bugs to project Team
Instituto Nokia de Tecnologia
9
Nokia Internal Use Only

11. Study Cases
Project 2:
Developers coding new features, unit testing, supporting UI
tests, using Continuous Integration (Hudson).
Testers supporting Unit testing, supporting Hudson
environment, automating UI Tests, Load Tests and Security
tests in the beginning of project.
Instituto Nokia de Tecnologia
10
Nokia Internal Use Only

12. Study Cases
Project 2 Problems found:
Accuracy data values. The tests for complex business roles
were missing for the layer behind the GUI.
No Prioritization of the regression test execution.
Project 2 Solution:
Team together includes Integration and API Tests using
Fitnesse Test Tool and prioritize regression tests in order of
risk.
Instituto Nokia de Tecnologia
11
Nokia Internal Use Only

13. Lessons Learned
Collaboration is an Essential Factor for the Success of Test
Automation in Agile Projects;
Fit Testing Tools with Test Strategy and Agile Method;
Automate Each Layer of Software When It Is Possible and Just for
Acceptance Tests in the Sprint;
Test Automation in the Agile Project Should Be Simple;
Prioritize the Regression Test Execution;
Automate and Run Security Tests and Stress Tests Early Reduces
Risks and Rework;
Use Test Automation for Documentation and Information
Feedback ;
Instituto Nokia de Tecnologia
12
Nokia Internal Use Only

14. Conclusion
It is feasible to adapt agile practices and values to test automation.
The automation was a resource to document software, reduce
cost, and allocate tasks in smaller parts.
Next steps:
Evaluate the effectiveness of the test automation agile model
in other development platforms;
Evaluate the impact of team collaboration levels in the test
automation.
To extract the testing effort, number of detected failures pre
and post-delivery, and schedule adherence.
Instituto Nokia de Tecnologia
13
Nokia Internal Use Only

15. Thank you.
Questions?
INFO + CONTACT
www.indt.org
ext-eliane.collins@nokia.com
Nokia Technology Institute