The enterprise applications being developed today implement multiple technologies to achieve goals such as real-time responses, secure handling of large data sets and highly intuitive user experience. In addition to this, there is a strong impetus to target greater audience and multiple devices. With this trend, the importance of non-functional testing is greatly amplified to ensure performance, scalability and security of the applications along the functional objectives. The challenges thrown by these aspects need to be addressed right from the early development phases or in agile terms, right when the first story is prepared. There is a definite need for a Test Framework that provides strong focus on integrating both the aspects of testing objectives under one roof. The framework should bring all the test activities under one platform and further drive it in such a way that one activity helps the other. On one hand, the functional tests can provide valuable inputs for performance issues and on the other hand, the performance tests can result into valuable inputs for security or usability. There is need for constant monitoring of the scope of optimization and re-use the existing test scripts across test flavors. The paper describes a Test Process Framework to integrate functional and non-functional requirements much earlier in scrum based agile projects. The framework utilized the benefits of their co-relation and validated design decisions in iterative builds. It also presents a case study on implementing this framework, using open source tools primarily Jmeter, Selenium Web Driver and OWASP utilities.

1. Integrated Functional and
Non-Functional Testing for Agile

2. Agenda
• Emerging Trends in Agile
• Test Challenges
• Functional and Non-Functional Test Process Framework
• Key Highlights
• Test Automation Strategy
• Integrating Performance and Security Analysis
• Choice of Open Source Solutions
• Conclusions
3

3. Emerging Trends in Agile
Users Stories
• Real Time Responses
• Rich User Interface
• Secure Handling of Large Data Sets
• Target Larger Audience and Multiple Devices
Client Requirements
•
•
•
•
Short Time Boxed Sprints
Early Visibility of the Product
Accommodate Important Feedbacks
Quick Time to Market
4

4. Major Challenges - Need For Systematic Test Approach
Test Strategy
Requirements
- Early sprints miss non-functional
requirements
- Unsynchronized test activities
- Complexity to define and measure nonfunctional requirements at module level
- Significant efforts in validating user
stories from the previous sprints
- Frequent changes in application UI
- Increased bug costs
- Coordination gap across test flavors
Agile Model
Costs
- Is it worth testing at design level? Client’s
thought
- Expensive test tools
- Complexity to mirror production
environment
- Reduced ROI on test automation
Time
- Quick time boxed sprints
- Decreased efficiency due to frequent
reworks across the test groups
- Significant impact on product release as
non functional testing occurs at last
stage
5

5. Test Process Framework
6

6. Key Highlights
• Precisely defines functional and non-functional
requirements
• Promotes synchronized test approach to uncover
performance bottlenecks and security threats at design level
• Defines test automation strategy to target frequent reworks
across the test groups and validates user stories in iterative
builds
• Correlates test activities to share common challenges,
solutions and critical observations thus creating a
collaborative test environment
• Reduces the time to market as the system is constantly
evaluated across the quality attributes from requirements till
production
7

7. Test Automation Strategy
• Select the Right Tool - Support for UI elements, report formats,
cost and integration with test management tools
• Modular Design for Test Repository Based on the application
modules being implemented in the current iterations
• XML Based Object Repository To accommodate frequent UI
changes
• Utility Classes and Low Level Functions For common
actions/frequent navigations in the application
• Prioritize Scenarios To first implement repeated application
workflows being performed prior to load/security test cycles
• Constantly Evolve the Automation Base Along the user stories
to accommodate build verification and regression tests
8

8. Integrating Performance and Security Testing
9

9. Choice of Open Source Solutions
The below tools come handy in implementing the framework:
• Selenium Web driver with Junit / TestNG
• Jmeter and Fiddler Proxy
• OWASP Security Tools
Selenium scripts can be integrated in Jmeter to perform the required actions.
Case snippet discussion:
10

10. Benefits
• A balanced test model for functional and non-functional test objectives
• Reduction in cost of quality with focused and synchronized test approach
• Early identification of risks in the development life cycle
• The continuous evolution process brings a positive change in the attitude
of the team towards meeting the quality standards
• Satisfactory results on production environment
• Builds a cohesive team experience among different test groups
11

11. Thank You
Visit us at: www.harbinger-systems.com
Write to us at: hsplinfo@harbingergroup.com
Follow us
Blog: blog.harbinger-systems.com
Twitter: twitter.com/HarbingerSys (@HarbingerSys)
Slideshare: slideshare.net/hsplmkting
Facebook: facebook.com/harbingersys
LinkedIn: linkedin.com/company/382306
17