Load and Performance tests in Agile Scrum
framework
S Ravindra
S R V Subrahmaniam
Siemens Limited
An approach for complex ...
Slide 2
Siemens
 Siemens a MNC with over 400000 people in more than 150 countries
 Siemens in India over 18000 people ac...
Slide 3
Agenda
Context
Challenges
Discussion
Solution
Further actions
Slide 4
New Product development – project context for our presentation
Enterprise grade
Not a typical web
application
N...
Slide 5
The Product : Communication platform
Touchscreen –
Thick client
application
Linux server Web server
IP PBX Digital...
Slide 6
Product Functionality
Client operator stations
Conference
Call forward
Make / Take calls
Send messages
Listen/talk...
Slide 7
Agenda
Context
Challenges
Discussion
Solution
Further actions
Slide 8
Uncertainty
over the
release dates
 Traditional waterfall
methodology
 Post code freeze, critical
Load and Perfo...
Slide 9
Traditional delivery model
System test
 Design
 Coding
 Unit tests
 Functional tests
Requirements
Code freeze
...
Slide 10
Nature of Load and Performance defects pose a challenge
Defects with high turn-around time
Occurs during
long run...
Slide 11
Agenda
Context
Challenges
Discussion
Solution
Further actions
Slide 12
Agile Scrum is the right choice
Complex technology
New product & need for early
customer feedback
Positive Organi...
Slide 13
Agile Scrum is fit for purpose
Niche
Generic
Simple /
Homegenous
Complex /
Heterogenous
Domain
Technical complexi...
Slide 14
Key considerations in defining a methodology within Sprint framework
Thick client
application, more
susceptible t...
Slide 15
Framing Agile testing for a Product in Safety critical environment
Sprint Test
Limited
feasibility
to do within
S...
Slide 16
Sprint testing
IN Scope for Sprint Testing
 User story working at a functional
level
 User story working at an ...
Slide 17
Testing model practiced in the project
Sprint 1 Sprint 2
Functional tested package
Sprint 3
Dedicated
test phases...
Slide 18
Testing methodology adopted
Sprint Testing
Enable test
Monitor resource utilization patterns; Focus on
tooling
Do...
Slide 19
Test Organization
Within Scrum teams
- Testers part of the Scrum teams
- Operationally reporting to Scrum Master;...
Slide 20
Other measures to foster Agile way of working
Management focus for moving towards Agile1
Visibility of the projec...
Slide 21
Benefits accrued..1
Team
Product
Methodology
 Enhanced collaboration between the development and test teams
 Le...
Slide 22
Defect turnaround rate improved by 20%
50% defects found early – i.e. During the Sprint testing
NFR test cycles r...
Slide 23
Agenda
Context
Challenges
Discussion
Solution
Further actions
Slide 24
Next improvements – to our project
Continous review
and
improvement of
tools &
techniques
 Invest in developing ...
Slide 25
Agenda
Context
Challenges
Discussion
Solution
Further actions
S. Ravindra (sm.ravindra@gmail.com)
S R V Subrahmaniam (srv.subbu@yahoo.in)
Thanks for your attention
Upcoming SlideShare
Loading in...5
×

Sgin2013 scrumfromthe trenches-loadandperformancetestsinagilescrumframework-subrahmaninamravindra

95

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
95
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
2
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Sgin2013 scrumfromthe trenches-loadandperformancetestsinagilescrumframework-subrahmaninamravindra

  1. 1. Load and Performance tests in Agile Scrum framework S Ravindra S R V Subrahmaniam Siemens Limited An approach for complex product development context Scrum Gathering India Regional 2013
  2. 2. Slide 2 Siemens  Siemens a MNC with over 400000 people in more than 150 countries  Siemens in India over 18000 people across locations  Over 5000 people in software development in India  Agile journey started in 2005  Today more than 45% of the projects following agile scrum  Basic Agile practices include: Time-boxed iterations, Continuous integration, test early & Self-organized, cross-functional teams
  3. 3. Slide 3 Agenda Context Challenges Discussion Solution Further actions
  4. 4. Slide 4 New Product development – project context for our presentation Enterprise grade Not a typical web application Niche product (domain / target users / use cases) Critical Non- Functional requirements Multiple deployment environments Centralized communication platform for use by emergency intervention forces Operators in the Command center will use it for communication with public and on-field intervention forces Safety critical application - used in Public safety and fire emergency situations Platform integrates Analog fixed, ISDN, Analog radio, digital radio (Tetra) Integration of heterogenous technologies
  5. 5. Slide 5 The Product : Communication platform Touchscreen – Thick client application Linux server Web server IP PBX Digital & Analog Radio Database Gateways for different types of incoming / outgoing calls Voice traffic carried over IP
  6. 6. Slide 6 Product Functionality Client operator stations Conference Call forward Make / Take calls Send messages Listen/talk in Radio Hold/Unhold
  7. 7. Slide 7 Agenda Context Challenges Discussion Solution Further actions
  8. 8. Slide 8 Uncertainty over the release dates  Traditional waterfall methodology  Post code freeze, critical Load and Performance test defects are uncovered – which sometime require fixes with high impact  The calendar duration for Load and Perfomance testing took 200% more time, compared to initial estimate  Technical challenges come to the fore much later in the project, when it is tested in a close to real environment Challenging environment  Complex interplay of different technologies including Java, .NET, messaging platform over VoIP /SIP.  Dependencies with telecom devices and equipments  Mixing audio from various streams and verifying for voice quality  Limited scope for test automation (using conventional tools) for this kind of a product  Multiple deployment environments Technology Process Uncertainty over Product quality. Management wanted to strengthen the tests given the criticality of the application
  9. 9. Slide 9 Traditional delivery model System test  Design  Coding  Unit tests  Functional tests Requirements Code freeze Critical bugs identified Related to: Long run stability Load and Performance Environment Heterogenous end-to- end operations Prolongs
  10. 10. Slide 10 Nature of Load and Performance defects pose a challenge Defects with high turn-around time Occurs during long run May require architecture fixes Bug verification – again takes time Fixes may have functional impact Hard to reproduce Defect occurrance does not follow a pattern
  11. 11. Slide 11 Agenda Context Challenges Discussion Solution Further actions
  12. 12. Slide 12 Agile Scrum is the right choice Complex technology New product & need for early customer feedback Positive Organizational mandate Late testing needs to be avoided Internal customer – less of contracting hassles
  13. 13. Slide 13 Agile Scrum is fit for purpose Niche Generic Simple / Homegenous Complex / Heterogenous Domain Technical complexity
  14. 14. Slide 14 Key considerations in defining a methodology within Sprint framework Thick client application, more susceptible to stability issues Used in emergency call centers. Zero tolerance for failure Given nature of application, limited automation options Device integrations are involved Defect turnaround time high
  15. 15. Slide 15 Framing Agile testing for a Product in Safety critical environment Sprint Test Limited feasibility to do within Sprint Testing for NFRs take time Accomodating the factors in the Agile test framework which is to be carried out within a Sprint is the critical decision NFRs need to be tested at earliest
  16. 16. Slide 16 Sprint testing IN Scope for Sprint Testing  User story working at a functional level  User story working at an end-to-end level Horses for the courses  Load and Performance tests  Stability tests run over long duration in multiple environments  Complex concurrent workflows in a full load scenario  Exploratory tests  Negative tests Production-like environment To be handled out of Sprint
  17. 17. Slide 17 Testing model practiced in the project Sprint 1 Sprint 2 Functional tested package Sprint 3 Dedicated test phases NFR test The Sprint build is taken up for long running stability, load and performance tests, multiple enviornment tests, exploratory tests and negative tests. Defects NFR test ..... Four weeks Product Backlog
  18. 18. Slide 18 Testing methodology adopted Sprint Testing Enable test Monitor resource utilization patterns; Focus on tooling Do tests where feasible – even if it is partial; Use test stubs where needed Plan the tests which need to be run; actively collaborate with Product Owner Bugs / tasks for forthcoming sprints Early detection will enable certainty in product release schedules Gather data Perform test Act on results
  19. 19. Slide 19 Test Organization Within Scrum teams - Testers part of the Scrum teams - Operationally reporting to Scrum Master; Officially reporting to Test Manager - Apprisal, Competency development taken care by Test Manager - Testers need to be good in inter-personal skills - Testers need to be aware of white box techniques, interpretation of logs, configuring and setting up parameters etc., System test team - Testers do not belong to Scrum teams - Testers report to Test Manager - Work on testing the software at System level and for non-functional requirements - Diversity in terms of technical skills is key. Each tester has to be a stand-alone expert in a chosen technical area as the breadth of testing would typically involve multiple areas
  20. 20. Slide 20 Other measures to foster Agile way of working Management focus for moving towards Agile1 Visibility of the project status and key metrics for all stakeholders2 A possible dip in productivity initially and a rebound afterwards3 Facilitating the transition with suitable trainings and coaching4 Transition with new processes and tools, in tandem5 Co-location and office layout to reflect the new work culture6
  21. 21. Slide 21 Benefits accrued..1 Team Product Methodology  Enhanced collaboration between the development and test teams  Less stress compared to fixing critical issues, late in the project lifecycle  Formulating Test organization for different test types  Re-skilling and Competency development  Riskier aspects of the Product tested upfront  Design refactoring for performance and stability is a continuous process  NFRs are given due focus - Enhanced visibility of the NFR related test cases to development and product teams  Product is as close to being production ready  Clear separation between the different levels of testing  NFR Tests planned and executed with due rigour without getting influenced by Sprint dynamics  Will support a move towards TDD Compliance to schedule and lesser critical bugs after code freeze
  22. 22. Slide 22 Defect turnaround rate improved by 20% 50% defects found early – i.e. During the Sprint testing NFR test cycles reduced by 25% 15% less of critical and major defects Benefits accrued..2    
  23. 23. Slide 23 Agenda Context Challenges Discussion Solution Further actions
  24. 24. Slide 24 Next improvements – to our project Continous review and improvement of tools & techniques  Invest in developing an automation framework at end-to-end and component Integration levels  Use of White-box tools and techniques for Load and Performance tests  Have specific user stories to improve performance Enhance Automation Sprint development
  25. 25. Slide 25 Agenda Context Challenges Discussion Solution Further actions
  26. 26. S. Ravindra (sm.ravindra@gmail.com) S R V Subrahmaniam (srv.subbu@yahoo.in) Thanks for your attention
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×