ING Direct: How we do performance
testing in agile environments and bridge
the gaps between IT teams

Todd DeCapua
Product...
Show of Hands: Who Is Running Agile?




                     2
Show of Hands: Do You Ever Worry About This?




                       3
Show of Hands: Are You a Performance Engineer?




                        4
Where does performance
testing fit within your Agile
    application lifecycle
  management process?

             5
Philosophical Turning Point:
    “Flipping The Triangle”
     • Agile 2008 – Toronto
     • Session Title:
           • Fl...
7
8
It Fits Throughout the Process




                       9
Key Barriers of Agile
    Performance Testing:
             People
(dedicated performance test engineers)


              ...
• Embedded Performance in All Teams at All Roles
   • Unit = Developer Role
   • Integration = Developer & QA & BA Role
  ...
Key Barriers of Agile
Performance Testing:
 Non-Functional
Acceptance Criteria
  (included in each story)

            12
User Stories: Mike Cohn – Simple Template

• “As a <type of user>, I want <some goal> so that <some reason>.”




        ...
Breakdown of Epic to Tasks with Acceptance Criteria

  • Epic (A very large idea or goal)
        • Story (One of many wor...
Feature “Doneness”

 The code is included in the proper branch in the source code control
  system.
 The code compiles f...
Key Barriers of Agile
   Performance Testing:
       Environment(s)
(Enabling proper iterative testing with
           dat...
How: Environments and Quality Gates

                              VIRTUAL ENV                                            ...
Development                         Pre Production                      Production
           Environment                 ...
Key Barriers of Agile
Performance Testing:
          Tools
(Simple, quick, and continuous
           feedback)

          ...
How: Simple Tool Integration to Support Process

          Software Delivery                  Test Management             ...
Show of Hands: What Do You Use?




            GUI    LoadRunner

                            Jmeter / Service Test
     ...
How we bridge the gap between
   testing, monitoring, and the
    relationship with Capacity
 Planning / Disaster Recovery...
Monitoring: How It Works Together
              Production Operational        Capacity Planning
                   “Real T...
Production Quality & Process


 Dev / QA       Pre Prod     Prod / DR
                             Continuity / DR
       ...
Key Learnings: Performance Testing in Agile

• QA Environment Management
   • Quick Refresh
   • VM Templates
• Test Data ...
Questions?




                          Todd DeCapua
             Production Quality & Process Group Leader
             ...
To learn more on this topic, and to connect with your peers after
   the conference, visit the HP Software Solutions Commu...
28
Upcoming SlideShare
Loading in...5
×

ING Direct: how we do performance testing in agile environments and bridge the gaps between IT teams

2,933

Published on

Where does performance testing fit within your agile application lifecycle management process? How do you confront the challenges of agile for performance testing and iterative performance regression testing—like staffing, defining non-functional acceptance criteria, environmental constraints, and tool selection? This interactive session, led by a scrum practitioner, will give you insights into these challenges. We will cover topics such as where performance testing fits into agile development, how to get continuous feedback to teams, establishing integrated best practices across business analysts, development, QA and infrastructure teams, and simple processes and tools for quickly delivering quality to your customers. We will also discuss how we bridge the gap between testing and monitoring, and the relationship with capacity planning, disaster recovery, continuity, and system performance validation.

0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,933
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
82
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

ING Direct: how we do performance testing in agile environments and bridge the gaps between IT teams

  1. 1. ING Direct: How we do performance testing in agile environments and bridge the gaps between IT teams Todd DeCapua Production Quality & Process Group Leader, ING DIRECT 1 ©2010 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice
  2. 2. Show of Hands: Who Is Running Agile? 2
  3. 3. Show of Hands: Do You Ever Worry About This? 3
  4. 4. Show of Hands: Are You a Performance Engineer? 4
  5. 5. Where does performance testing fit within your Agile application lifecycle management process? 5
  6. 6. Philosophical Turning Point: “Flipping The Triangle” • Agile 2008 – Toronto • Session Title: • Flipping the Automated Testing Triangle: The Upshot • Presented by Patrick Wilson Welsh • Focus: • You need three automated testing initiatives, one for each of the three kinds of tests. • These are: Unit, Integration, and GUI. • Where should you be investing your time & money. • Need to flip from the traditional approach. • Philosophical Turning Point For Me: • This applies also to performance testing, as we are proving the architecture & design works, so why not flip the triangle of performance testing initiatives; especially with Agile, as we are running bottom up more. http://patrickwilsonwelsh.com/wp-content/uploads/2008/08/flipping-the-triangle.pdf 6
  7. 7. 7
  8. 8. 8
  9. 9. It Fits Throughout the Process 9
  10. 10. Key Barriers of Agile Performance Testing: People (dedicated performance test engineers) 10
  11. 11. • Embedded Performance in All Teams at All Roles • Unit = Developer Role • Integration = Developer & QA & BA Role • Functional = QA & BA Role 11
  12. 12. Key Barriers of Agile Performance Testing: Non-Functional Acceptance Criteria (included in each story) 12
  13. 13. User Stories: Mike Cohn – Simple Template • “As a <type of user>, I want <some goal> so that <some reason>.” 13
  14. 14. Breakdown of Epic to Tasks with Acceptance Criteria • Epic (A very large idea or goal) • Story (One of many work concepts to achieve an Epic) • Acceptance Test Criteria (Defined objectives to consider a Story done) • Tasks (One of many specific work items to complete a Story) So, how does this apply to Performance Testing within Agile? STORY Front ACCEPTANCE TEST CRITERIA Back “As a Father, I want a vehicle that •Beige exterior with white leather •Must be able to carry 2 adults, and 2 can go forward, left, right, and interior. children at the same time. backward so that I can safely get •Four go at •Mustseats. least 60 mph in any my family places.” •Four tires direction. with a spare. •Must have front & side airbags. •I would like to know max speed with max payload. 14
  15. 15. Feature “Doneness”  The code is included in the proper branch in the source code control system.  The code compiles from a clean checkout without errors using production branch [ proposed: and is part of the AHP Build Life which is finally tested in QA. ]  The code is appropriately covered with unit tests and all tests are passing using the production branch.  The code has been peer reviewed by another developer.  Database changes have been reviewed and approved by a DBA.  The code has passed integration, regression, stress, and load testing.  Application Support is aware of the backlog item and the system impacts.  Deployment and rollback instructions are defined, tested and documented. Until all of these are true, the feature is just unfinished inventory. 15
  16. 16. Key Barriers of Agile Performance Testing: Environment(s) (Enabling proper iterative testing with data and scale) 16
  17. 17. How: Environments and Quality Gates VIRTUAL ENV PHYSICAL ENV DEV DESKTOP BVT DEV IT QA Pre-Prod Prod Development Development QA Integrated QA Staging Env Production Standalone Env for Integrated Env Env Execution Build Verification Execution of Execution of Workflow Tests Verify Smoke Tests Worklflow Tests Regression Tests App Deployment Execution of Performance Tests DB Deployment Regression Tests Release coordination BVT Tests Execution of Functional Tests QUALITY GATES 100% Successful XY% Smoke Tests Pass XY% Regression Tests XY% Regression Tests Deployment (or acceptable Pass Pass 100% Success Build workarounds) XY% Workflow Tests XY% Workflow Tests Verification Tests (or JMeter Pass Pass acceptable workarounds) XY% Functional Tests Acceptable Performance XY% JUNIT Pass Pass XY% Code Coverage LoadRunner JUnitPerf Performance Tests Pass 17
  18. 18. Development Pre Production Production Environment Environment Environment Infrastructure Pre Prod Stand In Build QA Mode Prod (n+1) Technology Live-Live Dev QA Prod (n) •Infrastructure & Technology Build, Dev, QA •Mock Deployment Tests •Customer Experience •Manual & Automated Functional Tests •System Performance Validation •Continuous Availability •Unit / Integration / GUI •Capacity Analysis & Planning •No Deployment or Failover Downtime •Automated Regression Tests •Integrated Release Testing •Production Quality •Performance / Security / Error Tests •Increased Capacity & Performance •Quality Metrics: Daily & Trend Reporting •Safety Net of Stand-In Mode •Quality Gate: Coverage % and Pass % •Ensure Disaster Recovery Capabilities •Governed & Enforced •Live-Live Redundancy •Desktop Lab Environment & Tests •Reduced Risk 18
  19. 19. Key Barriers of Agile Performance Testing: Tools (Simple, quick, and continuous feedback) 19
  20. 20. How: Simple Tool Integration to Support Process Software Delivery Test Management Auto Build, Test & Deploy Business Business / UAT Configuration Management People BA / Dev / QA BA / Dev / QA Continuous Integration Infrastructure Architecture Backlog Management Requirements (sometimes) Build / Merge / Deploy Story Maturity / Estimation Test Plan / Test Lab / Defect Automation Unit Test / Process Impediment Management Automation / Performance Execution Integration Test Sprint Planning Analysis / Reporting Acceptance Criteria by -Axure Tools -JUnitPerf / LoadRunner / -QuickTest Professional -JUnit / NUnit Service Test / HP Diag Profiler -LoadRunner / Diagnostics -FitNesse -Jmeter / LoadRunner / -Shunra ServiceTest 20
  21. 21. Show of Hands: What Do You Use? GUI LoadRunner Jmeter / Service Test Integration LoadRunner JUnitPerf / Unit Service Test / LoadRunner 21
  22. 22. How we bridge the gap between testing, monitoring, and the relationship with Capacity Planning / Disaster Recovery & Continuity / System Performance Validation 22
  23. 23. Monitoring: How It Works Together Production Operational Capacity Planning “Real Time” “Trending Current” Improved Increased Performance Performance Lab Capacity Planning Production Scenarios “Real Time” “Projection Future” Quality 23
  24. 24. Production Quality & Process Dev / QA Pre Prod Prod / DR Continuity / DR Release Management Capacity Planning Development System Quality Performance Validation
  25. 25. Key Learnings: Performance Testing in Agile • QA Environment Management • Quick Refresh • VM Templates • Test Data Management • Root Data Concept • Built Up & Torn Down within Scripts • Flip The Triangle • More on Unit, then Integration, and last on Functional GUI • Embedded Performance in All Teams at All Roles • Unit = Developer Role • Integration = Developer & QA & BA Role • Functional = QA & BA Role • Real-time push & pull reporting • Shift behavior to Quality – Commitment – Continuous Improvement • Establish Framework & Enable Team To Performance / Non-Functional 25
  26. 26. Questions? Todd DeCapua Production Quality & Process Group Leader ING DIRECT 1 S. Orange St., Wilmington, DE, 19801 tdecapua@ingdirect.com ph - 302.255.3537 www.INGDIRECT.com | 1-800 ING DIRECT 26
  27. 27. To learn more on this topic, and to connect with your peers after the conference, visit the HP Software Solutions Community: www.hp.com/go/swcommunity 27 ©2010 Hewlett-Packard Development Company, L.P.
  28. 28. 28
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×