Performance Testing in the Agile Enterprise

Performance Testing in the Agile Enterprise by: Scott Barber Chief Technologist, PerfTestPlus, Inc. sbarber@perftestplus.com www.perftestplus.com Co-Founder: Workshop On Performance and Reliability www.performance-workshop.org Author: Co-Author: Contributing Author:Books: www.perftestplus.com/pubs About me: about.me/scott.barber www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 1

Have you ever wondered… teams have no idea where performance “fits” in an Agile development lifecycle? www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 2

Overview Performance Testing is inherently Agile, but... Integrating Performance Testing into Agile Efforts effectively is not inherently easy! www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 5

Agility in Performance TestingPerformance Testingis a repeatingcycle of: Complicated by unknowns, estimations, & approximations www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 6

Agility in Software DevelopmentAgile is arepeatingcycle of: Gated by inexact and variable notions of acceptance www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 7

Conceptual Models“On Demand” • A.K.A. Center of Excellence • Call ‘em when you (think you) need ‘em • “Pay” only for what you (think you) need“On Retainer” • Performance Tester on team, but mostly independent • Always available“Full Immersion” • Performance is everyone’s job • Coordinated by embedded Performance Test Specialist • Spans entire lifecycle www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 9

On DemandRoles & Interactions: • Similar to outsourcing except that… • you expect familiarity with the project/product (unlikely) • you expect familiarity with the process (less likely) • you won’t get 100% focus on your project • the assigned resource(s) also report to someone who can trump the project manager • Minimal interaction with team • In general, they collect info, run tests, & report results.Strengths: • Fewer performance testers can service more projects • Pooled tools and environments can save time and money • Performance testers benefit from peer collaboration www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 10

On DemandChallenges: • Service tends to be “generic” • Skills/tools/environments may not work for *your* project • Availability often restricted to highest priority projects • Performance testing tends to be both late and infrequent • Validation of tuning often has to wait until the next “on demand cycle” • No continuity release to release • A core team member must serve as performance lead • Core team responsible for performance exploration∆ from non-Agile efforts: • Difficult to schedule • System under test a faster moving target • Testing can’t keep up with continuous production delivery www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 12

On RetainerRoles & Interactions: • Performance testing specialist • is embedded into the team full time • may be managed by Project Manager, Product Manager, or Sr. Engineer • Performance is not (yet) integral to project team cultureStrengths: • Performance testing specialist • knows project/product/culture/environment details • customizes service to those details • builds relationships that enhance “performance culture” • Always available • Many more tests conducted than On Demand • Performance exploration and tuning validation “in scope” www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 13

On RetainerChallenges: • Performance tester(s), tool(s), and environment(s) needed for every project • Performance testers • Are often seen as “outsiders” • Tend to loose peer collaboration • Might not be fully utilized • Performance assessments still tend to occur at end of sprints • No additional support during “crunch time”∆ from non-Agile efforts: • Planning is necessarily more reactive • Tool and environment sharing virtually impossible • Testing done in smaller segments www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 15

Full ImmersionRoles & Interactions: • Performance specialist coordinates performance related activities including: • Creation of performance stories • Code profiling and performance unit testing • Performance assessment • Collaboration with production support/capacity planning • Performance is part of everyone’s job, every dayStrengths: • Performance is, or is becoming, an integral part of the team culture • No delays, performance testing is done in line with dev • Cross-functional & cross-departmental collaboration • Best chance of no performance nightmares www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 16

Performance Test Cycle; Full Immersion See role and task details in additional resource slides included at the end of this deck www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 17

Full ImmersionChallenges: • Performance tester(s), tool(s), and environment(s) needed for every project • Organizational culture shift typically required • Demands an “architect level” performance specialist • Getting team dynamics “right” takes time – often 2-3 projects∆ from non-Agile efforts: • Performance testing at many application levels, daily • Root cause analysis dramatically easier • Tuning is continual • Unity of purpose and goals is essential • The only “external” gates are acceptance testing and executive business decisions www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 18

First StepsInvolve management in evolving performance cultureGet performance into user stories • Response times • Resource allocationsInvolve developers • Code profiling • Integrate performance measurement into unit testsMake performance part of story acceptance • Quantitative • QualitativeInvolve performance testers • Throughout the cycle • Gain respect as advisors as well as testers www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 19

Valuable ResourcesPerformance Testing Guidance for Web Applications • Microsoft patterns&practices 2009 • *especially* Chapters 1-7 • Free from http://perftestingguide.codeplex.com/How to Reduce the Cost of Testing • CRC Press, 2011 • Chapter 16 (free from PerfTestPlus.com) • Order book on AmazonOther Articles and Websites • An Explanation of Performance Testing on an Agile Team (part 1, part 2) • What is performance testing? • Software performance testing: There is no I in team www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 20

Contact Info Scott Barber Chief Technologist PerfTestPlus, Inc E-mail: Web Site: sbarber@perftestplus.com www.PerfTestPlus.com www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 22

Roles & Tasks (Reference) Oversee, Review and/or Approve• Executives set the tone• Articulate performance goals and risks to thebusiness• The rest of the team must • Achieve goals • Mitigate risks • Assess & report both -- continually www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 23

Roles & Tasks (Reference) Observe, Cross- Build Capacity Architect & Supervise, Advise, Validate n+2 Design n+2 Troubleshoot, Model for n+2 Model n+2 Troubleshoot, Validate and/ Capacity Validate and/ or Update or Update Predictions, Architecture Architecture Model Architecture Model Model Predictions, Perf Test Results in Staging• Design to achieve goals & mitigate risk• Establish assessment targets, goals & budgets• Set & enforce sound performance practices• Troubleshoot, tune, update design & refactor www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 24

Roles & Tasks (Reference) Reverse Cross- Provide Current Cross- Validate Baseline n+1 Build Capacity Validate Production Info & Validate n+2 in Prod Model for n+2 Models, Consult n+2 Release Capacity Readiness Perf When Needed Capacity Predictions, Tests in Staging Predictions Architecture Model in Prod Predictions, Perf Test Results in• Be “hardware architects” Staging• Design to achieve goals & mitigate risk• Establish assessment targets, goals & budgets• Set & enforce sound performance practices• Provide production feedback• Troubleshoot, tune, update design & refactor www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 25

Roles & Tasks (Reference) Develop Stories; Add Profile; Unit Design n+2 Performance & Level Profiling Stories Performance Testing• Use coding practices that support goodperformance• Code to achieve goals, targets, budgets &mitigate risk• Assess components continually• Troubleshoot, tune, & refactor www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 26

Roles & Tasks (Reference) Update Project Observe, Project System ProdReverse Cross- Performance Consult and Investigative Investigative Validate Readiness Cross- Release Strategy; Assist Performance Performance Perf TestingReadiness Perf Design/ Where Testing Testing Validate n+2 (System)Tests in Staging Develop Appropriate Capacity Investigative Predictions, Performance Architecture Tests Model Predictions, Perf Test Results in• Manage whole system assessment Staging• Own simulation (usage) model(s)• Contribute to component assessments• Cross-validate across environments www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 27

Performance Testing in the Agile Enterprise

by Scott Barber on Nov 04, 2011

Accessibility

Categories

Upload Details

Usage Rights

3 Embeds 19

Statistics

Performance Testing in the Agile Enterprise Presentation Transcript

http://a0.twimg.com	9
http://paper.li	6
http://www.linkedin.com	4