Your SlideShare is downloading. ×
Performance Testing in the Agile Enterprise
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Performance Testing in the Agile Enterprise

1,871
views

Published on

According to the presenter, performance testing is an inherently agile process, even though many teams and organizations experience significant difficulty integrating performance into their existing …

According to the presenter, performance testing is an inherently agile process, even though many teams and organizations experience significant difficulty integrating performance into their existing agile processes. In this session, learn how performance can be seamlessly integrated and how to employ and manage performance testing on Agile projects and within Agile enterprises. After this session, students will have the tools and knowledge they need to overcome many of the common difficulties organizations face when trying to make performance an integral part of their Agile processes.

Published in: Technology

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,871
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
130
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Performance Testing in the Agile Enterprise by: Scott Barber Chief Technologist, PerfTestPlus, Inc. sbarber@perftestplus.com www.perftestplus.com Co-Founder: Workshop On Performance and Reliability www.performance-workshop.org Author: Co-Author: Contributing Author:Books: www.perftestplus.com/pubs About me: about.me/scott.barber www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 1
  • 2. Have you ever wondered… teams have no idea where performance “fits” in an Agile development lifecycle? www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 2
  • 3. Because performance “fit” in an Agile development lifecycle… www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 3
  • 4. Performance is (or should be) in an Agile development lifecycle! www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 4
  • 5. Overview Performance Testing is inherently Agile, but... Integrating Performance Testing into Agile Efforts effectively is not inherently easy! www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 5
  • 6. Agility in Performance TestingPerformance Testingis a repeatingcycle of: Complicated by unknowns, estimations, & approximations www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 6
  • 7. Agility in Software DevelopmentAgile is arepeatingcycle of: Gated by inexact and variable notions of acceptance www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 7
  • 8. Agile Perf Testing in Agile DevPutting them togetheris… …a tad complicated www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 8
  • 9. Conceptual Models“On Demand” • A.K.A. Center of Excellence • Call ‘em when you (think you) need ‘em • “Pay” only for what you (think you) need“On Retainer” • Performance Tester on team, but mostly independent • Always available“Full Immersion” • Performance is everyone’s job • Coordinated by embedded Performance Test Specialist • Spans entire lifecycle www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 9
  • 10. On DemandRoles & Interactions: • Similar to outsourcing except that… • you expect familiarity with the project/product (unlikely) • you expect familiarity with the process (less likely) • you won’t get 100% focus on your project • the assigned resource(s) also report to someone who can trump the project manager • Minimal interaction with team • In general, they collect info, run tests, & report results.Strengths: • Fewer performance testers can service more projects • Pooled tools and environments can save time and money • Performance testers benefit from peer collaboration www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 10
  • 11. Performance Test Cycle; On Demand www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 11
  • 12. On DemandChallenges: • Service tends to be “generic” • Skills/tools/environments may not work for *your* project • Availability often restricted to highest priority projects • Performance testing tends to be both late and infrequent • Validation of tuning often has to wait until the next “on demand cycle” • No continuity release to release • A core team member must serve as performance lead • Core team responsible for performance exploration∆ from non-Agile efforts: • Difficult to schedule • System under test a faster moving target • Testing can’t keep up with continuous production delivery www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 12
  • 13. On RetainerRoles & Interactions: • Performance testing specialist • is embedded into the team full time • may be managed by Project Manager, Product Manager, or Sr. Engineer • Performance is not (yet) integral to project team cultureStrengths: • Performance testing specialist • knows project/product/culture/environment details • customizes service to those details • builds relationships that enhance “performance culture” • Always available • Many more tests conducted than On Demand • Performance exploration and tuning validation “in scope” www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 13
  • 14. Performance Test Cycle; On Retainer www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 14
  • 15. On RetainerChallenges: • Performance tester(s), tool(s), and environment(s) needed for every project • Performance testers • Are often seen as “outsiders” • Tend to loose peer collaboration • Might not be fully utilized • Performance assessments still tend to occur at end of sprints • No additional support during “crunch time”∆ from non-Agile efforts: • Planning is necessarily more reactive • Tool and environment sharing virtually impossible • Testing done in smaller segments www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 15
  • 16. Full ImmersionRoles & Interactions: • Performance specialist coordinates performance related activities including: • Creation of performance stories • Code profiling and performance unit testing • Performance assessment • Collaboration with production support/capacity planning • Performance is part of everyone’s job, every dayStrengths: • Performance is, or is becoming, an integral part of the team culture • No delays, performance testing is done in line with dev • Cross-functional & cross-departmental collaboration • Best chance of no performance nightmares www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 16
  • 17. Performance Test Cycle; Full Immersion See role and task details in additional resource slides included at the end of this deck www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 17
  • 18. Full ImmersionChallenges: • Performance tester(s), tool(s), and environment(s) needed for every project • Organizational culture shift typically required • Demands an “architect level” performance specialist • Getting team dynamics “right” takes time – often 2-3 projects∆ from non-Agile efforts: • Performance testing at many application levels, daily • Root cause analysis dramatically easier • Tuning is continual • Unity of purpose and goals is essential • The only “external” gates are acceptance testing and executive business decisions www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 18
  • 19. First StepsInvolve management in evolving performance cultureGet performance into user stories • Response times • Resource allocationsInvolve developers • Code profiling • Integrate performance measurement into unit testsMake performance part of story acceptance • Quantitative • QualitativeInvolve performance testers • Throughout the cycle • Gain respect as advisors as well as testers www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 19
  • 20. Valuable ResourcesPerformance Testing Guidance for Web Applications • Microsoft patterns&practices 2009 • *especially* Chapters 1-7 • Free from http://perftestingguide.codeplex.com/How to Reduce the Cost of Testing • CRC Press, 2011 • Chapter 16 (free from PerfTestPlus.com) • Order book on AmazonOther Articles and Websites • An Explanation of Performance Testing on an Agile Team (part 1, part 2) • What is performance testing? • Software performance testing: There is no I in team www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 20
  • 21. Questions www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 21
  • 22. Contact Info Scott Barber Chief Technologist PerfTestPlus, Inc E-mail: Web Site: sbarber@perftestplus.com www.PerfTestPlus.com www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 22
  • 23. Roles & Tasks (Reference) Oversee, Review and/or Approve• Executives set the tone• Articulate performance goals and risks to thebusiness• The rest of the team must • Achieve goals • Mitigate risks • Assess & report both -- continually www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 23
  • 24. Roles & Tasks (Reference) Observe, Cross- Build Capacity Architect & Supervise, Advise, Validate n+2 Design n+2 Troubleshoot, Model for n+2 Model n+2 Troubleshoot, Validate and/ Capacity Validate and/ or Update or Update Predictions, Architecture Architecture Model Architecture Model Model Predictions, Perf Test Results in Staging• Design to achieve goals & mitigate risk• Establish assessment targets, goals & budgets• Set & enforce sound performance practices• Troubleshoot, tune, update design & refactor www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 24
  • 25. Roles & Tasks (Reference) Reverse Cross- Provide Current Cross- Validate Baseline n+1 Build Capacity Validate Production Info & Validate n+2 in Prod Model for n+2 Models, Consult n+2 Release Capacity Readiness Perf When Needed Capacity Predictions, Tests in Staging Predictions Architecture Model in Prod Predictions, Perf Test Results in• Be “hardware architects” Staging• Design to achieve goals & mitigate risk• Establish assessment targets, goals & budgets• Set & enforce sound performance practices• Provide production feedback• Troubleshoot, tune, update design & refactor www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 25
  • 26. Roles & Tasks (Reference) Develop Stories; Add Profile; Unit Design n+2 Performance & Level Profiling Stories Performance Testing• Use coding practices that support goodperformance• Code to achieve goals, targets, budgets &mitigate risk• Assess components continually• Troubleshoot, tune, & refactor www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 26
  • 27. Roles & Tasks (Reference) Update Project Observe, Project System ProdReverse Cross- Performance Consult and Investigative Investigative Validate Readiness Cross- Release Strategy; Assist Performance Performance Perf TestingReadiness Perf Design/ Where Testing Testing Validate n+2 (System)Tests in Staging Develop Appropriate Capacity Investigative Predictions, Performance Architecture Tests Model Predictions, Perf Test Results in• Manage whole system assessment Staging• Own simulation (usage) model(s)• Contribute to component assessments• Cross-validate across environments www.PerfTestPlus.com© 2006-11 PerfTestPlus, Inc. All rights reserved. Performance Testing in Agile Page 27

×