MD
AM Tutorial
9/30/2013 8:30:00 AM

"Application Performance
Testing: A Simplified Universal
Approach"
Presented by:
Scot...
Scott Barber
PerfTestPlus, Inc.
CTO of PerfTestPlus and recognized worldwide as a thought-leader in software system
perfor...
8/27/2013

Managing Application Performance:
A Simplified Universal Approach
1

Managing Application Performance:
A Simpli...
8/27/2013

Chief Performance Evangelist
SmartBear Software, Inc.
scott.barber@smartbear.com
@sbarber

Co-Founder: Workshop...
8/27/2013

What is a Performant System?
A system optimized to balance:
- Cost
- Time to Market
- Capacity
…while remaining...
8/27/2013

Performance vs. Load Testing?

The Performance Lifecycle is:
Conception to Headstone

Not

Cradle to Grave

4
8/27/2013

Prevent Poor Performance…

… don’t just react when it happens.
© PerfTestPlus, Inc.

Who is Responsible?

5
8/27/2013

© PerfTestPlus, Inc.

Where does it fit in the SDLC?

6
8/27/2013

Delivering Performant Systems;
Commonly, Y2K -> Today
SPE (Software Performance Engineering)
Math-centric Archi...
8/27/2013

Performant Application Delivery

© PerfTestPlus, Inc.

Key Points
•
•
•
•

Entire Lifecycle
Entire Team Focused...
8/27/2013

Preventing Poor Performance with
a little work…
every day…
from every one.
© PerfTestPlus, Inc.

Proactive
Micr...
8/27/2013

© PerfTestPlus, Inc.

Unit-Level Testing Tools
(‘cause folks always ask)
FireBenchmarks; Performance testing ad...
8/27/2013

© PerfTestPlus, Inc.

An Ounce of Prevention…

11
8/27/2013

© PerfTestPlus, Inc.

© PerfTestPlus, Inc.

12
8/27/2013

What is it?
Load Tests
Stress Tests
Endurance Tests
Infrastructure/Configuration Verification
Load Variability ...
8/27/2013

RPT is:
Inspired by Rapid Software Testing
Consistent with Rapid Software Testing themes
Sanctioned by James Ba...
8/27/2013

…a common man’s way of
expressing the problem solving
approach that classical engineers
employ.
• Given: “What ...
8/27/2013

Conceptual Approach
1. Receive Question
•
•

2.
3.
4.
5.
6.

Clarify the question
Understand the driver(s) behi...
8/27/2013

© PerfTestPlus, Inc.

The Bottom Line

© PerfTestPlus, Inc.

17
8/27/2013

Addendum

Questions?

18
8/27/2013

Contact Info
about.me/scott.barber

Chief Performance Evangelist
SmartBear Softwear, Inc.
E-mail:
Scott.barber@...
Upcoming SlideShare
Loading in …5
×

Application Performance Testing: A Simplified Universal Approach

353 views
256 views

Published on

In response to increasing market demand for high performance applications, many organizations implement performance testing projects, often at great expense. Sadly, these solutions alone are often insufficient to keep pace with emerging expectations and competitive pressures. With specific examples from recent client implementations, Scott Barber shares the fundamentals of implementing T4APM™ a simple and universal approach that is valuable independently or as an extension of existing performance testing programs. The T4APM™ approach hinges on applying a simple and unobtrusive "Target, Test, Trend, Tune” cycle to tasks in your application lifecycle—from a single unit test through entire system production monitoring. Leveraging T4APM™ on a particular task may require knowledge specific to the task, but learning how to leverage the approach does not. Scott provides everything you need to become the T4APM™ coach and champion, and to help your team keep up with increasing demand for better performance, regardless of your current title or role.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
353
On SlideShare
0
From Embeds
0
Number of Embeds
17
Actions
Shares
0
Downloads
8
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Application Performance Testing: A Simplified Universal Approach

  1. 1. MD AM Tutorial 9/30/2013 8:30:00 AM "Application Performance Testing: A Simplified Universal Approach" Presented by: Scott Barber PerfTestPlus, Inc. Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
  2. 2. Scott Barber PerfTestPlus, Inc. CTO of PerfTestPlus and recognized worldwide as a thought-leader in software system performance and testing, Scott Barber describes himself as a “tester, author, speaker, disrupter, and dad.” Scott is the author of Web Load Testing for Dummies and co-/contributing author of Performance Testing Guidance for Web Applications, Beautiful Testing, and How to Reduce the Cost of Testing.
  3. 3. 8/27/2013 Managing Application Performance: A Simplified Universal Approach 1 Managing Application Performance: A Simplified Universal Approach Created for: By: Scott Barber Chief Performance Evangelist SmartBear Software, Inc. 1
  4. 4. 8/27/2013 Chief Performance Evangelist SmartBear Software, Inc. scott.barber@smartbear.com @sbarber Co-Founder: Workshop On Performance and Reliability www.performance-workshop.org Author: Co-Author: Contributing Author: Books: www.perftestplus.com/pubs About me: about.me/scott.barber What is Performance? System or application characteristics related to: Speed: - responsiveness - user experience Scalability: - capacity - load - volume Stability: - consistency - reliability - stress 2
  5. 5. 8/27/2013 What is a Performant System? A system optimized to balance: - Cost - Time to Market - Capacity …while remaining focused on: Quality of Service to System Users What is Performance Testing? Testing designed to: Inform software system optimization by balancing: - Cost - Time to Market - Capacity while remaining focused on the quality of service to system users. 3
  6. 6. 8/27/2013 Performance vs. Load Testing? The Performance Lifecycle is: Conception to Headstone Not Cradle to Grave 4
  7. 7. 8/27/2013 Prevent Poor Performance… … don’t just react when it happens. © PerfTestPlus, Inc. Who is Responsible? 5
  8. 8. 8/27/2013 © PerfTestPlus, Inc. Where does it fit in the SDLC? 6
  9. 9. 8/27/2013 Delivering Performant Systems; Commonly, Y2K -> Today SPE (Software Performance Engineering) Math-centric Architect/Design method + Profiling Context-free code optimization + Load Testing Demonstrate (too late) poor performance characteristics + Capacity Planning Math-centric guessing to determine hardware needs + APM (Application Performance Management) Proactively react to production issues => Inefficient & Reactionary Delivery & Maintenance Delivering Performant Systems; Moving Forward Minimize Need for Complex Mathematics Few teams have mathematicians w/ advanced degrees Disconnect Accuracy from Empirical Production Data Because we can’t depend on what we don’t have yet Remove Socio-Political Barriers Collaboration improves information sharing & efficiency Keep/Adapt most Valuable Practices Many good concepts have been poorly implemented Distribute Valuable Practices Throughout Life Cycle Many valuable practices have suffered from stove-piping I’m testing a model for the entire ‘Performance Journey’ 7
  10. 10. 8/27/2013 Performant Application Delivery © PerfTestPlus, Inc. Key Points • • • • Entire Lifecycle Entire Team Focused Individual, Role, & Team Accountability & Responsibility Continuous & Collaborative “DevOps” • • • • • • • Code Profiling Requirements Revision Design Validation Budgets Units & Components Monitoring Tuning Perf Team • • • • • Load/Stress Testing Environment Config Capacity Planning Concurrency Checks Situation Analysis Individual • • • • Snapshots Comparisons Niche Concerns Mini-Field Studies (UAT) • “What-if…” © PerfTestPlus, Inc. 8
  11. 11. 8/27/2013 Preventing Poor Performance with a little work… every day… from every one. © PerfTestPlus, Inc. Proactive Micro & Macro Establish Goals Update Targets Dev & Prod Times Resources Sizes Frequencies Dashboard! Units Stories Tiers Resources Goals Assess Compare Investigate Accept Answer © PerfTestPlus, Inc. 9
  12. 12. 8/27/2013 © PerfTestPlus, Inc. Unit-Level Testing Tools (‘cause folks always ask) FireBenchmarks; Performance testing addin for NUnit JUnitPerf; a collection of JUnit test decorators for performance Firefox Performance Tester's Pack HTTPerf 10
  13. 13. 8/27/2013 © PerfTestPlus, Inc. An Ounce of Prevention… 11
  14. 14. 8/27/2013 © PerfTestPlus, Inc. © PerfTestPlus, Inc. 12
  15. 15. 8/27/2013 What is it? Load Tests Stress Tests Endurance Tests Infrastructure/Configuration Verification Load Variability Analysis Background Load for Acceptance Tests… Or any other deliberate approximation of Production Conditions © PerfTestPlus, Inc. 13
  16. 16. 8/27/2013 RPT is: Inspired by Rapid Software Testing Consistent with Rapid Software Testing themes Sanctioned by James Bach, Michael Bolton & the RST instructors to as a specific implementation of the Rapid Testing Methodology For more information about RST, visit: http://www.satisfice.com/info_rst.shtml Evolved from: “What have we got? What do we want? How do we get there…?” --Bob Barber (Scott’s dad) … as quickly, simply, and cheaply as possible? 14
  17. 17. 8/27/2013 …a common man’s way of expressing the problem solving approach that classical engineers employ. • Given: “What have we got?” • Find: “What do we want?” • Solve: “How do we get there?” What is it? An approach to respond to a specific performance-related question after 4 or fewer hours of team effort with 1 or more of: A) The answer B) A partial answer • • To determine the value of additional effort The level of effort to provide the answer C) Better questions to address the underlying concern 15
  18. 18. 8/27/2013 Conceptual Approach 1. Receive Question • • 2. 3. 4. 5. 6. Clarify the question Understand the driver(s) behind the question Generate Test Coverage Outline (TCO) (~20 min) • Simplest path to (partial) answer(s) • Comprehensive path to (partial) answer(s) Transform TCO into Rapid Strategy (~20 min) • Only tasks that fit in time box • Stick to tasks requiring available resources Execute Strategy (~2.5 hrs) • Snapshots are your friends • Anecdotal is sufficient Consolidate/Analyze Data (~30 min) • Identify patterns • Confirm patterns (time permitting) Report Results (~20 min) • Answer(s) / Time or Effort to get answer(s) • Follow-on questions of interest ~4 hr © PerfTestPlus, Inc. © PerfTestPlus, Inc. 16
  19. 19. 8/27/2013 © PerfTestPlus, Inc. The Bottom Line © PerfTestPlus, Inc. 17
  20. 20. 8/27/2013 Addendum Questions? 18
  21. 21. 8/27/2013 Contact Info about.me/scott.barber Chief Performance Evangelist SmartBear Softwear, Inc. E-mail: Scott.barber@smartbear.com Web Site: SmartBear.com Blog: scott-barber.blogspot.com Twitter: @sbarber 19

×