Application Performance Testing: A Simplified Universal Approach
Upcoming SlideShare
Loading in...5

Application Performance Testing: A Simplified Universal Approach



In response to increasing market demand for high performance applications, many organizations implement performance testing projects, often at great expense. Sadly, these solutions alone are often ...

In response to increasing market demand for high performance applications, many organizations implement performance testing projects, often at great expense. Sadly, these solutions alone are often insufficient to keep pace with emerging expectations and competitive pressures. With specific examples from recent client implementations, Scott Barber shares the fundamentals of implementing T4APM™ a simple and universal approach that is valuable independently or as an extension of existing performance testing programs. The T4APM™ approach hinges on applying a simple and unobtrusive "Target, Test, Trend, Tune” cycle to tasks in your application lifecycle—from a single unit test through entire system production monitoring. Leveraging T4APM™ on a particular task may require knowledge specific to the task, but learning how to leverage the approach does not. Scott provides everything you need to become the T4APM™ coach and champion, and to help your team keep up with increasing demand for better performance, regardless of your current title or role.



Total Views
Views on SlideShare
Embed Views



1 Embed 5 5



Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

Application Performance Testing: A Simplified Universal Approach Application Performance Testing: A Simplified Universal Approach Document Transcript

  • MD AM Tutorial 9/30/2013 8:30:00 AM "Application Performance Testing: A Simplified Universal Approach" Presented by: Scott Barber PerfTestPlus, Inc. Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ ∙
  • Scott Barber PerfTestPlus, Inc. CTO of PerfTestPlus and recognized worldwide as a thought-leader in software system performance and testing, Scott Barber describes himself as a “tester, author, speaker, disrupter, and dad.” Scott is the author of Web Load Testing for Dummies and co-/contributing author of Performance Testing Guidance for Web Applications, Beautiful Testing, and How to Reduce the Cost of Testing.
  • 8/27/2013 Managing Application Performance: A Simplified Universal Approach 1 Managing Application Performance: A Simplified Universal Approach Created for: By: Scott Barber Chief Performance Evangelist SmartBear Software, Inc. 1
  • 8/27/2013 Chief Performance Evangelist SmartBear Software, Inc. @sbarber Co-Founder: Workshop On Performance and Reliability Author: Co-Author: Contributing Author: Books: About me: What is Performance? System or application characteristics related to: Speed: - responsiveness - user experience Scalability: - capacity - load - volume Stability: - consistency - reliability - stress 2
  • 8/27/2013 What is a Performant System? A system optimized to balance: - Cost - Time to Market - Capacity …while remaining focused on: Quality of Service to System Users What is Performance Testing? Testing designed to: Inform software system optimization by balancing: - Cost - Time to Market - Capacity while remaining focused on the quality of service to system users. 3
  • 8/27/2013 Performance vs. Load Testing? The Performance Lifecycle is: Conception to Headstone Not Cradle to Grave 4
  • 8/27/2013 Prevent Poor Performance… … don’t just react when it happens. © PerfTestPlus, Inc. Who is Responsible? 5
  • 8/27/2013 © PerfTestPlus, Inc. Where does it fit in the SDLC? 6
  • 8/27/2013 Delivering Performant Systems; Commonly, Y2K -> Today SPE (Software Performance Engineering) Math-centric Architect/Design method + Profiling Context-free code optimization + Load Testing Demonstrate (too late) poor performance characteristics + Capacity Planning Math-centric guessing to determine hardware needs + APM (Application Performance Management) Proactively react to production issues => Inefficient & Reactionary Delivery & Maintenance Delivering Performant Systems; Moving Forward Minimize Need for Complex Mathematics Few teams have mathematicians w/ advanced degrees Disconnect Accuracy from Empirical Production Data Because we can’t depend on what we don’t have yet Remove Socio-Political Barriers Collaboration improves information sharing & efficiency Keep/Adapt most Valuable Practices Many good concepts have been poorly implemented Distribute Valuable Practices Throughout Life Cycle Many valuable practices have suffered from stove-piping I’m testing a model for the entire ‘Performance Journey’ 7
  • 8/27/2013 Performant Application Delivery © PerfTestPlus, Inc. Key Points • • • • Entire Lifecycle Entire Team Focused Individual, Role, & Team Accountability & Responsibility Continuous & Collaborative “DevOps” • • • • • • • Code Profiling Requirements Revision Design Validation Budgets Units & Components Monitoring Tuning Perf Team • • • • • Load/Stress Testing Environment Config Capacity Planning Concurrency Checks Situation Analysis Individual • • • • Snapshots Comparisons Niche Concerns Mini-Field Studies (UAT) • “What-if…” © PerfTestPlus, Inc. 8
  • 8/27/2013 Preventing Poor Performance with a little work… every day… from every one. © PerfTestPlus, Inc. Proactive Micro & Macro Establish Goals Update Targets Dev & Prod Times Resources Sizes Frequencies Dashboard! Units Stories Tiers Resources Goals Assess Compare Investigate Accept Answer © PerfTestPlus, Inc. 9
  • 8/27/2013 © PerfTestPlus, Inc. Unit-Level Testing Tools (‘cause folks always ask) FireBenchmarks; Performance testing addin for NUnit JUnitPerf; a collection of JUnit test decorators for performance Firefox Performance Tester's Pack HTTPerf 10
  • 8/27/2013 © PerfTestPlus, Inc. An Ounce of Prevention… 11
  • 8/27/2013 © PerfTestPlus, Inc. © PerfTestPlus, Inc. 12
  • 8/27/2013 What is it? Load Tests Stress Tests Endurance Tests Infrastructure/Configuration Verification Load Variability Analysis Background Load for Acceptance Tests… Or any other deliberate approximation of Production Conditions © PerfTestPlus, Inc. 13
  • 8/27/2013 RPT is: Inspired by Rapid Software Testing Consistent with Rapid Software Testing themes Sanctioned by James Bach, Michael Bolton & the RST instructors to as a specific implementation of the Rapid Testing Methodology For more information about RST, visit: Evolved from: “What have we got? What do we want? How do we get there…?” --Bob Barber (Scott’s dad) … as quickly, simply, and cheaply as possible? 14
  • 8/27/2013 …a common man’s way of expressing the problem solving approach that classical engineers employ. • Given: “What have we got?” • Find: “What do we want?” • Solve: “How do we get there?” What is it? An approach to respond to a specific performance-related question after 4 or fewer hours of team effort with 1 or more of: A) The answer B) A partial answer • • To determine the value of additional effort The level of effort to provide the answer C) Better questions to address the underlying concern 15
  • 8/27/2013 Conceptual Approach 1. Receive Question • • 2. 3. 4. 5. 6. Clarify the question Understand the driver(s) behind the question Generate Test Coverage Outline (TCO) (~20 min) • Simplest path to (partial) answer(s) • Comprehensive path to (partial) answer(s) Transform TCO into Rapid Strategy (~20 min) • Only tasks that fit in time box • Stick to tasks requiring available resources Execute Strategy (~2.5 hrs) • Snapshots are your friends • Anecdotal is sufficient Consolidate/Analyze Data (~30 min) • Identify patterns • Confirm patterns (time permitting) Report Results (~20 min) • Answer(s) / Time or Effort to get answer(s) • Follow-on questions of interest ~4 hr © PerfTestPlus, Inc. © PerfTestPlus, Inc. 16
  • 8/27/2013 © PerfTestPlus, Inc. The Bottom Line © PerfTestPlus, Inc. 17
  • 8/27/2013 Addendum Questions? 18
  • 8/27/2013 Contact Info Chief Performance Evangelist SmartBear Softwear, Inc. E-mail: Web Site: Blog: Twitter: @sbarber 19