Scott Barber, chief performance evangelist for SmartBear, presented on managing application performance through a simplified universal approach. The presentation discussed defining performance, the performance testing lifecycle, preventing poor performance through continuous collaboration across the development team, and using rapid performance testing techniques to quickly answer performance-related questions.

1.
Presented by:
Scott
Brought to you by:
340 Corporate Way, Suite Orange Park, FL 32073
888‐2
MH
PM Tutorial
4/7/2014
1:00 PM
“Managing Application Performance:
A Simplified Universal Approach”
Barber
SmartBear
300,
68‐8770 ∙ 904‐278‐0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

2.
hief performance evangelist for SmartBear Scott Barber is a respected leader in the
r
as director of
e
Scott Barber
SmartBear
C
advancement of software testing practices, an industry activist, and load testing
celebrity of sorts. Scott authored several books―Performance Testing Guidance fo
Web Applications, Beautiful Testing, How to Reduce the Cost of Testing, and Web
Load Testing for Dummies―and more than 100 articles and blog posts.
Founder/president of PerfTestPlus, Scott co-founded the WOPR, served
the AST and CMG, and is a founding member of ISST. His industry writing, speaking,
and activism focus on improving the effectiveness and business alignment of softwar
development practices. Learn more about Scott Barber.

3. 2/19/2014
1
Managing Application Performance:
A Simplified Universal Approach
Revised for:
Managing Application Performance:
A Simplified Universal Approach
By:
Scott Barber
Product Owner – Load Testing Tools
SmartBear Software, Inc.

4. 2/19/2014
2
Product Owner – Load Testing Tools
SmartBear Software, Inc.
scott barber@smartbear comscott.barber@smartbear.com
@sbarber
Co‐Founder: Workshop On Performance and Reliability
www.performance‐workshop.org
Co‐Author:Author: Contributing Author:
Books: www.perftestplus.com/pubs
About me: about.me/scott.barber
System or application characteristics related to:
Speed:
responsiveness
What is Performance?
‐ responsiveness
‐ user experience
Scalability:
‐ capacity
‐ load
‐ volume
Stability:
‐ consistency
‐ reliability
‐ stress

5. 2/19/2014
3
What is a Performant System?
A system optimized to balance:
‐ Cost‐ Cost
‐ Time to Market
‐ Capacity
…while remaining focused on:
Quality of Service to System Users
What is Performance Testing?
Testing designed to:
Inform software system optimization byInform software system optimization by
balancing:
‐ Cost
‐ Time to Market
Capacity‐ Capacity
while remaining focused on the quality of
service to system users.

6. 2/19/2014
4
Performance vs. Load Testing?
Conception to Headstone
The Performance LifeCycle is…
Not
Cradle to Grave

7. 2/19/2014
5
Prevent Poor Performance…
© PerfTestPlus, Inc.
… don’t just react when it happens.
Who is Responsible?

8. 2/19/2014
6
© PerfTestPlus, Inc.
Where does it fit in the SDLC?

9. 2/19/2014
7
Delivering Performant Systems;
Commonly, Y2K ‐> Today
SPE (Software Performance Engineering)
Math‐centric Architect/Design method
+ Profiling
Context‐free code optimization
+ Load Testing
Demonstrate (too late) poor performance characteristics
+ Capacity Planning+ Capacity Planning
Math‐centric guessing to determine hardware needs
+ APM (Application Performance Management)
Proactively react to production issues
=> Inefficient & Reactionary Delivery & Maintenance
Delivering Performant Systems;
Moving Forward
Minimize Need for Complex Mathematics
Few teams have mathematicians w/ advanced degreesFew teams have mathematicians w/ advanced degrees
Disconnect Accuracy from Actual Production Data
Because we can’t depend on what we don’t have yet
Remove Socio‐Political Barriers
Collaboration improves information sharing & efficiency
Keep/Adapt most Valuable PracticesKeep/Adapt most Valuable Practices
Many good concepts have been poorly implemented
Distribute Valuable Practices Throughout LifeCycle
Many valuable practices have suffered from stove‐piping
We need one model for the whole ‘Performance Journey’

10. 2/19/2014
8
© PerfTestPlus, Inc.
Key Points
• Entire Lifecycle
• Entire Team Focused
• Individual, Role, & Team Accountability & Responsibility
• Continuous & Collaborative
“DevOps” Perf Team Individual
• Code Profiling
• Requirements Revision
• Load/Stress Testing
• Environment Config
• Snapshots
• Comparisonsq
• Design Validation
• Budgets
• Units & Components
• Monitoring
• Tuning
g
• Capacity Planning
• Concurrency Checks
• Situation Analysis
p
• Niche Concerns
• Mini‐Field Studies
(UAT)
• “What‐if…”
© PerfTestPlus, Inc.

11. 2/19/2014
9
Preventing Poor Performance with
© PerfTestPlus, Inc.
Preventing Poor Performance with
a little work…
every day…
from every one.
Units
Stories
Tiers
Resources
Goals
Proactive
Micro & Macro
Establish Goals
Update Targets
Dev & Prod
© PerfTestPlus, Inc.
Assess
Compare
Investigate
Accept
Answer
Dev & Prod
Times
Resources
Sizes
Frequencies
Dashboard!

12. 2/19/2014
10
© PerfTestPlus, Inc.
© PerfTestPlus, Inc.

13. 2/19/2014
11
© PerfTestPlus, Inc.
Unit‐Level Testing Tools
(‘cause folks always ask)
FireBenchmarks; Performance testing addin for NUnit
JUnitPerf; a collection of JUnit test decorators for performance
Firefox Performance Tester's Pack
HTTPerf

14. 2/19/2014
12
© PerfTestPlus, Inc.
An Ounce of Prevention…

15. 2/19/2014
13
© PerfTestPlus, Inc.
© PerfTestPlus, Inc.

16. 2/19/2014
14
Load Tests
Stress Tests
d
What is it?
Endurance Tests
Infrastructure/Configuration Verification
Load Variability Analysis
Background Load for Acceptance Tests…
Or any other deliberate approximation of
Production Conditions
© PerfTestPlus, Inc.

17. 2/19/2014
15
RPT is:
Inspired by Rapid Software TestingInspired by Rapid Software Testing
Consistent with Rapid Software Testing themes
Sanctioned by James Bach, Michael Bolton & the RST
instructors to as a specific implementation of the
Rapid Testing MethodologyRapid Testing Methodology
For more information about RST, visit:
http://www.satisfice.com/info_rst.shtml
“What have we got?
What do we want?
Evolved from:
What do we want?
How do we get there…?”
‐‐Bob Barber (Scott’s dad)
… as quickly, simply, and … as qu c y, s p y, a d
cheaply as possible?

18. 2/19/2014
16
…a common man’s way of
expressing the problem solving
h h l i l iapproach that classical engineers
employ.
• Given: “What have we got?”
• Find: “What do we want?”
• Solve: “How do we get there?”
An approach to respond to a specific
performance‐related question after 4 or fewer
What is it?
hours of team effort with 1 or more of:
A) The answer
B) A partial answer
• To determine the value of additional effort
Th l l f ff t t id th• The level of effort to provide the answer
C) Better questions to address the underlying
concern

19. 2/19/2014
17
1. Receive Question
• Clarify the question
• Understand the driver(s) behind the question
2. Generate Test Coverage Outline (TCO) (~20 min)
• Simplest path to (partial) answer(s)
Conceptual Approach
• Simplest path to (partial) answer(s)
• Comprehensive path to (partial) answer(s)
3. Transform TCO into Rapid Strategy (~20 min)
• Only tasks that fit in time box
• Stick to tasks requiring available resources
4. Execute Strategy (~2.5 hrs)
• Snapshots are your friends
• Anecdotal is sufficient
~4 hr
5. Consolidate/Analyze Data (~30 min)
• Identify patterns
• Confirm patterns (time permitting)
6. Report Results (~20 min)
• Answer(s) / Time or Effort to get answer(s)
• Follow‐on questions of interest
© PerfTestPlus, Inc.
© PerfTestPlus, Inc.

20. 2/19/2014
18
© PerfTestPlus, Inc.
The Bottom Line
© PerfTestPlus, Inc.

21. 2/19/2014
19
Addendum
Questions?

22. 2/19/2014
20
Contact Info
about me/scott barberabout.me/scott.barber
Product Owner – Load Testing Tools
SmartBear Softwear, Inc.
E-mail:
Scott barber@smartbear com
Web Site:
SmartBear.comScott.barber@smartbear.com
Blog:
scott-barber.blogspot.com
SmartBear.com
Twitter:
@sbarber