Performance Testing: How to build a Successful Program

1. Building a Performance A-Team

2. © 2015 SOASTA. All rights reserved. June 8, 2015 2CONFIDENTIAL – Not for Distribution
Presenters
Howard Chorney
Technical Director
James Pulley
Chief Geek, LiteSquare
Moderator: Kathy Lam
Product Marketing

3. It’s All About the Benjamin's!!!!!!!
Why does it matter?

4. © 2014 SOASTA. All rights reserved. June 8, 2015 4CONFIDENTIAL – Not for Distribution
All I want is to add
my stuff to my cart.
Slow Site = Unhappy Customer = Bye Bye
Is it ever going
to load?!?

5. © 2014 SOASTA. All rights reserved. June 8, 2015 5CONFIDENTIAL – Not for Distribution
Assumptions
You Understand:
• Industry Standard Automated Performance Testing Tools
• How to Write a Performance Requirements Document
• Test Planning
• Basic Reporting and Analysis

6. © 2014 SOASTA. All rights reserved. June 8, 2015 6CONFIDENTIAL – Not for Distribution
Agenda
• Today’s Performance Testing Challenges
• Step One “Team”
• Step Two “Key Requirements”
• Step Three “ Four Levels of Performance Testing
• Step Four “ Key Performance Indicators”
• Q & A

7. Software
Network
Hardware
The First Challenge of Testing for Website
Performance

8. © 2014 SOASTA. All rights reserved. June 8, 2015 8CONFIDENTIAL – Not for Distribution
It’s All of Us Against the Beastie
• The Business
• Development
• Performance
Engineering
• Ops

9. © 2014 SOASTA. All rights reserved. June 8, 2015 9CONFIDENTIAL – Not for Distribution
Use What You Can Control to Optimize for What
You Cannot Control

10. © 2014 SOASTA. All rights reserved. June 8, 2015 10CONFIDENTIAL – Not for Distribution
Hardware
o Servers
o Storage
o CPU
o Memory
o NW Layer One
Internal - Control

11. © 2014 SOASTA. All rights reserved. June 8, 2015 11CONFIDENTIAL – Not for Distribution
Software
o OS
o eCommerce Suite
• Webserver
• App-server
• Database
• Search Engine
• Ordering
• Order Fulfillment
o Service Bus
o Third Party Callouts
o Content Management
o Consumer Access
• Web
• Mobile Web
• Native Mobile
Internal - Control

12. © 2014 SOASTA. All rights reserved. June 8, 2015 12CONFIDENTIAL – Not for Distribution
Network•
Internal
-
Control
External
-
No Control

13. © 2014 SOASTA. All rights reserved. June 8, 2015 13CONFIDENTIAL – Not for Distribution
Step One:
o Performance Architect
• Strategic Vision
• Technical Communications
• Technical Leadership
• Business Leadership
• All Technical Teams
o Sr. Performance Engineer
• Performance Analytics
• Lead Test Development
o Performance Engineer
• Coding and Testing

14. © 2014 SOASTA. All rights reserved. June 8, 2015 14CONFIDENTIAL – Not for Distribution
Tools 5-
15%
Foundation
Skills
85-95%

15. © 2014 SOASTA. All rights reserved. June 8, 2015 15CONFIDENTIAL – Not for Distribution
o Test Acumen
o Architecture
o Development
o Requirements
o Communications
o Project Management
Platform Expertise
(Database/Web/App Server/
…)
Systems Analysis

16. © 2014 SOASTA. All rights reserved. June 8, 2015 16CONFIDENTIAL – Not for Distribution
Put it all together and it spells - YIKES!!!!!!

17. Step Two: Gathering Three Key Requirements
• SLAs
• Business Flows
• Transaction Volumes

18. Step Three: Four Levels of Performance Testing
Four Level Test Approach

19. © 2014 SOASTA. All rights reserved. June 8, 2015 19CONFIDENTIAL – Not for Distribution
Level One: Coding for Performance
o Performance Testing Can Take the Lead
• Define best practices and introduce to your development group
• Plug yourself in.
• Write a paper on it.
o Champion Code Review Rules
Engine
• SONAR
• 88 Performance Based Rules
o Go For The Ounce Of Prevention!!!!

20. © 2014 SOASTA. All rights reserved. June 8, 2015 20CONFIDENTIAL – Not for Distribution
o If Your Test Tool Can’t Handle It, Get One That Can
Level Two: Attack the Service Level

21. © 2014 SOASTA. All rights reserved. June 8, 2015 21CONFIDENTIAL – Not for Distribution
o Sample Business Flow
• Navigate to site
• Browse for item
• Select item
• Read product description page
• Add to cart
• Complete checkout
o Virtualize Outside Calls
• LISA
Level Three: Logical End to End Business Process

22. © 2014 SOASTA. All rights reserved. June 8, 2015 22CONFIDENTIAL – Not for Distribution
o Execute all Business Process in Parallel
o Test Under Peak Traffic
• Black Friday
• Cyber Monday
o Deploy Funneling
o Execute With All User Types
• Web
• Mobile Web
• Native Mobie
o Add API with the Business Process Flows
o Virtualize Third Party Callouts include Delays
Level Four: Full Site Test Execution

23. © 2014 SOASTA. All rights reserved. June 8, 2015 23CONFIDENTIAL – Not for Distribution
No Better Place
To Ensure Your
Site Will Scale.
GO FOR IT!!!
BUT
Be Prepared!

24. © 2014 SOASTA. All rights reserved. June 8, 2015 24CONFIDENTIAL – Not for Distribution
o Product Health
o Project Health
o Process Health
o Conversion Rate
o Production Stats
Level Five: Monitoring KPI’s

25. © 2014 SOASTA. All rights reserved. June 8, 2015 25CONFIDENTIAL – Not for Distribution
Fast Site = Happy Customer = Higher Revenue
$$ CHA-CHING $$
Your order
Confirmation
is 0U812

26. © 2014 SOASTA. All rights reserved. June 8, 2015 26CONFIDENTIAL – Not for Distribution
? Questions ?