This document summarizes a presentation on performance testing. It discusses why performance testing is important, including predicting system behavior under load, verifying configurations, and discovering concurrency issues. Everyday use cases like marketing campaigns, events, and performance tuning are provided. Benefits for customers, businesses, and maintainers are outlined. The presentation then covers how performance testing is done, including using tools to record and run tests and analyze results, as well as following a methodology to identify weak spots in infrastructure, applications, and more. Finally, an example workshop walkthrough shows how to plan and conduct a performance test on a university assignment system.

1. by Jan Verner
18.5.2016 CzechTest 2016
PERFORMANCE TESTING
DONE BETTER AND
FASTER

2. “Application performance is an integral part
of user experience. Take responsibility for
it.”

3. Agenda
Introductory presentation
Short break
Workshop
Installing the tool
Preparing test scripts
Coffee break
Tests execution
Result analysis

4. Why do
performance
testing
What performance tests
offer
Everyday use cases
Benefits

5. Predict System Behaviour Under Load
To be able to simulate
and measure system’s
behaviour under load.
Anticipate the future.

6. Verify Configuration Settings
Super-performing
environments can be
limited by bad
configurations.
Verify and fix
configurations.

7. Discover Concurrency Issues
Even small load with
unlucky timing can
cause great troubles.
Discover concurrency
issues.

8. Everyday Use Cases
Marketing campaigns (newsletters, TV, AdWords, ..)
Events (sales, sports, ..)
Trends (mobile, virtualization)
Changes in configuration
Performance tuning
Parallelism errors (multithreading)

9. Everyday Use Cases
Marketing campaigns (newsletters, TV, AdWords, ..)
Events (sales, sports, ..)
Trends (mobile, virtualization)
Changes in configuration
Performance tuning
Parallelism errors (multithreading)

10. Everyday Use Cases
Marketing campaigns (newsletters, TV, AdWords, ..)
Events (sales, sports, ..)
Trends (mobile, virtualization)
Changes in configuration
Performance tuning
Parallelism errors (multithreading)

11. Benefits
For customers
• Improved user experience (latency, availability)

12. Benefits
For customers
• Improved user experience (latency, availability)
For business owners
• Planning (expansion, campaigns)
• Keeping SLAs
• Managing suppliers

13. Benefits
For customers
• Improved user experience (latency, availability)
For business owners
• Planning (expansion, campaigns)
• Keeping SLAs
• Managing suppliers
For maintainers
• Knowing of system limits
• Calm nights and weekends

14. Predict System Behaviour Under Load
Verify Configuration Settings
Discover Concurrency Issues
Everyday Use Cases
Benefits
Q&A
Summary “Why”

15. How do
performance
testing
Tools
Methodology

16. Tool
Record & edit test scenarios
Easy to use
Accurate
Distributed
Run tests
Analyze results
Create reports
Archive results

17. Distributed Load

18. Methodology
Workshop focus

19. Weak Spots
Infrastructure Configuration
Applications

20. Performance test - verify criteria
Stress test - discover system limits
Soak test - resource management
Failover test - verify configurations
Volume test - test for future
Finding Weak Spots

21. Cooperation!
Tests result (data) → Test report (information)
Need to understand the tested system
Log analysis
Monitoring
Pinpointing Weak Spots

22. Tools
Methodology
Weak Spots
Finding Weak Spots
Q&A
Summary “How”

23. Workshop
Walkthrough
Story
Applying the methodology
Assignment

24. Story - The Smart University
Assignments for students
● questions published at 8pm
● submit answers in 24 hours

25. Assignment Details
500 students bit.ly/workshop-demo Contact Mr. John Doe
Design and execute performance tests that will verify
University’s readiness.

26. Identify Test Environment
Tested system
• URL bit.ly/workshop-demo
• Architecture - Single server, Linux based, Apache Tomcat,
MySQL, proprietary CMS built on Java platform
Tool
• SmartMeter.io (www.smartmeter.io)
Test environment
• Your laptop, Wi-Fi connectivity

27. Identify Acceptance Criteria
Assignment download
• Throughput - 200 downloads in one minute
• Response time - 90% percentile under 5 secs
Assignment submit
• Throughput - 100 submits in one minute
• Response time - 90% percentile under 3 secs

28. Plan and Design Tests
Identify key scenarios
• homepage → assignment page → download assignment
• homepage → submit page → submit assignment
Determine variability
• students names
• CSRF token in submit form
Test data
• students names (CSV)

29. Configure Test Environment
Prepare test environment
Get familiar with resource monitoring (CPU, RAM, ..)
Install SmartMeter.io tool (requires 64-bit OS)
Make sure you can reach the tested system

30. Implement and Verify Tests
Record test scenarios
Gather test data
Modify test scenarios (variability)
Run smoke tests
Set up load profiles
Be sure you are ready to go!

31. Execute Tests
Be vigilant!
Monitor system resources
Communicate in real time with system owners
Execute various test types (stress, soak, ..)

32. Analyze, Report, and Retest
Gather necessary data (test results, logs, monitoring data, ..)
Make notes
Transform data into meaningful report
Retest

33. Identify Acceptance criteria
Prepare test environment
Create test scripts
Execute tests
Reporting
Q&A
Summary “Workshop Walkthrough”

34. Presentation
Summary
We learnt “why”
Areas, use cases, benefits
We learnt “how”
Tool & Methodology
Workshop Walkthrough

35. Get Ready for Workshop
● After break ..
○ Step by step solution
○ Best practices
○ Hands on tool and
methodology
● Get your laptops ready!

36. Thank you for
your attention!
Share your feedback
bit.ly/performance-tutorial
Jan Verner
Product Developer
Performance tester
jan.verner@smartmeter.io
@SmartMeterIO
#SmartMeterIO