Выступление Игоря Колосова (Lead Performance Engineer, GlobalLogic) на QA Evening от Клуба Большого Мозга

2. 2
Performance Testing:
Metrics & Measurements
Igor Kolosov

3. Who am I?
Igor Kolosov
Lead Performance Engineer
• 8 years in Software Testing
• 5 years in Test Automation & Performance
Engineering
• Wide experience – from solo projects to
team leadership
Skype: h.chlor
E-Mail: h.chlor@gmail.com

4. Screamer time!

5. 5
What is Metric?
Metrics can be defined as “standards of
measurement
Metric is a unit used for describing or
measuring an attribute

6. Main input sources...
• Performance testing tools
• Monitoring tools
• APMs
• Server logs
• Client logs
• System logs
• Custom events

7. Key characteristics
• Responsiveness
• Throughput
• Scalability
• Volumes
• Resource utilization
- Processor utilization
- Memory utilization
- Disk input/output (I/O)
- Network input/output (I/O)

8. 8
Key calculation types
• Peak values
• Averages
• Percentiles
• Items/actions per time unit
• Percentages
• Counts
• Deviations
• Time spent
• Evaluations against fixed criteria
• Relations of two dynamic parameters

9. Example: 333three-tier system

10. Requirements phase: server usage stats
• Total Page Views per Week
• Total Hits per Week
• Total User Sessions per Week
• Average Page Size
• Average Hit Size
• Page Request Distribution
• User Abandonment

11. Requirements phase: client usage stats
• Interaction Speed
• Latency Tolerance
• Familiarity
• Connection speed
• Location

12. 12
General stats & layer interaction
• Resource utilization
• Time spent between tiers
• Requests waiting between tiers
• Throughput
• Connection pool
• Aquisition time

13. Client
• Images Load Time
• JavaScript Files Load Time
• CSS Files Load Time
• HTML Resources Load Time
• DNS Lookup Time
• TCP Connection Time
• SSL Handshake Time
• Time to First Byte
• Content Download Time
• Content Caching Policy
• Availability Content Delivery Network (CDN)
• Resources Rendering Time
• Number of HTTP Requests
• HTTP Response Status
• HTTP Response Size

14. Web server
• Bytes Sent per Second
• Bytes Received per Second
• Get Requests per Second
• Post Requests per Second
• Maximum Connections
• Not Found (404) Error per Second
• Private Bytes
• Percent of Processor Time
• System Processor Queue Length
• Percent Disk Time
• Cache Hit Percent
• Request Wait Time
• Request Queued
• Transactions per Second

15. 15
App server
•Active Threads
•Total Threads
•Percent Time Maxed
•Connection Wait Time
•Connection Time
•Connection Percent Used
•Active Transactions
•Transactions Suspended
•Rolled Back
•Timeouts
•Servlet Errors
•Sessions Invalidated

16. Database
• Slow queries
• Total queries
• Frequent queries
• Prepared vs non-prepared statements
• Transactions per Second
• Log Cache Hit Ratio
• Log Cache Reads per Second
• Log Bytes Flushed per Second
• Log Flush Wait Time
• Log Flush Waits per Second
• Log Flushes per Second
• Percent Log Used

17. Networking
• Bytes per Second Sent to Network Interface Card
• Bytes per Second Received by NIC
• TCP Segments Sent per Second
• TCP Segments Received per Second
• TCP Segments per Second
• TCP Segments Retransmitted per Second
• TCP Connections Failures
• TCP Connections Reset
• TCP Connections Established

18. 18
Example: APM

19. ?

20. 20
Thank you!