The PAC aims to promote engagement between various experts from around the world, to create relevant, value-added content sharing between members. For Neotys, to strengthen our position as a thought leader in load & performance testing. Since its beginning, the PAC is designed to connect performance experts during a single event. In June, during 24 hours, 20 participants convened exploring several topics on the minds of today’s performance tester such as DevOps, Shift Left/Right, Test Automation, Blockchain and Artificial Intelligence.

1. The Lost Art of Log Analysis
- Stephen Townshend -
Senior Performance Test Consultant
Assurity Consulting (New Zealand)

3. Requests
Events
Timings
Errors

4. Errors ThroughputResponse
Time
Concurrency

5. No APM
tools
Workload
Modelling
Break down
Response
time
Validate
Testing or
Production

6. Client
• OS logs
• Client logs
Web
• Access logs
Application
• Application Logs
• Error Logs
• Event Logs
• Debug Logs
Database
• Traces
• Querying Tables
Cloud
• API logs
• Log analytics

7. Considerations
Standard
VS
Bespoke
Security Cloud

13. datetime http_method url url_query http_port user_id client_ip http_status_code response_time
18/03/2018 19:44 GET /MySystemProd/default.aspx - 80 - 10.11.12.13 401 1248
18/03/2018 19:44 GET /MySystemProd/default.aspx - 80 - 10.11.12.13 401 156
18/03/2018 19:44 GET /MySystemProd/default.aspx - 80 MyDomainmy_user_2 10.11.12.13 200 3400
18/03/2018 19:44 POST /MySystemProd/default.aspx - 80 MyDomainmy_user_2 10.11.12.13 200 109
18/03/2018 19:44 GET /MySystemProd/lang1033/ConsoleIn.aspx - 80 MyDomainmy_user_2 10.11.12.13 200 46
18/03/2018 19:44 GET /MySystemProd/lang1033/ConsoleOut.aspx - 80 MyDomainmy_user_2 10.11.12.13 200 62
18/03/2018 19:44 POST /MySystemProd/lang1033/ConsoleOut.aspx - 80 MyDomainmy_user_2 10.11.12.13 200 31
18/03/2018 19:44 GET /MySystemProd/lang1033/ConsoleIn.aspx - 80 MyDomainmy_user_2 10.11.12.13 200 31
18/03/2018 19:44 POST /MySystemProd/lang1033/ConsoleOut.aspx - 80 MyDomainmy_user_2 10.11.12.13 200 62
18/03/2018 19:44 GET /MySystemProd/BrowserClose.aspx - 80 MyDomainmy_user_2 10.11.12.13 200 78
18/03/2018 20:06 GET /MySystemProd/default.htm - 80 - 192.168.1.100 401 0
18/03/2018 20:06 GET /MySystemProd/default.htm - 80 MyDomainmy_user 192.168.1.100 200 15
18/03/2018 20:06 GET /MySystemProd/default.aspx WelcomeToaMySystemSession 80 - 192.168.1.100 401 0
18/03/2018 20:06 GET /MySystemProd/default.htm - 80 - 192.168.1.100 401 0
18/03/2018 20:06 GET /MySystemProd/default.htm - 80 MyDomainmy_user 192.168.1.100 200 0
18/03/2018 20:06 GET /MySystemProd/default.aspx WelcomeToaMySystemSession 80 - 192.168.1.100 401 15
18/03/2018 20:06 GET /MySystemProd/default.htm - 80 - 192.168.1.100 401 0
18/03/2018 20:06 GET /MySystemProd/default.htm - 80 MyDomainmy_user 192.168.1.100 200 15
18/03/2018 20:06 GET /MySystemProd/default.htm - 80 - 192.168.1.100 401 0
18/03/2018 20:06 GET /MySystemProd/default.aspx WelcomeToaMySystemSession 80 - 192.168.1.100 401 0
18/03/2018 20:06 GET /MySystemProd/default.htm - 80 MyDomainmy_user 192.168.1.100 200 15
18/03/2018 20:06 GET /MySystemProd/default.aspx WelcomeToaMySystemSession 80 - 192.168.1.100 401 0
18/03/2018 20:06 GET /MySystemProd/default.htm - 80 - 192.168.1.100 401 0
18/03/2018 20:06 GET /MySystemProd/default.htm - 80 MyDomainmy_user 192.168.1.100 200 15
18/03/2018 20:06 GET /MySystemProd/default.aspx WelcomeToaMySystemSession 80 - 192.168.1.100 401 0
18/03/2018 20:06 GET /MySystemProd/default.htm - 80 - 192.168.1.100 401 0
18/03/2018 20:06 GET /MySystemProd/default.htm - 80 MyDomainmy_user 192.168.1.100 200 15
18/03/2018 20:06 GET /MySystemProd/default.htm - 80 - 192.168.1.100 401 0
18/03/2018 20:06 GET /MySystemProd/default.aspx WelcomeToaMySystemSession 80 - 192.168.1.100 401 15

15. Averages (or percentiles, any aggregate) can be misleading.
Our tool reports an average response time of just over 3 seconds:
It’s natural to assume the user experience is 3 seconds.

16. Average plotted over time (5 minute interval):

17. Average plotted over time (1 minute interval):

18. Average plotted over time (1 second interval):

19. What’s *actually* happening? 1 or 5, never 3

20. Sample Size Percentiles are not
Perfect
Raw Data & Density
Look from multiple
angles