The document discusses technical testing of trading systems. It outlines the key approaches to performance, stability, and operability testing. The main tasks involved are using test tools to emulate real loads and conditions, preparing test environments and load shapes based on production data, executing automated tests, and thoroughly analyzing results to validate findings. Technical testing helps validate systems' non-functional requirements around performance, stability, and handling high volumes of data and users.

1. Technical Testing Alexey Zverev EXTENT February 2011

2. Contents Technical Testing Our world of trading systems Approach – Performance Testing Approach – Stability Testing Approach – Operability Testing Approach – Debugging Tasks – Test Tools Tasks – Test Preparation/Load Shape Tasks – Test Execution Tasks – Result analysis

3. Technical Testing Technical Testing or NFT Validation of Non-functional Requirements Performance Stability Operability These requirements are usually applied to complex distributed server applications Supports many interactive users Processing high volume of data coming from and disseminated to several sources Example: Web Sites or Web Services Our world – Trading Systems, Exchanges, SOR, DMS, Brokerage, Trading Desks

4. Our world of Trading Systems

5. The Approach to Performance Testing Load Test Emulate load Measure KPIs (Latency, Throughput, Capacity) Determine the load, under which the KPIs become worse than required Stress Test Emulate extreme load Monitor system behaviour Ensure that we sustain it Soak Test (Long Running Test) Emulate load for a long time Check whether we have any deterioration

6. Approach Stability Testing Apply Load kill unplug do something crazy Operability Testing Manage the environment Debugging Help developers identify issues Complex vs. Simple load

7. Tasks Test Tools Load injector PCAP analyzer These are some of the most essential pieces of any NFT project: Correctly emulate the connections, implement protocol, use API Make sure that KPI measurements are valid and accurate Make sure that a tool does not experience performance issues itself and doesn’t influence KPI measurements Developers whose application you are testing will blame your tools! You will require a proof of your figures based on valid sources (as the developers think)

8. Tasks Test Preparation Load Shape Live production system is the ultimate source of information

9. Tasks Test Execution Automated system restart and test execution is a good idea Our system is a complex look at what’s going on Be ready to run tests again and again

10. Tasks Result Analysis Monitoring hardware reconciles your latency and hardware results Be ready to advocate test results Perform in-depth analysis Make sure results are repeatable – unicorns are non-existent Most of your odd results are probably invalid test runs

11. Questions & Answers Thank you.