The Art of Testing Trading Platforms 23 rd July 2008 Ravi Gurrapadi

Agenda Exchange/Trading Platform Overview Key Challenges/Risks Acceptance Criteria Test Governance/Management Functional Testing/Complex scenarios Integration Testing Performance Testing Failover/Recovery Testing Summary Questions

Exchange/Trading Platform Overview

Key Challenges/Risks

Acceptance Criteria

Test Governance/Management

Functional Testing/Complex scenarios

Integration Testing

Performance Testing

Failover/Recovery Testing

Summary

Questions

Typical Stock Exchange/ Trading Platform Overview

Key Challenges/Risks Business Challenges Developing complex trading scenarios & test cases To reflect real time trading Integrating different modules (Front, Middle and Back office functions) Compliance to evolving regulations like RegNMS, MiFID Building & maintaining the domain competence Technical Challenges Interaction of multiple systems through several interfaces and gateways Automation of the complex scenarios & interface gateways Performance testing to ensure acceptable levels of latency as per the SLAs Risks Business – changes in business model, rules & regulatory requirements, introduction of new products, down time (revenue lost), time to market Technology – risk of failure, new technology integration, legacy applications, migration to a new application, 3 rd party application issues affecting your own system

Business Challenges

Developing complex trading scenarios & test cases

To reflect real time trading

Integrating different modules (Front, Middle and Back office functions)

Compliance to evolving regulations like RegNMS, MiFID

Building & maintaining the domain competence

Technical Challenges

Interaction of multiple systems through several interfaces and gateways

Automation of the complex scenarios & interface gateways

Performance testing to ensure acceptable levels of latency as per the SLAs

Risks

Business – changes in business model, rules & regulatory requirements, introduction of new products, down time (revenue lost), time to market

Technology – risk of failure, new technology integration, legacy applications, migration to a new application, 3 rd party application issues affecting your own system

Acceptance Criteria Testing for core functionalities as specified in the functional requirements User Management Instrument Management Quote/Order Management Session Management Trade Management Risk Management Compliance to Regulations Testing for agreed specifications as per SLAs Performance Security Network Reliability

Testing for core functionalities as specified in the functional requirements

User Management

Instrument Management

Quote/Order Management

Session Management

Trade Management

Risk Management

Compliance to Regulations

Testing for agreed specifications as per SLAs

Performance

Security

Network

Reliability

Governance Structure

Test Governance/Management Major stakeholders Client organization, development organization Management Domain Technical Client's clients/vendors Adherence to agreements with all the stakeholders Test Plans Deliverables Schedule Change management SLAs Metrics Escalation procedures Sub contracts

Major stakeholders

Client organization, development organization

Management

Domain

Technical

Client's clients/vendors

Adherence to agreements with all the stakeholders

Test Plans

Deliverables

Schedule

Change management

SLAs

Metrics

Escalation procedures

Sub contracts

Test Planning & Design

Level 1 Business Process Flows

Level 2 Test Scenarios Test Scenario

Level 2 Test Scenarios

Level 3 Test Cases Test Case

Integration Testing/Interface Testing

Performance Testing Response times are critical New generation systems have minimal latency Time is literally money Highly complex Multiple API’s and connections to external systems Need to performance test sub-systems Variety of scenarios Examples: day in life of, daily peaks, fast markets Soak tests for memory leaks, etc. Performance test data Huge volumes of test data required Needs to be generated and managed efficiently

Response times are critical

New generation systems have minimal latency

Time is literally money

Highly complex

Multiple API’s and connections to external systems

Need to performance test sub-systems

Variety of scenarios

Examples: day in life of, daily peaks, fast markets

Soak tests for memory leaks, etc.

Performance test data

Huge volumes of test data required

Needs to be generated and managed efficiently

Failover/Recovery Testing Test system resilience to failure and the ability to continue to provide service during failure & recovery Test standard operating procedures to ensure they are robust and comprehensive Ensure recovery is with acceptable time scales and performance criteria Areas to cover: Software (operating system, process and application) Hardware Power Failure Network

Test system resilience to failure and the ability to continue to provide service during failure & recovery

Test standard operating procedures to ensure they are robust and comprehensive

Ensure recovery is with acceptable time scales and performance criteria

Areas to cover:

Software (operating system, process and application)

Hardware

Power Failure

Network

Summary Stock Exchanges have inherent complexity in terms of business flow and business rules – testing is a critical element in ensuring business delivery Key areas of interest for stock exchanges: Increased volumes Speedy execution Effective surveillance to safeguard investor and market interest These go in as requirements to a trading application and the testing process to ensure the implementation is a business success Having a well tested application that doesn't fail in real time gives an edge over the competition – areas of failure in this area become public news

Stock Exchanges have inherent complexity in terms of business flow and business rules – testing is a critical element in ensuring business delivery

Key areas of interest for stock exchanges:

Increased volumes

Speedy execution

Effective surveillance to safeguard investor and market interest

These go in as requirements to a trading application and the testing process to ensure the implementation is a business success

Having a well tested application that doesn't fail in real time gives an edge over the competition – areas of failure in this area become public news

Questions? For any questions following the presentation, then please e-mail [email_address]

For any questions following the presentation, then please e-mail [email_address]