Extent 2013 Obninsk Trading Systems: Testing at the Confluence of FT & NFT

  • 706 views
Uploaded on

Exactpro Systems - "Trading Systems: Testing at the Confluence of FT & NFT" …

Exactpro Systems - "Trading Systems: Testing at the Confluence of FT & NFT"

Alexey Zverev, Managing Director, Exactpro Systems
Alyona Bulda, QA Project Lead, Exactpro Systems
Ivan Bobrov, HFT Analyst, Exactpro Systems

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
706
On Slideshare
0
From Embeds
0
Number of Embeds
3

Actions

Shares
Downloads
3
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • *
  • *
  • *
  • *
  • *

Transcript

  • 1. Trading Systems:Testing at the Confluence of FT & NFTAlexey Zverev, Managing DirectorAlyona Bulda, QA Project LeadIvan Bobrov, HFT Analyst
  • 2. Introduction 1. Introduction 2. Our topic - how to find / reproduce "toxic issues" which can happen under the load or in real live environment conditions 3. We will talk about: a. A classical approach where FT and NFT are separate streams b. What is missed here c. Some approaches we use or may apply to fill this gap between FT and NFT (applying to trading systems) 4. Our presenters
  • 3. FT NFT focus on: focus on:• Test scenarios are based on functional • Testing is based on non-functionalrequirements requirements• Test tools which allow to run simplistic tests • Load scenarios are driven by thesuch as to send an order -> to verify a response technical characteristics in PROD.• Prepare a functional library starting from • The main target of the tests is to assesssimple scenarios and adding more complex ones non-functional requirements: latency,step by step capacity and fault tolerance• Test scenarios are automated • A lot of efforts are invested in correct• A test Library is enhanced as we proceed latency measurements and determiningwith its testing and production clients scenarios conditions when the system breaks• An automated library will be reused as a • Further regressions are done toregression test library which is executed compare system performance indicatorsagainst each release after go live and can be with the previous releasestaken for consequent projects
  • 4. What do we miss when we keep FT and NFT separated ?- Any complex system must sustain a certain level of concurrency. Some issuesappear when several events happen simultaneously (Race conditions).Example 1: Clients send a CxlReplace request and approx. at the same timethe ME sends a fill back.These 2 events meet atthe Order Managementserver, and the state ofthe order can get corrupted.
  • 5. What do we miss when we keep FT and NFT separated ?- Production order books and various system object states are more complexthan in our FT scripts, and no amount of test cases can cover all possiblescenariosExample 2: A User tries tohit two different impliedorders coming from thesame parent orderwhich is being restatedin the middle of the process.
  • 6. What do we miss when we keep FT and NFT separated ?- Components may behave differently under the loadExample 3: The System must support variousthrottling limitations and the requirements ofthe system behaviour. The correctnessof functional system behavior can betested only under the load.- Integer memory overflowExample 4: Turnover (VWAP) fields in the statistics messages and incrementalOrderID overflow;* the picture was taken from https://encrypted-tbn1.gstatic.com/images?q=tbn:ANd9GcS9nlDdLAmuvmyRqqZsanbyfwL601nJlOvtHGEfJBS4DKnmSTANUA
  • 7. What do we miss when we keep FTand NFT separated ?- Dynamic Mass events which happen during DLCExample 5: Market Open,Market Close, Auctions,..The state of the systemat exact instancewhen the system goesfrom one session to anothershould be verified with the loadapplied against the system.
  • 8. Detailed Verification Of The System Behavior Under Random Load  We need to make sure that we provide extensive coverage - basic consistency check  We need to make sure that the load test mimics - market data must be internally consistent what happens in the production system - order book should be available and not be  We need to verify that the system different works as expected + Recreation of exact steps of real production issues - It is difficult to record multiple network streams
  • 9. Performing Functional Testing Against The System Under The Load to leave a part of the system intact by the load to adjust functional tests to cope with the activity of the load to make scripts in case when we perform tests actively interacting with the load * scenarios must be flexible * sustain interaction with the script
  • 10. Testing With The Baseline The example of the cycle of a Clearing System A detailed, step by step analysis
  • 11. The “Mini Robots” Solution
  • 12. The “Mini Robots” Solution
  • 13. Questions & Answers-tO Thank You-tO