Technology Risks in Exchange Trading Exactpro and HSE Seminar 14 Oct Iosif Itkin, Exactpro CEO, London Stock Exchange Group

1. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com1
Технологические риски в биржевой
торговле
Open Access Quality Assurance & Related Software Development for Financial Markets
Tel: +7 495 640 2460, +1 415 830 38 49
www.exactpro.com
14 Октября 2016
Иосиф Иткин, Exactpro CEO, London Stock Exchange Group
Научно-практический семинар
Моделирование и анализ технологических рисков

2. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com2
Technology Risk Typology
Financial Services
Money and
Reputation
Minimal Health & Safety

3. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com3
It argues that if machine brains surpass human
brains in general intelligence, then this new
superintelligence could replace humans as the
dominant lifeform on Earth.
Sufficiently intelligent machines could improve
their own capabilities faster than human
computer scientists. As the fate of gorillas now
depends more on humans than on the actions of
gorillas themselves, so will the fate of future
humanity depend on the actions of the machine
superintelligence. The outcome could be an
existential catastrophe for humans.
Solving the control problem is surprisingly
difficult because most goals, when translated
into machine-implementable code, lead to
unforeseen and undesirable consequences.
Advanced Technology

4. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com4
Existential Risk
Google DeepMind published a paper claiming that it is
possible to implement "a button" to interrupt undesired behaviour
Will we have enough time to press the button once?

5. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com5
Existential Risk

6. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com6
Robot Apocalypse Lessons
The only reason the characters were able to
make it until the end of the second season is
that they were relying on the very same
technology that was trying to kill them.
They were protected by systems having the
same level of sophistication as those they
were struggling with. This is the mentality
that we should adapt.
When we use a complex platform, we need
to build software to test our software. And
you do not want your risk control and test
instruments to be inferior to what will hit you.
Having a good robot on your side is the only
way to survive the robot apocalypse.

7. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com7
Build Software to Test Software
Automated testing
Compares intended and received
results
Automated end 2 end
clearing system testing
Automated monitoring,
analysis and reporting
75K messages / second from a single CPU core
Measures latencies in microsecond range
A variety of algoes simulating end-clients
will run and see how system performs

8. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com8
Build Software to Test Software
Cat on a
Screen
Bug in the
Code

9. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com9
Build Software to Test Software
Cat on a
Screen
Bug in the
Code
The Next Generation of Software Testing Tools

10. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com10
Learning from failure has the status of a cliché,
but this book reveals the astonishing story
behind the most powerful method of learning
known to mankind, and reveals the arsenal of
techniques wielded by some of the world's most
innovative organizations. It also reveals the
dangers of failing to learn from mistakes. In
healthcare, hundreds of thousands of patients
die from preventable medical errors every year
due to a chronic lack of Black Box Thinking.
Using gripping case studies, exclusive
interviews and really practical takeaways,
Matthew Syed – the award-winning journalist
and best-selling author of Bounce - explains how
to turn failure into success, and shows us how
we can all become better Black Box Thinkers.
Learning from Failure

11. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com11
What Can Possible Go Wrong

12. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com12
Fat Finger Errors
12 October 2005
Attempts to sell a single J-Com stock for 610,000 Yen
($5,041)
Mistakenly price and quantity were swapped
Risk systems failure:
• Mizuho Securities
• Tokyo Stock Exchange
Estimated loss $225 millions
This type of errors is called:
Fat Finger

13. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com13
Fat Finger Errors
18 September 2012, Stockholm. A trader had the intention
of posting a sell order for 5,000 SKF B shares. Due to an
input error with the Client, the order volume field was
populated with a negative value (-5,000)
Instead of returning an error, the system converted the
value into a random 9-digit figure - 294,962,296
The Sell Order corresponded to approximately 71 % of the
total outstanding volume in the SKF B share. The Sell
Order resulted in execution of 813,442 shares

14. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com14
Knight Capital Events
• 1 August 2012, USA
• Knight Capital – one of the most successful HFT firms
• Implemented changes related to Retail Liquidity Program
at NYSE
• SMARS – an ultra-fast order router
• Source code responsible for legacy functionality PowerPeg
• 212 parent orders, millions child orders
• Accumulated loss – $460m or $170k/sec
• Incorrectly configured risk systems
• Deployment on 7 servers instead of 8…

15. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com15
Knight Capital Events

16. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com16
Knight Capital Events

17. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com17
Regulated Exchange

18. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com18
Facebook IPO on NASDAQ
• 18 May 2012, NASDAQ, One of the largest IPOs in history
• Secondary trading is preceded by a designate Display Only Period
(DOP)
• Multi-component architecture that included Matching Engine, IPO
Cross Application and Execution Application
• At the end of the DOP, NASDAQ’s “IPO Cross Application” analyzes all
of the buy and sell orders to determine the price at which the largest
number of shares will trade; then NASDAQ’s matching engine matches
the buy and sell orders at that price. Usually takes 1-2 ms
• NASDAQ allowed orders to be cancelled at any time up until the end of
the DOP, including the very brief interval during which the IPO cross
price is calculated. After calculation was completed, the system
performed orders validation check between ME and “IPO Cross
Application”. If any of the orders were cancelled after the start of the
cross, the system would have to repeat the calculation

19. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com19
Failover Proposal

20. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com20
Facebook IPO on NASDAQ
• Over 496k orders participated in the cross, and its duration exceeded
20ms
• Order cancellation arrived during this period, and the application had to
repeat the calculation. Two more cancellations arrived during the
second iteration, and four more - during the third one
• IPO Cross Application went into infinite loop at 11:05
• The NASDAQ team switched off validation check on the secondary
system and performed failover 25 minutes after the start of the loop
• Unknown at that moment, 38k orders submitted between 11:11 and
11:30 were stuck and did not participate in the uncross. It created
another discrepancy, this time with Execution App and Members who
were not able to receive confirmation for orders executed in the cross
until 13:50

21. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com21
Failover Proposal

22. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com22
Facebook IPO Stock Chart
SEC Release No. 34-69655 / May 29, 2013

23. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com23
Flash Crash
The May 6, 2010, Flash Crash, was a United States trillion-dollar stock market crash,
which started at 2:32 p.m. and lasted for approximately 36 minutes.
Event is attributed to HFT, large directional sell, index arbitrage and market manipulation

24. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com24
Defects Management
Areas of research in defect management:
• automatic defect fixing
• automatic defect detection
• metrics and predictions of defect reports
• quality of defect reports
• triaging defect reports

25. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com25
Defects Management
Defect clustering
• Understand weaknesses of software
• Improve testing strategy
Use obtained data to:
• Predict the probability of a defect to be re-opened
• Highlight the area of the code where the bug is most likely located
• Identify requirements and areas most prone to defects in the next release of software

26. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com26
The seventh EXTENT conference will be held in
London, UK .
Find out more:
www.extentconf.com
– LSEG Technology and Quality Assurance
– Risk controls and FPGA
– Market Surveillance systems
– Blockchain and trading technology trends
– Artificial intelligence
Thank You
Software Testing Conferences