Build Software to Test Software
exactpro.com
Machine Learning and RoboCop Testing
(Market Surveillance Systems)
Veronika Kuverova,
QA Analyst, Exactpro
Youcon
Saratov Meetup Week, Saratov
4-10th February, 2019
2 Build Software to Test Software
exactpro.com
What are We Talking About?
- What is the Exchange and how it works
- Who, why and how can manipulate the market
- Market monitoring and control systems (Market Surveillance Systems):
● functional and non-functional requirements
● architecture
● further development
- Functional testing of Market Surveillance Systems: best practices
3 Build Software to Test Software
exactpro.com
What Is the Exchange and How It Works?
The main customers of the
exchange
Credit organizations (banks) and broker
companies through which private and
corporate investors carry out exchange
operations
The Stock Exchange
Specialized marketplace where buyers
and sellers can make deals
4 Build Software to Test Software
exactpro.com
What Is the Exchange and How It Works?
Principles of the exchange:
● publicity and fair pricing;
● equality and legal protection of participants;
● "Transparency" and safety of trade and settlement operations.
● All bidders are required to comply with the rules of trade and fulfill the obligations
established by a particular exchange.
5 Build Software to Test Software
exactpro.com
What Is the Exchange and How It Works?
6 Build Software to Test Software
exactpro.com
What Is the Exchange and How It Works?
Technologies allow:
- organize electronic interaction between trade participants;
- develop universal platforms supporting remote trading channels;
- provide remote identification and customer support for clients from all over
the world.
7 Build Software to Test Software
exactpro.com
What Is the Exchange and How It Works?
If a market participant is trying to make profit
by distorting conventional market
forces or laws then we can talk about
manipulation.
8 Build Software to Test Software
exactpro.com
What Is the Exchange and How It Works?
The supervisor (regulator) must:
- monitor the market;
- monitor the machinations with the prices and the effect
on insider information;
- record the facts of suspicious transactions;
- take measures to bring unfair participants to justice.
9 Build Software to Test Software
exactpro.com
Landscape on Surveillance Solution Providers
Source: Opimas Analysis
10 Build Software to Test Software
exactpro.com
The Main Functional Requirements
- The whole market is in full view.
- Detect instances of manipulation or suspicious behavior and report it in real time or close to it with a
delay of up to 5 seconds (built-in alert system).
- Provide an evidence base (i.e. show the events that led to the alert and who was involved) - the
supervisor must have access to all the necessary information.
- Replay the market (back in time)
- Counting various statistics at different levels (what, where, when, how much and who).
- Show news.
- Be flexible by the users themselves.
- Have a monitoring subsystem of itself.
- Have the ability to graphically represent the state of a particular financial instrument and even the entire
market.
11 Build Software to Test Software
exactpro.com
Market Surveillance System Architecture
Exchange Surveillance
Market Surveillance
Analyst
GUI
Component
s
Data Management
Partition
Storage
Surveillance
Gateway
DownStream
Gateway
Matching
Partition
User
Gateway
12 Build Software to Test Software
exactpro.com
The Manipulations Include
Price manipulations:
Information Manipulation Artificiality of control
for the value of assets - price
(and not a violation of the law in the
transaction)
Misleading other market
participants,
falsity of price signals
13 Build Software to Test Software
exactpro.com
What Is a Notification in the Monitoring System?
Market
Surveillance
Analyst
Alert - a message in the system that occurs when
implementing a scenario (mold, pattern) that
describes suspicious behavior in the market.
!
!
!
!
Alert
Database
Magic
Pattern
Detection
Pattern
DetectionTransactions
!!!!
14 Build Software to Test Software
exactpro.com
How Does the Notification Work?
Benchmark - these are statistics
collected over a period of time.
Threshold - This is the range of values in which a notification might work.
The complexity of checking for notification depends on incoming data and the
sensitivity of threshold ranges.
15 Build Software to Test Software
exactpro.com
Protection Levels Against the Triggering of Misbehaving
Notifications
Functional part
done by Market Surveillance Analyst
● Real-time threshold ranges change
● Turn off “wrong” notifications “on the fly”
Technical part
● Setting limits on the number of triggered notifications at the level of the
notification itself and at the level of the component responsible for their
search
● Setting CPU / memory limits
16 Build Software to Test Software
exactpro.com
Automation in Testing Alerts
MiniRoborts Sailfish
17 Build Software to Test Software
exactpro.com
Market Surveillance from Machine Learning Perspective
18 Build Software to Test Software
exactpro.com
Independent Research by Chartis
19 Build Software to Test Software
exactpro.com
References
https://www.jfokus.se/jfokus17/preso/Machine-Learning-in-Market-Surveillance.pdf
https://info.nice.com/rs/338-EJP-431/images/Opimas_Shortlisting_Trade_Surveillance_Solutions.pdf
https://www.ey.com/Publication/vwLUAssets/ey-trader-surveillance-report/$FILE/EY%20Trader%20Surveillance%20report.pdf
http://www.it.uu.se/edu/course/homepage/projektDV/ht17/final-2017-presentation.pdf
http://www.it.uu.se/edu/course/homepage/projektDV/ht17/project-2017-technical.pdf
20 Build Software to Test Software
exactpro.com
More
Presentations
This Week
21 Build Software to Test Software
exactpro.com
Click to edit the title text format Thank you!

Machine Learning and RoboCop Testing

  • 1.
    Build Software toTest Software exactpro.com Machine Learning and RoboCop Testing (Market Surveillance Systems) Veronika Kuverova, QA Analyst, Exactpro Youcon Saratov Meetup Week, Saratov 4-10th February, 2019
  • 2.
    2 Build Softwareto Test Software exactpro.com What are We Talking About? - What is the Exchange and how it works - Who, why and how can manipulate the market - Market monitoring and control systems (Market Surveillance Systems): ● functional and non-functional requirements ● architecture ● further development - Functional testing of Market Surveillance Systems: best practices
  • 3.
    3 Build Softwareto Test Software exactpro.com What Is the Exchange and How It Works? The main customers of the exchange Credit organizations (banks) and broker companies through which private and corporate investors carry out exchange operations The Stock Exchange Specialized marketplace where buyers and sellers can make deals
  • 4.
    4 Build Softwareto Test Software exactpro.com What Is the Exchange and How It Works? Principles of the exchange: ● publicity and fair pricing; ● equality and legal protection of participants; ● "Transparency" and safety of trade and settlement operations. ● All bidders are required to comply with the rules of trade and fulfill the obligations established by a particular exchange.
  • 5.
    5 Build Softwareto Test Software exactpro.com What Is the Exchange and How It Works?
  • 6.
    6 Build Softwareto Test Software exactpro.com What Is the Exchange and How It Works? Technologies allow: - organize electronic interaction between trade participants; - develop universal platforms supporting remote trading channels; - provide remote identification and customer support for clients from all over the world.
  • 7.
    7 Build Softwareto Test Software exactpro.com What Is the Exchange and How It Works? If a market participant is trying to make profit by distorting conventional market forces or laws then we can talk about manipulation.
  • 8.
    8 Build Softwareto Test Software exactpro.com What Is the Exchange and How It Works? The supervisor (regulator) must: - monitor the market; - monitor the machinations with the prices and the effect on insider information; - record the facts of suspicious transactions; - take measures to bring unfair participants to justice.
  • 9.
    9 Build Softwareto Test Software exactpro.com Landscape on Surveillance Solution Providers Source: Opimas Analysis
  • 10.
    10 Build Softwareto Test Software exactpro.com The Main Functional Requirements - The whole market is in full view. - Detect instances of manipulation or suspicious behavior and report it in real time or close to it with a delay of up to 5 seconds (built-in alert system). - Provide an evidence base (i.e. show the events that led to the alert and who was involved) - the supervisor must have access to all the necessary information. - Replay the market (back in time) - Counting various statistics at different levels (what, where, when, how much and who). - Show news. - Be flexible by the users themselves. - Have a monitoring subsystem of itself. - Have the ability to graphically represent the state of a particular financial instrument and even the entire market.
  • 11.
    11 Build Softwareto Test Software exactpro.com Market Surveillance System Architecture Exchange Surveillance Market Surveillance Analyst GUI Component s Data Management Partition Storage Surveillance Gateway DownStream Gateway Matching Partition User Gateway
  • 12.
    12 Build Softwareto Test Software exactpro.com The Manipulations Include Price manipulations: Information Manipulation Artificiality of control for the value of assets - price (and not a violation of the law in the transaction) Misleading other market participants, falsity of price signals
  • 13.
    13 Build Softwareto Test Software exactpro.com What Is a Notification in the Monitoring System? Market Surveillance Analyst Alert - a message in the system that occurs when implementing a scenario (mold, pattern) that describes suspicious behavior in the market. ! ! ! ! Alert Database Magic Pattern Detection Pattern DetectionTransactions !!!!
  • 14.
    14 Build Softwareto Test Software exactpro.com How Does the Notification Work? Benchmark - these are statistics collected over a period of time. Threshold - This is the range of values in which a notification might work. The complexity of checking for notification depends on incoming data and the sensitivity of threshold ranges.
  • 15.
    15 Build Softwareto Test Software exactpro.com Protection Levels Against the Triggering of Misbehaving Notifications Functional part done by Market Surveillance Analyst ● Real-time threshold ranges change ● Turn off “wrong” notifications “on the fly” Technical part ● Setting limits on the number of triggered notifications at the level of the notification itself and at the level of the component responsible for their search ● Setting CPU / memory limits
  • 16.
    16 Build Softwareto Test Software exactpro.com Automation in Testing Alerts MiniRoborts Sailfish
  • 17.
    17 Build Softwareto Test Software exactpro.com Market Surveillance from Machine Learning Perspective
  • 18.
    18 Build Softwareto Test Software exactpro.com Independent Research by Chartis
  • 19.
    19 Build Softwareto Test Software exactpro.com References https://www.jfokus.se/jfokus17/preso/Machine-Learning-in-Market-Surveillance.pdf https://info.nice.com/rs/338-EJP-431/images/Opimas_Shortlisting_Trade_Surveillance_Solutions.pdf https://www.ey.com/Publication/vwLUAssets/ey-trader-surveillance-report/$FILE/EY%20Trader%20Surveillance%20report.pdf http://www.it.uu.se/edu/course/homepage/projektDV/ht17/final-2017-presentation.pdf http://www.it.uu.se/edu/course/homepage/projektDV/ht17/project-2017-technical.pdf
  • 20.
    20 Build Softwareto Test Software exactpro.com More Presentations This Week
  • 21.
    21 Build Softwareto Test Software exactpro.com Click to edit the title text format Thank you!