EXTENT Trading Technology Trends & Quality Assurance Conference in Obninsk, 2 March, 2013
Trading Systems:
Testing at the Confluence of FT & NFT
Alexey Zverev, Managing Director
Alyona Bulda, QA Project Lead
Ivan Bobrov, HFT Analyst
Extent 2013 Obninsk Trading Systems: Testing at the Confluence of FT & NFTextentconf Tsoy
Exactpro Systems - "Trading Systems: Testing at the Confluence of FT & NFT"
Alexey Zverev, Managing Director, Exactpro Systems
Alyona Bulda, QA Project Lead, Exactpro Systems
Ivan Bobrov, HFT Analyst, Exactpro Systems
Extent 2013 Obninsk Trading Systems: Testing at the Confluence of FT & NFTextentconf Tsoy
Exactpro Systems - "Trading Systems: Testing at the Confluence of FT & NFT"
Alexey Zverev, Managing Director, Exactpro Systems
Alyona Bulda, QA Project Lead, Exactpro Systems
Ivan Bobrov, HFT Analyst, Exactpro Systems
SystemVerilog based OVM and UVM Verification MethodologiesRamdas Mozhikunnath
Introduction to System Verilog based verification methodologies - OVM and UVM concepts
For more online courses and resources follow http://verificationexcellence.in/
Our Performance Testing Center of Excellence analyzes our client’s performance requirements, defines performance test strategies, roadmap and metrics, assesses reports and provides recommendations. Our testing team has in-depth expertise across various open-source and commercial performance testing tools.
Tradecope - Low-Latency Solution for Algorithmic and High Frequency Trading
Milan Dvorak, Netcope Technologies
11 November 2015
Trading Technology Trends & Quality Assurance Conference in St. Petersburg
Basics of Functional Verification - Arrow DevicesArrow Devices
Are you new to functional verification? Or do you need a refresher? This presentation takes you through the basics of functional verification - overall scope and process with examples. Also included are some tips on do's and don'ts!
Astute's PeopleSoft Integration Testing Utilities and Tools ImplementationBeastute
The utilities and tools discussed in the product documentation are automatically installed with PeopleTools. Review the information provided in this section for additional requirements, prerequisites and considerations.visit: http://www.beastute.com/peoplesoft-implementation/
Introduction to SOC Verification Fundamentals and System Verilog language coding. Explains concepts on Functional Verification methodologies used in industry like OVM, UVM
Difference between functional testing and non functional testingpooja deshmukh
Up till now, you must have seen separate articles for Functional Testing and Non-Functional testing. In any case, in today’s article we will see the real distinction between Functional Testing and Non-Functional Testing.
Tellefsen and Exactpro Systems Team Up to Help Clients with REG SCI
Meeting the demanding new requirements imposed upon firms by SEC Regulation SCI
is a key issue for many firms, especially in the areas of independent systems testing
and certification. Tellefsen and Company, L.L.C. (Tellefsen) and Exactpro Systems,
LLC (Exactpro) have today announced a marketing partnership in which both firms will
collaborate to provide key industry constituents with market structure consulting, financial
technology infrastructure testing and software quality assurance testing services.
Tellefsen is a management consulting firm founded in 1984 to provide counsel and
professional services to meet the growing needs of the financial services industry. The
firm’s principals and consultants have a wealth of experience in regulatory compliance
and U.S. market infrastructure (exchanges, ATS, dark pools etc). One of Tellefsen’s core
competencies has been is assisting firms with their regulatory compliance requirements
through technology analysis, planning, testing, and evidencing.
Exactpro is a rapidly growing quality assurance services firm specializing in market
infrastructure. Exactpro provides both functional and non-functional testing for global
exchanges, dark pools and clearing houses using a range of sophisticated in-house built
testing and evidencing tools.
John Rapa, President and CEO of Tellefsen, indicates: “The implementation of Reg SCI
is another example of market regulations where market participants must provide evidence
of their comprehensive infrastructure testing, policies and procedures and market oversight
compliance. We are delighted to team up with Exactpro whom we know from other customer
engagements. We are very impressed by their capabilities, diligence and effectiveness in
planning, executing and evidencing tests that prove regulatory compliance”.
Iosif Itkin, Co-Founder of Exactpro, says: “Our firm has a great set of tools and services that
we believe can help clients meet Regulation SCI in a cost effective manner. We are very
conversant with this domain and are delighted to be working with Tellefsen on this; our two
firms’ capabilities are entirely complementary”.
Delivering Large Post-Trade Initiatives: Quality Assurance and Key ChallengesIosif Itkin
Difficulties in coping with functional complexity
experienced by the project team:
• Aligning tests correctly in batches that correspond to events in
the daily life cycle, including multiple days
• Accelerating the daily life cycle to speed up test execution and to cover
Settlement Failure processes in testing within a reasonable timeframe
• Accurately simulating and controlling test inputs from upstream systems
and outputs to downstream systems
• Management of complex Reference Data setup
• Involvement of complex components such as Risk Management
and Collateral Management systems
• Test cycle turnaround time, client certification and test automation
Keynote Speakers:
Alyona Lamash FRM Head of Risk Management Practice, Exactpro Systems
Mark Ryland Financial Markets Technology Consultant, Independent Consulting, United Kingdom
SystemVerilog based OVM and UVM Verification MethodologiesRamdas Mozhikunnath
Introduction to System Verilog based verification methodologies - OVM and UVM concepts
For more online courses and resources follow http://verificationexcellence.in/
Our Performance Testing Center of Excellence analyzes our client’s performance requirements, defines performance test strategies, roadmap and metrics, assesses reports and provides recommendations. Our testing team has in-depth expertise across various open-source and commercial performance testing tools.
Tradecope - Low-Latency Solution for Algorithmic and High Frequency Trading
Milan Dvorak, Netcope Technologies
11 November 2015
Trading Technology Trends & Quality Assurance Conference in St. Petersburg
Basics of Functional Verification - Arrow DevicesArrow Devices
Are you new to functional verification? Or do you need a refresher? This presentation takes you through the basics of functional verification - overall scope and process with examples. Also included are some tips on do's and don'ts!
Astute's PeopleSoft Integration Testing Utilities and Tools ImplementationBeastute
The utilities and tools discussed in the product documentation are automatically installed with PeopleTools. Review the information provided in this section for additional requirements, prerequisites and considerations.visit: http://www.beastute.com/peoplesoft-implementation/
Introduction to SOC Verification Fundamentals and System Verilog language coding. Explains concepts on Functional Verification methodologies used in industry like OVM, UVM
Difference between functional testing and non functional testingpooja deshmukh
Up till now, you must have seen separate articles for Functional Testing and Non-Functional testing. In any case, in today’s article we will see the real distinction between Functional Testing and Non-Functional Testing.
Tellefsen and Exactpro Systems Team Up to Help Clients with REG SCI
Meeting the demanding new requirements imposed upon firms by SEC Regulation SCI
is a key issue for many firms, especially in the areas of independent systems testing
and certification. Tellefsen and Company, L.L.C. (Tellefsen) and Exactpro Systems,
LLC (Exactpro) have today announced a marketing partnership in which both firms will
collaborate to provide key industry constituents with market structure consulting, financial
technology infrastructure testing and software quality assurance testing services.
Tellefsen is a management consulting firm founded in 1984 to provide counsel and
professional services to meet the growing needs of the financial services industry. The
firm’s principals and consultants have a wealth of experience in regulatory compliance
and U.S. market infrastructure (exchanges, ATS, dark pools etc). One of Tellefsen’s core
competencies has been is assisting firms with their regulatory compliance requirements
through technology analysis, planning, testing, and evidencing.
Exactpro is a rapidly growing quality assurance services firm specializing in market
infrastructure. Exactpro provides both functional and non-functional testing for global
exchanges, dark pools and clearing houses using a range of sophisticated in-house built
testing and evidencing tools.
John Rapa, President and CEO of Tellefsen, indicates: “The implementation of Reg SCI
is another example of market regulations where market participants must provide evidence
of their comprehensive infrastructure testing, policies and procedures and market oversight
compliance. We are delighted to team up with Exactpro whom we know from other customer
engagements. We are very impressed by their capabilities, diligence and effectiveness in
planning, executing and evidencing tests that prove regulatory compliance”.
Iosif Itkin, Co-Founder of Exactpro, says: “Our firm has a great set of tools and services that
we believe can help clients meet Regulation SCI in a cost effective manner. We are very
conversant with this domain and are delighted to be working with Tellefsen on this; our two
firms’ capabilities are entirely complementary”.
Delivering Large Post-Trade Initiatives: Quality Assurance and Key ChallengesIosif Itkin
Difficulties in coping with functional complexity
experienced by the project team:
• Aligning tests correctly in batches that correspond to events in
the daily life cycle, including multiple days
• Accelerating the daily life cycle to speed up test execution and to cover
Settlement Failure processes in testing within a reasonable timeframe
• Accurately simulating and controlling test inputs from upstream systems
and outputs to downstream systems
• Management of complex Reference Data setup
• Involvement of complex components such as Risk Management
and Collateral Management systems
• Test cycle turnaround time, client certification and test automation
Keynote Speakers:
Alyona Lamash FRM Head of Risk Management Practice, Exactpro Systems
Mark Ryland Financial Markets Technology Consultant, Independent Consulting, United Kingdom
EXTENT Trading Test Tools Evolution TheoryIosif Itkin
EXTENT Trading Technology Trends & Quality Assurance Conference in Obninsk, 2 March, 2013
Trading Test Tools: Evolution Theory
Alexander Barch, Technical Analyst, Exactpro Systems
Anastasia Matveeva, QA Manager, Exactpro Systems
Innovation in Quality Assurance – What is the Impact on Trading technology?Iosif Itkin
Agile methodologies and the death of software testing.
Can finance industry adopt the same formal methods that allowed NASA to reach Pluto?
Lessons learned from the Terminator/BS Galactica and the next generation of risk controls.
What is crowd-sourced verification in the age when machines rule the market?
Cognitive technologies and building test harness for algo trading systems.
EXTENT Conference - October 2011
Test Automation for Trading Systems
Verification of Financial Models:Duplication of Development Efforts?
Alyona Lamash, Head of Risk Management Systems Practice Innovative Trading Systems
Boris Rabinovich, Senior QA Analyst ITS
Behavior Driven Development Pros and ConsIosif Itkin
The Cons of Behavior Driven Development (BDD)
Ivan Bobrov, ClubQA Co-Founder, Kostroma
The Pros of Behavior Driven Development (BDD): Business User Scenarios
Natalia Zaitseva, Exchange Functional Test Automation Lead Innovative Trading Systems
EXTENT Conference.
October 29-30, 2011
Test Automation for Trading Systems
Renaissance Hotel Moscow
Testing real-time Linux. What to test and how Chirag Jog
This paper describes testing of the real-time (CONFIG_PREEMPT_RT) Linux kernel. It explains how testing the real-time kernel is different from testing the mainline Linux kernel and provides some tips and guidelines about writing test cases for the real-time kernel. It illustrates real-time tests in the Linux Test Project (LTP) suite using examples. It also briefly covers real-time tests that are not part of LTP.
Why test automation is getting more difficult, and what can be done about it. This slides are from a presentation by Group Director, Product Management at TestPlant, Gordon McKeown, which was presented at the Northern Lights conference in Manchester in April 2016.
DBTest 2013 - In Data Veritas - Data Driven Testing for Distributed SystemsMihir Gandhi
The increasing deployment of distributed systems to solve
large data and computational problems has not seen a con-
comitant increase in tools and techniques to test these sys-
tems. In this paper, we propose a data driven approach to
testing. We translate our intuitions and expectations about
how the system should behave into invariants, the truth of
which can be verified from data emitted by the system. Our
particular implementation of the invariants uses Q, a high-
performance analytical database, programmed with a vector
language.
Performance tuning Grails Applications GR8Conf US 2014Lari Hotari
Grails has great performance characteristics but as with all full stack frameworks, attention must be paid to optimize performance. In this talk Lari will discuss common missteps that can easily be avoided and share tips and tricks which help profile and tune Grails applications.
Nonfunctional Testing: Examine the Other Side of the CoinTechWell
Creating a highly available, scalable, and high-performing system requires a substantial amount of what we call nonfunctional testing. Developing nonfunctional testing skills is a must for many of today’s quality engineers (QEs). For the past several years, Balaji Arunachalam’s quality team for Intuit Core Services has experienced several highly available and disaster recovery buildup and testing challenges. Their journey includes the evolution of functional QEs into hybrid QEs who are capable of doing both functional and nonfunctional testing. Nonfunctional testing includes capacity, stability, benchmarking, FMEA/RAS, datacenter failover, and scalability testing. Balaji shares nonfunctional testing best practices, learnings, and mistakes they encountered on this journey. If you or your team is ready flip the coin and take a serious look at nonfunctional testing methods, opportunities, challenges, and solutions, this session is for you.
Performance doesn’t have the same definition between system administrators, developpers and business teams. What is Performance ? High CPU usage, not scalable web site, low business transaction rate per sec, slow response time, … This presentation is about maths, code performance, load testing, web performance, best practices, … Working on performance optimizaton is a very broad topic. It’s important to really understand main concepts and to have a clean and strong methodology because it could be a very time consumming activity. Happy reading !
Divide and stress: the journey to component load test talk was given at ExpoQA 2017 under the track of Quality Assurance and Performance.
Describes what are the most common pains that big companies suffer on load testing processes with expesive cost of 1:1s replicas of production environment for performance testing.
In order to reduce those expesive costs, The Workshop designed a new methodology that aims to reduce operational costs, human errors and enables the performance testing in Continuous Delivery pipelines, that it also can be adopted by Continuous Deployment scenarios.
Component Based Load Testing (CBT) is a methodology designed in The Workshop (http://theworkshop.com) that rethinks what the future of performance testing should do.
CBT introduces the load test executon as part of CD pipelines, ensuring the quality of our products through defined exit criteria for the main metrics, determining if the changes of a new release is ready or not to progress to next environments (stage, prod, etc).
CBT tries to use a pool resources efficiently, making them avaiable for any load test execution requested by any of our products. CBT main mission is trying to reduce all operative costs of maintain 1:1 replicas of production environments, by having a reduce pool of resources where the performance tests are executed using dockers by a reduced time under really volatile environments.
Similar to Trading Systems: Testing at the Confluence of FT and NFT (20)
Foundations of Software Testing Lecture 4Iosif Itkin
This lecture is a part of the online course on Software Testing for Complex Intelligent Systems and Autonomous Vehicles. The course lectures provide the theoretical basics of testing autonomous systems based on artificial intelligence.
The fourth lecture of the course entitled Foundations of Software Testing reviews the ‘absence-of-errors fallacy’ and other principles of software testing, as well as the types and levels of software testing. The lecture also provides a fuller picture of the understanding of test objectives and methodologies by different schools of thought within the software testing domain.
QA Financial Forum London 2021 - Automation in Software Testing. Humans and C...Iosif Itkin
Speaker: Iosif Itkin, co-CEO & co-founder, Exactpro Systems
9th November 2021
Hilton Canary Wharf
Exactpro is an independent software testing business focused on mission-critical financial market infrastructures, primarily exchanges and clearing houses. In his presentation, Iosif will give a brief overview of research on the concept of model-based testing and the principal challenges of its application while testing complex distributed systems. He will also outline the broader context of interaction between humans and complex computer models.
Exactpro FinTech Webinar - Global Exchanges Test OraclesIosif Itkin
Global Exchanges series webinar to discuss Test Oracles. A test oracle is a mechanism for determining whether a test has passed or failed. The use of oracles involves comparing the output(s) of the system under test for a given test-case input, to the output(s) that the oracle determines the product should have. We will review various types of test oracles using examples from Exactpro’s Global Exchanges division projects and protocol-based interactions in trading systems.
Exactpro FinTech Webinar - Global Exchanges FIX ProtocolIosif Itkin
Exactpro’s Global Exchanges Division training session on FIX Trading Protocol.
The essence of the FIX protocol and its place in the overall structure of network protocols, FIX message attributes and the internal data types of the protocol.
Operational Resilience in Financial Market InfrastructuresIosif Itkin
A4Q World Congress 13-16 April 2021
Iosif Itkin
Exactpro provides independent software testing services for mission critical technology that underpins global financial markets – exchanges and clearing houses. Half of the top 20 global exchange groups on all continents around the globe rely on processes, platforms and people from Exactpro to improve their quality and reliability. The company has spent the last 11 years studying operational resilience in this crucial sector. The presentation will outline the key principles for software testing of the systems that process hundreds of millions of orders per day with roundtrip latencies below one hundred microseconds.
20 Simple Questions from Exactpro for Your Enjoyment This Holiday SeasonIosif Itkin
Warmest wishes for a happy holiday season and a wonderful New Year!
We look forward to our continued collaboration in 2020. Thank you for your support.
QA-Financial Forum 2019 in New York
13 November
Iosif Itkin, CEO and co-founder
Elena Treshcheva, Business Development Manager and Researcher
An October 2019 survey by BoE and FCA found that ML in financial organizations has already passed an initial development phase, and the usage of live ML applications is about to dramatically increase over the next three years. Artificial Intelligence systems are used in market surveillance, they are providing intellectual analysis of news feeds, and they are an important part of the conversational agents facing users and helping them with their business needs from identity verification to trading and portfolio management. How to ensure that an AI-powered system is up to its task? And what would that mean from the software testing perspective?
EXTENT 2019: Exactpro Quality Assurance for Financial Market InfrastructuresIosif Itkin
On Complex Software Systems Testing — Alexey Zverev, co-CEO & co-founder, Exactpro
Software Testing and Machine Learning
Mind the Gap. Applying Process Mining
Learning from Failure is not just for Humans
Dancing with Whales. Adaptive Log Classification System
On Traceability and the Illusion of Control
Building Partnerships
Demystifying DLT Testing One Network at a Time
Get the MOST from FIX
Georgia on My Mind
Build Software to Test Software — Iosif Itkin, co-CEO & co-founder, Exactpro
ClearTH Test Automation Framework: Case Study in IRS & CDS Swaps Lifecycle Mo...Iosif Itkin
Synchronize Europe
18th June 2019
Iosif Itkin, co-CEO and co-founder, Exactpro
Using the ISDA CDM Swaps application, simultaneously execute multiple end-to-end scenarios for DAML applications in capital markets - validate with actual contract data on ledger.
EXTENT Talks 2019 Tbilisi: Failover and Recovery Test Automation - Ivan ShamraiIosif Itkin
Ivan Shamray, Senior NFT Analyst, Exactpro
20 April 2019 EXTENT Talks, Tbilisi, Georgia
Tbilisi QA Community
EXTENT Talks is a meeting place for IT specialists working in various industries and seeking professional growth, practitioners from IT firms, as well as Quality Assurance enthusiasts of all backgrounds interested in actively participating in local IT events.
EXTENT Talks QA Community Tbilisi 20 April 2019 - Conference OpenIosif Itkin
EXTENT Talks is a meeting place for IT specialists working in various industries and seeking professional growth, practitioners from IT firms, as well as Quality Assurance enthusiasts of all backgrounds interested in actively participating in local IT events. The first EXTENT Talks were held in Tbilisi on 22 February 2019, initiating the creation a QA Community in Tbilisi and laying a foundation for an international platform for exchanging experience and knowledge in the field of software testing, development and IT. The program of the inaugural event included presentations on ISTQB, Software Testing, and Agile methodology from senior specialists. The next EXTENT Talks in Tbilisi will take place on 20 April 2019.
User-Assisted Log Analysis for Quality Control of Distributed Fintech Applica...Iosif Itkin
The First IEEE International Conference On Artificial Intelligence Testing (2019 IEEE AITest)
Iosif Itkin, Anna Gromova, Anton Sitnikov, Elena Treshcheva, Rostislav Yavorskiy, Evgenii Tsymbalov, Andrey Novikov and Kirill Rudakov
1 Exactpro, UK, Georgia, USA, Russia
2 Skolkovo Institute of Science and Technology, Russia
3 Higher School of Economics, Russia
Speakers: Iosif Itkin, CEO and Co-Founder and Elena Treshcheva, Business Development Manager and Researcher - Exactpro
Exactpro provides software testing services for mission-critical technology that underpins global financial markets. Exactpro clients are regulated by FCA, Bank of England and their counterparts from other countries. During this session, Elena and Iosif will talk about end-to-end software testing for post-trade systems in financial market infrastructures. What are the key challenges in quality assurance at this scale? What kind of cognitive biases affect SDLC? How precise is the knowledge about the systems under test? What constitutes good test evidence? How to deal with complexity in regulated environments?
Behaviour Driven Development: Oltre i limiti del possibileIosif Itkin
The QA Financial Forum: Milan 2019
23 January at the Excelsior Hotel Gallia.
Anna-Maria Lukina, Exactpro Business Development Director
The QA Financial Forum: Milan is one of the leading fintech conferences in Italy. The event focuses on the latest achievements in software risk management and automation of software testing. The predominant theme of the Milan event will be Quality Assurance for the entire Software Development Life Cycle (SDLC).
The topics under discussion will feature:
- Technologies for Automation & AI
- DevOps & CI/CD
- Value Stream Management
- Test Data Management
- Regulatory Compliance
- App Security & DevSecOps
- Testing and quality assurance of Blockchain platforms
The official language of the event is Italian.
On 17th January 2018 Exactpro successfully completed a management buyout from London Stock Exchange Group (LSEG), signed a new multi-year master services agreement with LSEG, and opened its head office in London.
What else has happened in 2018?
I wanted to take the opportunity to reflect on what has been an unusual year for Exactpro.
Integration front to back - Mr. Custodian tear down that wall
The scope of the application level has been continuous extended over the years, albeit with a focus on the area of pre-trade and trade.
Recently, there has been an increased interest to move further into the area of post-trade which is predominantly driven by the ISO 20022 standard. Is there really a need for new FIX messages in areas such as payments and
what are the integration problems needing a resolution?
Panellists
- Iosif Itkin, CEO, Exactpro
- Jim Northey, Co-Chair Global Technical Committee, Americas Region, FIX Trading Community, Chair Elect, ISO TC68 Financial
Services Technical Committee, and Consultant and Industry Standards Liaison, Itiviti
- Barry Young, Director, Aladdin Product Manager, BlackRock
BDD. The Outer Limits. Iosif Itkin at Youcon (in Russian)Iosif Itkin
Exactpro is supporting the 3rd annual IT-conference YouCon to take place on 14th October in Saratov, Russia. Over 900 programmers, systems engineers and architects, software QA engineers, and marketing specialists will gather to discuss the latest trends in programming technology. It is the largest IT industry event in Saratov.
Iosif Itkin, CEO of Exactpro, part of London Stock Exchange Group, will deliver a "BDD. The Outer Limits" presentation named after Iosif's favorite Sci-Fi series.
The topics to be covered are:
Behavior Driven Development concepts
Applying BDD in trading and clearing systems
Specification by Example and using production data
Combining Model-based testing and BDD
The Outer Limits
There will be an opportunity to ask questions, share thoughts and expertise in BDD, or just chat with a representative at the Exactpro stand at any time during the event.
Don't miss out, stop by and ask how you can get your Exactpro souvenir :)
We look forward to meeting you there!
#Exactpro #Youconsaratov
BDD. The Outer Limits. Iosif Itkin at Youcon (in Russian)
Trading Systems: Testing at the Confluence of FT and NFT
1. Trading Systems:
Testing at the Confluence of FT & NFT
Alexey Zverev, Managing Director
Alyona Bulda, QA Project Lead
Ivan Bobrov, HFT Analyst
2. Introduction
1. Introduction
2. Our topic - how to find / reproduce "toxic issues" which can happen under
the load or in real live environment conditions
3. We will talk about:
a. A classical approach where FT and NFT are separate streams
b. What is missed here
c. Some approaches we use or may apply to fill this gap between FT and
NFT (applying to trading systems)
4. Our presenters
3. FT NFT
focus on: focus on:
• Test scenarios are based on functional • Testing is based on non-functional
requirements requirements
• Test tools which allow to run simplistic tests • Load scenarios are driven by the
such as to send an order -> to verify a response technical characteristics in PROD.
• Prepare a functional library starting from • The main target of the tests is to assess
simple scenarios and adding more complex ones non-functional requirements: latency,
step by step capacity and fault tolerance
• Test scenarios are automated
• A lot of efforts are invested in correct
• A test Library is enhanced as we proceed latency measurements and determining
with its testing and production clients scenarios conditions when the system breaks
• An automated library will be reused as a • Further regressions are done to
regression test library which is executed compare system performance indicators
against each release after go live and can be with the previous releases
taken for consequent projects
4. What do we miss when we keep FT
and NFT separated ?
- Any complex system must sustain a certain level of concurrency. Some issues
appear when several events happen simultaneously (Race conditions).
Example 1: Clients send a CxlReplace request and approx. at the same time
the ME sends a fill back.
These 2 events meet at
the Order Management
server, and the state of
the order can get corrupted.
5. What do we miss when we keep FT
and NFT separated ?
- Production order books and various system object states are more complex
than in our FT scripts, and no amount of test cases can cover all possible
scenarios
Example 2: A User tries to
hit two different implied
orders coming from the
same parent order
which is being restated
in the middle of the process.
6. What do we miss when we keep FT
and NFT separated ?
- Components may behave differently under the load
Example 3: The System must support various
throttling limitations and the requirements of
the system behaviour. The correctness
of functional system behavior can be
tested only under the load.
- Integer memory overflow
Example 4: Turnover (VWAP) fields in the statistics messages and incremental
OrderID overflow;
* the picture was taken from https://encrypted-tbn1.gstatic.com/images?q=tbn:ANd9GcS9nlDdLAmuvmyRqqZsanbyfwL601nJlOvtHGEfJBS4DKnmSTANUA
7. What do we miss when we keep FT
and NFT separated ?
- Dynamic Mass events which happen during DLC
Example 5: Market Open,
Market Close, Auctions,..
The state of the system
at exact instance
when the system goes
from one session to another
should be verified with the load
applied against the system.
8. Detailed Verification Of The System Behavior Under Random Load
We need to make sure that we
provide extensive coverage
- basic consistency check
We need to make sure that the load test mimics - market data must be internally consistent
what happens in the production system - order book should be available and not be
We need to verify that the system different
works as expected
+ Recreation of exact steps of real production issues
- It is difficult to record multiple network streams
9. Performing Functional Testing Against The System Under The Load
to leave a part of the system intact by the load
to adjust functional tests to cope with the
activity of the load
to make scripts in case when we
perform
tests actively interacting with the load
* scenarios must be flexible
* sustain interaction with the script
10. Testing With The Baseline
The example of the cycle of a Clearing System
A detailed, step by step analysis
Talk Synopsis ‘ How a great QA Team can make a Disproportionate Contribution to Project Success’ Intro Explore project team dynamics – how the different parts of the team work together – focussing on the contribution that QA can make Have in mind financial services electronic trading platforms *
Talk Synopsis ‘ How a great QA Team can make a Disproportionate Contribution to Project Success’ Intro Explore project team dynamics – how the different parts of the team work together – focussing on the contribution that QA can make Have in mind financial services electronic trading platforms *