QA-Financial Forum 2019 in New York 13 November Iosif Itkin, CEO and co-founder Elena Treshcheva, Business Development Manager and Researcher An October 2019 survey by BoE and FCA found that ML in financial organizations has already passed an initial development phase, and the usage of live ML applications is about to dramatically increase over the next three years. Artificial Intelligence systems are used in market surveillance, they are providing intellectual analysis of news feeds, and they are an important part of the conversational agents facing users and helping them with their business needs from identity verification to trading and portfolio management. How to ensure that an AI-powered system is up to its task? And what would that mean from the software testing perspective?

1. Testing the Intelligence of your AI
Iosif Itkin, CEO and co-founder
Elena Treshcheva, Researcher
Iosif Itkin, CEO and co-founder
Elena Treshcheva, Business Development Manager and Researcher

2. Exactpro Overview
● A specialist firm focused on functional and non-functional testing of
exchanges, clearing houses, depositories, trade repositories and other
financial market infrastructures.
● Incorporated in 2009 with 10 people, our company has experienced
significant growth and is now employing over 550 specialists.
● We were part of the London Stock Exchange Group (LSEG) from May 2015
till January 2018. Exactpro management buyout from LSEG was successfully
completed in January 2018. We are headquartered in the UK and have
operations in the US, Georgia and Russia.

3. Exactpro Client Network

4. AI-based Systems in Finance
Machine Learning in financial organizations:
- already passed an initial development phase
- the usage of live ML applications is about to
dramatically increase over the next three years
https://www.bankofengland.co.uk/-
/media/boe/files/report/2019/machine-learning-in-uk-
financial-services.pdf

5. AI-based Systems in Finance
Machine Learning in financial organizations:
- already passed an initial development phase
- the usage of live ML applications is about to
dramatically increase over the next three years
● Market Surveillance Systems
● Conversational Assistants
● Algo Trading Systems
● Pricing Calculators
● Machine Readable News
● Insurance Claims
https://www.bankofengland.co.uk/-
/media/boe/files/report/2019/machine-learning-in-uk-
financial-services.pdf

6. AI-based Systems’ Quality Characteristics:
- Ability to learn: The capacity of the system to learn from use for the
system itself, or data and events it is exposed to.
- Trustworthiness: The degree to which the system is trusted by
stakeholders, for example a health diagnostic
- Ability to generalize: The ability of the system to apply to different
and previously unseen scenarios.
A4Q AI and Software Testing
Foundation
Syllabus https://www.gasq.org/en/exam-modules/a4q-ai-and-software-testing.html
Testing the
Intelligence
of your AI

7. Ability to Learn:
https://www.deeplearning.ai/
• Training set — Which you run your learning algorithm on.
• Development set — Which you use to tune parameters, select
features, and make other decisions regarding the learning algorithm.
Sometimes also called the hold-out cross validation set.
• Test set — which you use to evaluate the performance of the algorithm,
but not to make any decisions regarding what learning algorithm or
parameters to use.

8. Trustworthiness:
https://innovation.defense.gov/ai/
During the DIB’s quarterly public meeting on October 31, 2019, the DIB
members voted to approve the proposed AI Principles.

9. Trustworthiness:
https://www.mas.gov.sg/news/media-releases/2019/mas-partners-financial-
industry-to-create-framework-for-responsible-use-of-ai

10. Ability to Generalize: Scope of End-to-End and Negative Testing

11. Congruence bias
Confirmation
bias
Law of triviality
Zero-risk bias
Anthropocentric
thinking
Illusion of control
Cognitive Biases Affecting Software Testing of AI-based Systems
Automation bias

12. AI-based Systems: Machine-Readable News

13. Confirmation Bias

14. Salman, I. (2016). Cognitive biases in software quality and testing. Proceedings of
the 38th International Conference on Software Engineering Companion - ICSE ’16.
Pp. 823-826.

15. Mohanani, R., Salman, I., Turhan, B., Rodríguez, P., & Ralph, P. (2018).
Cognitive Biases in Software Engineering: A Systematic Mapping Study.
IEEE Transactions on Software Engineering

16. AI-based Systems: Conversational Assistants (Chatbots)
Chatbot

17. Anthropocentric Bias
We should not
humanize computers.

18. Anthropocentric bias
They dislike it a lot!

19. Anthropocentric Bias: Testing a Mine-Defusing Robot

20. Anthropocentric Bias: Why We Treat Robots Like Humans
Darling, Kate and Nandy, Palash and Breazeal,
Cynthia “Empathic Concern and the Effect of
Stories in Human-Robot Interaction” (2015).
Proceedings of the IEEE International Workshop on
Robot and Human Communication (ROMAN),
2015. 6 p.
https://www.ted.com/talks/kate_darling_why_we_ha
ve_an_emotional_connection_to_robots

21. Anthropocentric Bias: Testing Chatbots
Anaphora / Context
Human: I bought 500 Company X shares two years ago. The stocks’
cost was 60,000 USD. What’s their today’s cost?
Chatbot: What currency would you like to have for the rate? X
Spelling / overall correctness
Human: What is the setlement date of the tradeId XXX??
Chatbot: ???

22. AI-based Systems: Algo Trading

23. Congruence Bias
Direct
Testing

24. Indirect Testing Methods
Information
extraction and
Machine learning
End-to-End
Automated Test
Library
Whatever it
takes!
Test execution
data and log
analysis
Passive Testing
Whatever it
takes!

25. Applications of the Proposed Approach
https://unsplash.com/search/photos/san-francisco
The First IEEE International Conference on Artificial
Intelligence Testing (IEEE AITest 2019), April 4-9 2019,
San Francisco East Bay, CA, USA
User-Assisted Log Analysis for Quality
Control of Distributed Fintech Systems
Iosif Itkin, Anna Gromova, Anton Sitnikov, Rostislav Yavorskiy,
Evgenii Tsymbalov, Andrey Novikov and Kirill Rudakov.

26. AI-based Systems: Pricing Calculator

27. Law of Triviality (the Bike-Shed Effect)

28. Automation Bias

29. AI-based Systems: Fraud Detection and Market Surveillance

30. Build Software to Test Software
Click to know more about
Exactpro Test Tools

31. AI-based Systems: Insurance Claims

32. Zero-Risk Bias

33. Non-deterministic Systems: Financial Market Infrastructures

34. The Illusion of Control and Happiness
Sherman, G. D., Lee, J. J., Cuddy, A. J. C., Renshon, J., Oveis, C., Gross, J. J., &
Lerner, J. S. (2012). Leadership is associated with lower levels of stress.
Proceedings of the National Academy of Sciences, 109(44), 17903–17907.

35. Fenton-O’Creevy, M., Nicholson, N., Soane, E.,
& Willman, P. (2003). “Trading on illusions:
Unrealistic perceptions of control and trading
performance”. Journal of Occupational and
Organizational Psychology, 76(1), 53–68.
The Illusion of Control and Performance

36. Thank you