On Complex Software Systems Testing — Alexey Zverev, co-CEO & co-founder, Exactpro
Software Testing and Machine Learning
Mind the Gap. Applying Process Mining
Learning from Failure is not just for Humans
Dancing with Whales. Adaptive Log Classification System
On Traceability and the Illusion of Control
Building Partnerships
Demystifying DLT Testing One Network at a Time
Get the MOST from FIX
Georgia on My Mind
Build Software to Test Software — Iosif Itkin, co-CEO & co-founder, Exactpro
Exactpro FinTech Webinar - Global Exchanges FIX ProtocolIosif Itkin
Exactpro’s Global Exchanges Division training session on FIX Trading Protocol.
The essence of the FIX protocol and its place in the overall structure of network protocols, FIX message attributes and the internal data types of the protocol.
ClearTH Test Automation Framework: Case Study in IRS & CDS Swaps Lifecycle Mo...Iosif Itkin
Synchronize Europe
18th June 2019
Iosif Itkin, co-CEO and co-founder, Exactpro
Using the ISDA CDM Swaps application, simultaneously execute multiple end-to-end scenarios for DAML applications in capital markets - validate with actual contract data on ledger.
User-Assisted Log Analysis for Quality Control of Distributed Fintech Applica...Iosif Itkin
The First IEEE International Conference On Artificial Intelligence Testing (2019 IEEE AITest)
Iosif Itkin, Anna Gromova, Anton Sitnikov, Elena Treshcheva, Rostislav Yavorskiy, Evgenii Tsymbalov, Andrey Novikov and Kirill Rudakov
1 Exactpro, UK, Georgia, USA, Russia
2 Skolkovo Institute of Science and Technology, Russia
3 Higher School of Economics, Russia
Static analysis works for mission-critical systems, why not yours? Rogue Wave Software
Take a deep dive into the world of static code analysis (SCA) by immersing yourself into different analysis techniques, examples of the problems they find, and learning how SCA fits into various types of environments, from the developer desktop to the QA team. The goal is to provide a solid foundation for you to make the best decision for testing technology and process selection, including: Types of defects found by SCA;
Typical myths and barriers to adoption; and How SCA aligns to different testing maturity levels.
Accurate and Reliable What-If Analysis of Business Processes: Is it Achievable?Marlon Dumas
In this talk, I discuss the problem of how to discover simulation models that can be used to, accurately and reliably, predict the impact of a change on a business process, e.g. what-if we automate an activity? what-if 10% of our workers become unavailable? I focus on recent approaches that exploit the availability of data in enterprise systems to address this question.
Exactpro FinTech Webinar - Global Exchanges FIX ProtocolIosif Itkin
Exactpro’s Global Exchanges Division training session on FIX Trading Protocol.
The essence of the FIX protocol and its place in the overall structure of network protocols, FIX message attributes and the internal data types of the protocol.
ClearTH Test Automation Framework: Case Study in IRS & CDS Swaps Lifecycle Mo...Iosif Itkin
Synchronize Europe
18th June 2019
Iosif Itkin, co-CEO and co-founder, Exactpro
Using the ISDA CDM Swaps application, simultaneously execute multiple end-to-end scenarios for DAML applications in capital markets - validate with actual contract data on ledger.
User-Assisted Log Analysis for Quality Control of Distributed Fintech Applica...Iosif Itkin
The First IEEE International Conference On Artificial Intelligence Testing (2019 IEEE AITest)
Iosif Itkin, Anna Gromova, Anton Sitnikov, Elena Treshcheva, Rostislav Yavorskiy, Evgenii Tsymbalov, Andrey Novikov and Kirill Rudakov
1 Exactpro, UK, Georgia, USA, Russia
2 Skolkovo Institute of Science and Technology, Russia
3 Higher School of Economics, Russia
Static analysis works for mission-critical systems, why not yours? Rogue Wave Software
Take a deep dive into the world of static code analysis (SCA) by immersing yourself into different analysis techniques, examples of the problems they find, and learning how SCA fits into various types of environments, from the developer desktop to the QA team. The goal is to provide a solid foundation for you to make the best decision for testing technology and process selection, including: Types of defects found by SCA;
Typical myths and barriers to adoption; and How SCA aligns to different testing maturity levels.
Accurate and Reliable What-If Analysis of Business Processes: Is it Achievable?Marlon Dumas
In this talk, I discuss the problem of how to discover simulation models that can be used to, accurately and reliably, predict the impact of a change on a business process, e.g. what-if we automate an activity? what-if 10% of our workers become unavailable? I focus on recent approaches that exploit the availability of data in enterprise systems to address this question.
Monitoring Complex Systems: Keeping Your Head on Straight in a Hard WorldBrian Troutwine
This talk will provide motivation for the extensive instrumentation of complex computer systems and make the argument that such systems. This talk will provide practical starting points in Erlang projects and maintain a perspective on the human organization around the computer system. Brian will focus on getting started with instrumentation in a systematic way and follow up with the challenge of interpreting and acting on metrics emitted from a production system in a way which does not overwhelm operators’ ability to effectively control or prioritize faults in the system. He’ll use historical examples and case studies from my work to keep the talk anchored in the practical.
Talk objectives:
Brian hopes to convince the audience of two things:
* that monitoring and instrumentation is an essential component of any long-lived system and
* that it's not so hard to get started, after all.
He’ll keep a clear-eyed view of what works and is difficult in practice so that the audience can make a reasoned decision after the talk.
Target audience:
This talk would appeal to engineers with long-running production employments, operations folks and Erlangers in general.
These are the slides I used to present "MySQL Performance Schema" at FOSSASIA, 2015 Singapore. It gives an overview of Performance Schema and also explains how it could be used to diagnose issues using few use cases.
Autonomous Transaction Processing (ATP): In Heavy Traffic, Why Drive Stick?Jim Czuprynski
Autonomous Transaction Processing (ATP) - the second in the family of Oracle’s Autonomous Databases – offers Oracle DBAs the ability to apply a force multiplier for their OLTP database application workloads. However, it’s important to understand both the benefits and limitations of ATP before migrating any workloads to that environment. I'll offer a quick but deep dive into how best to take advantage of ATP - including how to load data quickly into the underlying database – and some ideas on how ATP will impact the role of Oracle DBA in the immediate future. (Hint: Think automatic transmission instead of stick-shift.)
Kafka summit SF 2019 - the art of the event-streaming appNeil Avery
Have you ever imagined what it would be like to build a massively scalable streaming application on Kafka, the challenges, the patterns and the thought process involved? How much of the application can be reused? What patterns will you discover? How does it all fit together? Depending upon your use case and business, this can mean many things. Starting out with a data pipeline is one thing, but evolving into a company-wide real-time application that is business critical and entirely dependent upon a streaming platform is a giant leap. Large-scale streaming applications are also called event streaming applications. They are classically different from other data systems; event streaming applications are viewed as a series of interconnected streams that are topologically defined using stream processors; they hold state that models your use case as events. Almost like a deconstructed realtime database.
In this talk, I step through the origins of event streaming systems, understanding how they are developed from raw events to evolve into something that can be adopted at an organizational scale. I start with event-first thinking, Domain Driven Design to build data models that work with the fundamentals of Streams, Kafka Streams, KSQL and Serverless (FaaS). Building upon this, I explain how to build common business functionality by stepping through patterns for Scalable payment processing Run it on rails: Instrumentation and monitoring Control flow patterns (start, stop, pause) Finally, all of these concepts are combined in a solution architecture that can be used at enterprise scale. I will introduce enterprise patterns such as events-as-a-backbone, events as APIs and methods for governance and self-service. You will leave talk with an understanding of how to model events with event-first thinking, how to work towards reusable streaming patterns and most importantly, how it all fits together at scale.
The art of the event streaming application: streams, stream processors and sc...confluent
Have you ever imagined what it would be like to build a massively scalable streaming application on Kafka, the challenges, the patterns and the thought process involved? How much of the application can be reused? What patterns will you discover? How does it all fit together? Depending upon your use case and business, this can mean many things. Starting out with a data pipeline is one thing, but evolving into a company-wide real-time application that is business critical and entirely dependent upon a streaming platform is a giant leap. Large-scale streaming applications are also called event streaming applications. They are classically different from other data systems; event streaming applications are viewed as a series of interconnected streams that are topologically defined using stream processors; they hold state that models your use case as events. Almost like a deconstructed realtime database. In this talk I step through the origins of event streaming systems, understanding how they are developed from raw events to evolve into something that can be adopted at an organizational scale. I start with event-first thinking, Domain Driven Design to build data models that work with the fundamentals of Streams, Kafka Streams, KSQL and Serverless (FaaS). Building upon this, I explain how to build common business functionality by stepping through patterns for Scalable payment processing Run it on rails: Instrumentation and monitoring Control flow patterns (start, stop, pause) Finally, all of these concepts are combined in a solution architecture that can be used at enterprise scale. I will introduce enterprise patterns such as events-as-a-backbone, events as APIs and methods for governance and self-service. You will leave talk with an understanding of how to model events with event-first thinking, how to work towards reusable streaming patterns and most importantly, how it all fits together at scale.
Troy Lea's presentation on Leveraging and Understanding Performance Data and Graphs.
The presentation was given during the Nagios World Conference North America held Sept 20-Oct 2nd, 2013 in Saint Paul, MN. For more information on the conference (including photos and videos), visit: http://go.nagios.com/nwcna
As test managers, we face challenging situations that require us to draw on our past experiences, principles, and good practices in order to have any chance at all for success. Michael Wasielczyk faced this challenge immediately after joining T. Rowe Price. He started his new job, responsible for managing the testing effort on a mission-critical data warehouse, and he had no experience managing a data warehouse testing effort. In addition, Michael found there were no formal practices for software development or testing. In fact, the establishment of an independent test team was a foreign concept to the corporate culture. A team of on-shore and off-shore testers arrived at the same time as Michael. Development was underway. A delivery date for the project had been communicated and could not be missed. So, how did he possibly succeed? Michael describes the challenges he faced over the following 205 days, the guiding principles he used, and the key decisions he made along the way. He shares the value of management support, tailoring best practices to the situation at hand, and understanding what battles are worth fighting for. Join Michael to learn how you can better prepare yourself when faced with a similar situation.
Foundations of Software Testing Lecture 4Iosif Itkin
This lecture is a part of the online course on Software Testing for Complex Intelligent Systems and Autonomous Vehicles. The course lectures provide the theoretical basics of testing autonomous systems based on artificial intelligence.
The fourth lecture of the course entitled Foundations of Software Testing reviews the ‘absence-of-errors fallacy’ and other principles of software testing, as well as the types and levels of software testing. The lecture also provides a fuller picture of the understanding of test objectives and methodologies by different schools of thought within the software testing domain.
QA Financial Forum London 2021 - Automation in Software Testing. Humans and C...Iosif Itkin
Speaker: Iosif Itkin, co-CEO & co-founder, Exactpro Systems
9th November 2021
Hilton Canary Wharf
Exactpro is an independent software testing business focused on mission-critical financial market infrastructures, primarily exchanges and clearing houses. In his presentation, Iosif will give a brief overview of research on the concept of model-based testing and the principal challenges of its application while testing complex distributed systems. He will also outline the broader context of interaction between humans and complex computer models.
More Related Content
Similar to EXTENT 2019: Exactpro Quality Assurance for Financial Market Infrastructures
Monitoring Complex Systems: Keeping Your Head on Straight in a Hard WorldBrian Troutwine
This talk will provide motivation for the extensive instrumentation of complex computer systems and make the argument that such systems. This talk will provide practical starting points in Erlang projects and maintain a perspective on the human organization around the computer system. Brian will focus on getting started with instrumentation in a systematic way and follow up with the challenge of interpreting and acting on metrics emitted from a production system in a way which does not overwhelm operators’ ability to effectively control or prioritize faults in the system. He’ll use historical examples and case studies from my work to keep the talk anchored in the practical.
Talk objectives:
Brian hopes to convince the audience of two things:
* that monitoring and instrumentation is an essential component of any long-lived system and
* that it's not so hard to get started, after all.
He’ll keep a clear-eyed view of what works and is difficult in practice so that the audience can make a reasoned decision after the talk.
Target audience:
This talk would appeal to engineers with long-running production employments, operations folks and Erlangers in general.
These are the slides I used to present "MySQL Performance Schema" at FOSSASIA, 2015 Singapore. It gives an overview of Performance Schema and also explains how it could be used to diagnose issues using few use cases.
Autonomous Transaction Processing (ATP): In Heavy Traffic, Why Drive Stick?Jim Czuprynski
Autonomous Transaction Processing (ATP) - the second in the family of Oracle’s Autonomous Databases – offers Oracle DBAs the ability to apply a force multiplier for their OLTP database application workloads. However, it’s important to understand both the benefits and limitations of ATP before migrating any workloads to that environment. I'll offer a quick but deep dive into how best to take advantage of ATP - including how to load data quickly into the underlying database – and some ideas on how ATP will impact the role of Oracle DBA in the immediate future. (Hint: Think automatic transmission instead of stick-shift.)
Kafka summit SF 2019 - the art of the event-streaming appNeil Avery
Have you ever imagined what it would be like to build a massively scalable streaming application on Kafka, the challenges, the patterns and the thought process involved? How much of the application can be reused? What patterns will you discover? How does it all fit together? Depending upon your use case and business, this can mean many things. Starting out with a data pipeline is one thing, but evolving into a company-wide real-time application that is business critical and entirely dependent upon a streaming platform is a giant leap. Large-scale streaming applications are also called event streaming applications. They are classically different from other data systems; event streaming applications are viewed as a series of interconnected streams that are topologically defined using stream processors; they hold state that models your use case as events. Almost like a deconstructed realtime database.
In this talk, I step through the origins of event streaming systems, understanding how they are developed from raw events to evolve into something that can be adopted at an organizational scale. I start with event-first thinking, Domain Driven Design to build data models that work with the fundamentals of Streams, Kafka Streams, KSQL and Serverless (FaaS). Building upon this, I explain how to build common business functionality by stepping through patterns for Scalable payment processing Run it on rails: Instrumentation and monitoring Control flow patterns (start, stop, pause) Finally, all of these concepts are combined in a solution architecture that can be used at enterprise scale. I will introduce enterprise patterns such as events-as-a-backbone, events as APIs and methods for governance and self-service. You will leave talk with an understanding of how to model events with event-first thinking, how to work towards reusable streaming patterns and most importantly, how it all fits together at scale.
The art of the event streaming application: streams, stream processors and sc...confluent
Have you ever imagined what it would be like to build a massively scalable streaming application on Kafka, the challenges, the patterns and the thought process involved? How much of the application can be reused? What patterns will you discover? How does it all fit together? Depending upon your use case and business, this can mean many things. Starting out with a data pipeline is one thing, but evolving into a company-wide real-time application that is business critical and entirely dependent upon a streaming platform is a giant leap. Large-scale streaming applications are also called event streaming applications. They are classically different from other data systems; event streaming applications are viewed as a series of interconnected streams that are topologically defined using stream processors; they hold state that models your use case as events. Almost like a deconstructed realtime database. In this talk I step through the origins of event streaming systems, understanding how they are developed from raw events to evolve into something that can be adopted at an organizational scale. I start with event-first thinking, Domain Driven Design to build data models that work with the fundamentals of Streams, Kafka Streams, KSQL and Serverless (FaaS). Building upon this, I explain how to build common business functionality by stepping through patterns for Scalable payment processing Run it on rails: Instrumentation and monitoring Control flow patterns (start, stop, pause) Finally, all of these concepts are combined in a solution architecture that can be used at enterprise scale. I will introduce enterprise patterns such as events-as-a-backbone, events as APIs and methods for governance and self-service. You will leave talk with an understanding of how to model events with event-first thinking, how to work towards reusable streaming patterns and most importantly, how it all fits together at scale.
Troy Lea's presentation on Leveraging and Understanding Performance Data and Graphs.
The presentation was given during the Nagios World Conference North America held Sept 20-Oct 2nd, 2013 in Saint Paul, MN. For more information on the conference (including photos and videos), visit: http://go.nagios.com/nwcna
As test managers, we face challenging situations that require us to draw on our past experiences, principles, and good practices in order to have any chance at all for success. Michael Wasielczyk faced this challenge immediately after joining T. Rowe Price. He started his new job, responsible for managing the testing effort on a mission-critical data warehouse, and he had no experience managing a data warehouse testing effort. In addition, Michael found there were no formal practices for software development or testing. In fact, the establishment of an independent test team was a foreign concept to the corporate culture. A team of on-shore and off-shore testers arrived at the same time as Michael. Development was underway. A delivery date for the project had been communicated and could not be missed. So, how did he possibly succeed? Michael describes the challenges he faced over the following 205 days, the guiding principles he used, and the key decisions he made along the way. He shares the value of management support, tailoring best practices to the situation at hand, and understanding what battles are worth fighting for. Join Michael to learn how you can better prepare yourself when faced with a similar situation.
Similar to EXTENT 2019: Exactpro Quality Assurance for Financial Market Infrastructures (20)
Foundations of Software Testing Lecture 4Iosif Itkin
This lecture is a part of the online course on Software Testing for Complex Intelligent Systems and Autonomous Vehicles. The course lectures provide the theoretical basics of testing autonomous systems based on artificial intelligence.
The fourth lecture of the course entitled Foundations of Software Testing reviews the ‘absence-of-errors fallacy’ and other principles of software testing, as well as the types and levels of software testing. The lecture also provides a fuller picture of the understanding of test objectives and methodologies by different schools of thought within the software testing domain.
QA Financial Forum London 2021 - Automation in Software Testing. Humans and C...Iosif Itkin
Speaker: Iosif Itkin, co-CEO & co-founder, Exactpro Systems
9th November 2021
Hilton Canary Wharf
Exactpro is an independent software testing business focused on mission-critical financial market infrastructures, primarily exchanges and clearing houses. In his presentation, Iosif will give a brief overview of research on the concept of model-based testing and the principal challenges of its application while testing complex distributed systems. He will also outline the broader context of interaction between humans and complex computer models.
Exactpro FinTech Webinar - Global Exchanges Test OraclesIosif Itkin
Global Exchanges series webinar to discuss Test Oracles. A test oracle is a mechanism for determining whether a test has passed or failed. The use of oracles involves comparing the output(s) of the system under test for a given test-case input, to the output(s) that the oracle determines the product should have. We will review various types of test oracles using examples from Exactpro’s Global Exchanges division projects and protocol-based interactions in trading systems.
Operational Resilience in Financial Market InfrastructuresIosif Itkin
A4Q World Congress 13-16 April 2021
Iosif Itkin
Exactpro provides independent software testing services for mission critical technology that underpins global financial markets – exchanges and clearing houses. Half of the top 20 global exchange groups on all continents around the globe rely on processes, platforms and people from Exactpro to improve their quality and reliability. The company has spent the last 11 years studying operational resilience in this crucial sector. The presentation will outline the key principles for software testing of the systems that process hundreds of millions of orders per day with roundtrip latencies below one hundred microseconds.
20 Simple Questions from Exactpro for Your Enjoyment This Holiday SeasonIosif Itkin
Warmest wishes for a happy holiday season and a wonderful New Year!
We look forward to our continued collaboration in 2020. Thank you for your support.
QA-Financial Forum 2019 in New York
13 November
Iosif Itkin, CEO and co-founder
Elena Treshcheva, Business Development Manager and Researcher
An October 2019 survey by BoE and FCA found that ML in financial organizations has already passed an initial development phase, and the usage of live ML applications is about to dramatically increase over the next three years. Artificial Intelligence systems are used in market surveillance, they are providing intellectual analysis of news feeds, and they are an important part of the conversational agents facing users and helping them with their business needs from identity verification to trading and portfolio management. How to ensure that an AI-powered system is up to its task? And what would that mean from the software testing perspective?
EXTENT Talks 2019 Tbilisi: Failover and Recovery Test Automation - Ivan ShamraiIosif Itkin
Ivan Shamray, Senior NFT Analyst, Exactpro
20 April 2019 EXTENT Talks, Tbilisi, Georgia
Tbilisi QA Community
EXTENT Talks is a meeting place for IT specialists working in various industries and seeking professional growth, practitioners from IT firms, as well as Quality Assurance enthusiasts of all backgrounds interested in actively participating in local IT events.
EXTENT Talks QA Community Tbilisi 20 April 2019 - Conference OpenIosif Itkin
EXTENT Talks is a meeting place for IT specialists working in various industries and seeking professional growth, practitioners from IT firms, as well as Quality Assurance enthusiasts of all backgrounds interested in actively participating in local IT events. The first EXTENT Talks were held in Tbilisi on 22 February 2019, initiating the creation a QA Community in Tbilisi and laying a foundation for an international platform for exchanging experience and knowledge in the field of software testing, development and IT. The program of the inaugural event included presentations on ISTQB, Software Testing, and Agile methodology from senior specialists. The next EXTENT Talks in Tbilisi will take place on 20 April 2019.
Speakers: Iosif Itkin, CEO and Co-Founder and Elena Treshcheva, Business Development Manager and Researcher - Exactpro
Exactpro provides software testing services for mission-critical technology that underpins global financial markets. Exactpro clients are regulated by FCA, Bank of England and their counterparts from other countries. During this session, Elena and Iosif will talk about end-to-end software testing for post-trade systems in financial market infrastructures. What are the key challenges in quality assurance at this scale? What kind of cognitive biases affect SDLC? How precise is the knowledge about the systems under test? What constitutes good test evidence? How to deal with complexity in regulated environments?
Behaviour Driven Development: Oltre i limiti del possibileIosif Itkin
The QA Financial Forum: Milan 2019
23 January at the Excelsior Hotel Gallia.
Anna-Maria Lukina, Exactpro Business Development Director
The QA Financial Forum: Milan is one of the leading fintech conferences in Italy. The event focuses on the latest achievements in software risk management and automation of software testing. The predominant theme of the Milan event will be Quality Assurance for the entire Software Development Life Cycle (SDLC).
The topics under discussion will feature:
- Technologies for Automation & AI
- DevOps & CI/CD
- Value Stream Management
- Test Data Management
- Regulatory Compliance
- App Security & DevSecOps
- Testing and quality assurance of Blockchain platforms
The official language of the event is Italian.
On 17th January 2018 Exactpro successfully completed a management buyout from London Stock Exchange Group (LSEG), signed a new multi-year master services agreement with LSEG, and opened its head office in London.
What else has happened in 2018?
I wanted to take the opportunity to reflect on what has been an unusual year for Exactpro.
Integration front to back - Mr. Custodian tear down that wall
The scope of the application level has been continuous extended over the years, albeit with a focus on the area of pre-trade and trade.
Recently, there has been an increased interest to move further into the area of post-trade which is predominantly driven by the ISO 20022 standard. Is there really a need for new FIX messages in areas such as payments and
what are the integration problems needing a resolution?
Panellists
- Iosif Itkin, CEO, Exactpro
- Jim Northey, Co-Chair Global Technical Committee, Americas Region, FIX Trading Community, Chair Elect, ISO TC68 Financial
Services Technical Committee, and Consultant and Industry Standards Liaison, Itiviti
- Barry Young, Director, Aladdin Product Manager, BlackRock
BDD. The Outer Limits. Iosif Itkin at Youcon (in Russian)Iosif Itkin
Exactpro is supporting the 3rd annual IT-conference YouCon to take place on 14th October in Saratov, Russia. Over 900 programmers, systems engineers and architects, software QA engineers, and marketing specialists will gather to discuss the latest trends in programming technology. It is the largest IT industry event in Saratov.
Iosif Itkin, CEO of Exactpro, part of London Stock Exchange Group, will deliver a "BDD. The Outer Limits" presentation named after Iosif's favorite Sci-Fi series.
The topics to be covered are:
Behavior Driven Development concepts
Applying BDD in trading and clearing systems
Specification by Example and using production data
Combining Model-based testing and BDD
The Outer Limits
There will be an opportunity to ask questions, share thoughts and expertise in BDD, or just chat with a representative at the Exactpro stand at any time during the event.
Don't miss out, stop by and ask how you can get your Exactpro souvenir :)
We look forward to meeting you there!
#Exactpro #Youconsaratov
Sibos 2017: Disruptive functional testing - the next frontier in post-trade s...Iosif Itkin
16 October 2017
Sibos Toronto
Presentation by Iosif Itkin, Exactpro CEO and co-founder
London Stock Exchange Group
Disruptive functional testing - the next frontier in post-trade systems verification
The operators of complex back office and post-trade systems interconnected by SWIFT and other communication protocols are trying to keep pace with the front office technology advances, rising data volumes and changing regulatory landscape. The industry actively adopts flexible development methodologies to speed up the software delivery process. What aspects of software quality are not covered by regular performance / failover testing and the agile development life cycle? Disruptive functional testing is a way to prepare your mission-critical infrastructure for any challenges the future may bring. Ideas for this presentation are drawn from our work with LCH, T2S, clearing and settlement systems worldwide.
Using Cluster Analysis for Characteristics Detection in Software Defect ReportsIosif Itkin
Anna Gromova and Olga Moskaleva, Exactpro Researchers, participated in the Sixth International Conference on Analysis of Images, Social Networks and Texts, AIST 2017
AIST is a conference on the Analysis of Images, Social Networks, and Texts. This year, it was held at Moscow Polytechnic University on June 27-29. The conference is intended for computer scientists and industry professionals whose research interests involve applications of data mining and machine learning techniques to various problem domains: image processing, analysis of social networks, and natural language processing. AIST boasts its interdisciplinary nature, bridging the gap between researchers and practitioners.
All research papers submitted to the conference went through a double-blind reviewing process and were reviewed by at least three program committee members. The accepted papers were presented by their authors on July 27 and 28.
Anna Gromova’s research on Using Cluster Analysis for Characteristics Detection in Software Defect Reports was among this year’s accepted papers. Anna, who is an Exactpro researcher, presented her work in the General Topics of Data Analysis section of the conference.
Anna’s research focuses on the metrics and predictions of defect reports. In her research paper, Anna explores the workings of how extended bug attribute analysis can effectively reveal software weaknesses and improve the testing strategy, and how machine learning techniques can facilitate the process of building an automated recommendation system for project managers and QA team leads. Such a system would help them get a better picture of the risks associated with software defects, as well as obtain more accurate information about a bug's lifecycle.
“In future, we are planning to build an automated recommendation system based on the cluster analysis of our defect reports. A system like that could be used by all QA team members to improve the existing processes of developing the testing strategies and plans”, says Anna Gromova, Exactpro Researcher.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
Search and Society: Reimagining Information Access for Radical FuturesBhaskar Mitra
The field of Information retrieval (IR) is currently undergoing a transformative shift, at least partly due to the emerging applications of generative AI to information access. In this talk, we will deliberate on the sociotechnical implications of generative AI for information access. We will argue that there is both a critical necessity and an exciting opportunity for the IR community to re-center our research agendas on societal needs while dismantling the artificial separation between the work on fairness, accountability, transparency, and ethics in IR and the rest of IR research. Instead of adopting a reactionary strategy of trying to mitigate potential social harms from emerging technologies, the community should aim to proactively set the research agenda for the kinds of systems we should build inspired by diverse explicitly stated sociotechnical imaginaries. The sociotechnical imaginaries that underpin the design and development of information access technologies needs to be explicitly articulated, and we need to develop theories of change in context of these diverse perspectives. Our guiding future imaginaries must be informed by other academic fields, such as democratic theory and critical theory, and should be co-developed with social science scholars, legal scholars, civil rights and social justice activists, and artists, among others.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
"Impact of front-end architecture on development cost", Viktor TurskyiFwdays
I have heard many times that architecture is not important for the front-end. Also, many times I have seen how developers implement features on the front-end just following the standard rules for a framework and think that this is enough to successfully launch the project, and then the project fails. How to prevent this and what approach to choose? I have launched dozens of complex projects and during the talk we will analyze which approaches have worked for me and which have not.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
11. Workflows comparison:
- Gap analysis/coveragein the test library/run;
- Efficiency in test execution;
- Edge cases/raceconditions;
- Are all the use cases from Prod/UAT covered?
Specifications
Test runs (Logs,msgs)
FT / NFT
UAT/Production
(Logs, msgs)
Single database
creation
Process mining
model
Process mining workflows
19. What is Sailfish?
#step #description #user_name #action #message_type SecurityID OrdType Side OrderQty Price Status
Test Case 1: Trade test case start
1 Send BUY Order with Qty = 1000, Price = 10 (O1) User_1 send NewOrderSingle 12345 LIMIT BUY 1000 10
2 Receive Execution Report with Status = NEW (O1) User_1 receive ExecutionReport 12345 LIMIT BUY 1000 10 NEW
3 Send SELL Order with Qty = 1000, Price = 10 (O2) User_2 send NewOrderSingle 12345 LIMIT SELL 1000 10
4 Receive Execution Report with Status = FILLED (O2) User_2 receive ExecutionReport 12345 LIMIT SELL 1000 10 FILLED
5 Receive Execution Report with Status = FILLED (O1) User_1 receive ExecutionReport 12345 LIMIT BUY 1000 10 FILLED
test case end
20. The main actions of Sailfish
#step #description #user_name #action #message_type SecurityID OrdType Side OrderQty Price Status
Test Case 1: Trade test case start
1 Send BUY Order with Qty = 1000, Price = 10 (O1) User_1 send NewOrderSingle 12345 LIMIT BUY 1000 10
2 Receive Execution Report with Status = NEW (O1) User_1 receive ExecutionReport 12345 LIMIT BUY 1000 10 NEW
3 Send SELL Order with Qty = 1000, Price = 10 (O2) User_2 send NewOrderSingle 12345 LIMIT SELL 1000 10
4 Receive Execution Report with Status = FILLED (O2) User_2 receive ExecutionReport 12345 LIMIT SELL 1000 10 FILLED
5 Receive Execution Report with Status = FILLED (O1) User_1 receive ExecutionReport 12345 LIMIT BUY 1000 10 FILLED
test case end
21. Passed and Failed test steps
send NewOrderSingle (PASSED) [0.001s]
- receive ExecutionReport (PASSED) [0.001s]:
+ Input Parameters
- Verification: Message (PASSED):
Field Expected Result Actual Result Status
... ... ... ...
OrdType LIMIT LIMIT PASSED
Status NEW NEW PASSED
... ... ... ...
22. Passed and Failed test steps
send NewOrderSingle (PASSED) [0.001s]
- receive ExecutionReport (FAILED) [0.421s]:
+ Input Parameters
+ Verification: Similar message [1]. Failed/Passed/Conditionally Passed/NA: 4/26/0/6 (FAILED)
+ Verification: Similar message [2]. Failed/Passed/Conditionally Passed/NA: 2/28/0/6 (FAILED)
+ Verification: Similar message [3]. Failed/Passed/Conditionally Passed/NA: 1/29/0/6 (FAILED)
- Verification: Similar message [4]. Failed/Passed/Conditionally Passed/NA: 1/29/0/6 (FAILED):
Field Expected Result Actual Result Status
... ... ... ...
OrdType LIMIT LIMIT PASSED
Status NEW REJECTED FAILED
... ... ... ...
23. Supervised Machine Learning through
unpleasant experience
send NewOrderSingle (PASSED) [0.001s]
- receive ExecutionReport (FAILED) [0.421s]:
+ Input Parameters
+ Verification: Similar message [1]. Failed/Passed/Conditionally Passed/NA: 4/26/0/6 (FAILED)
+ Verification: Similar message [2]. Failed/Passed/Conditionally Passed/NA: 2/28/0/6 (FAILED)
+ Verification: Similar message [3]. Failed/Passed/Conditionally Passed/NA: 1/29/0/6 (FAILED)
- Verification: Similar message [4]. Failed/Passed/Conditionally Passed/NA: 1/29/0/6 (FAILED):
Field Expected Result Actual Result Status
... ... ... ...
OrdType LIMIT LIMIT PASSED
Status NEW REJECTED FAILED
... ... ... ...
24. Supervised Machine Learning through
unpleasant experience
send NewOrderSingle (PASSED) [0.001s]
- receive ExecutionReport (FAILED) [0.421s]:
+ Input Parameters
- Verification: Similar message [1]. Failed/Passed/Conditionally Passed/NA: 1/29/0/6 (FAILED): 89%
+ Verification: Similar message [2]. Failed/Passed/Conditionally Passed/NA: 4/26/0/6 (FAILED) 81%
+ Verification: Similar message [3]. Failed/Passed/Conditionally Passed/NA: 2/28/0/6 (FAILED) 54%
+ Verification: Similar message [4]. Failed/Passed/Conditionally Passed/NA: 1/29/0/6 (FAILED) 42%
Field Expected Result Actual Result Status
... ... ... ...
OrdType LIMIT LIMIT PASSED
Status NEW REJECTED FAILED
... ... ... ...
26. Supervised Machine Learning: going forward
This test step failed
with the following
results...
This test step failed
with the following
results…
And the reason is...
27. Dancing with Whales:
Adaptive Log Classification System
Elena Treshcheva,
Business Development Manager,
Researcher, Exactpro
32. WhaleShark
WhaleShark
The First IEEE International Conference on Artificial Intelligence Testing
(IEEE AITest 2019), April 4-9 2019, San Francisco East Bay, CA, USA
User-Assisted Log Analysis for Quality Control of Distributed Fintech
Systems
Iosif Itkin, Anna Gromova, Anton Sitnikov, Rostislav Yavorskiy, Evgenii Tsymbalov,
Andrey Novikov and Kirill Rudakov.
33. One error class - different text strings
text text text text
0100101010001010000
ERROR
text text text text
text text text text
text text text text
text text text text
FATAL……………………………………
text text text text
text text text text
It’s so cold and
dark... I’m scared...
34. Step 1: Normalization
ERROR 07 Aug 2018 13:04:39 - Unable to load messages from file
'/UAT/collectors/mq1'
ERROR 25 Sep 2018 18:24:55 - Unable to load messages from file
'/prod/collectors/mq3'
ERROR 16 Oct 2018 15:15:10 - Unable to load messages from file
'/prod/collectors/mq5'
35. Step 1: Normalization
ERROR 07 Aug 2018 13:04:39 - Unable to load messages from file
'/UAT/collectors/mq1'
ERROR 25 Sep 2018 18:24:55 - Unable to load messages from file
'/prod/collectors/mq3'
ERROR ** *** **** **:**:** - Unable to load messages from file
'/*/*/*'
ERROR 16 Oct 2018 15:15:10 - Unable to load messages from file
'/prod/collectors/mq5'
41. When I was 5 years old, my mother always
told me that happiness was the key to life.
When I went to school, they asked me
what I wanted to be when I grew up.
I wrote down ‘happy’. They told me I didn’t
understand the assignment, and I told
them they didn’t understand life.
John Lennon
42. Hey Stress, don't make it bad
Sherman, G. D., Lee, J. J., Cuddy, A. J. C., Renshon, J., Oveis, C., Gross, J. J., &
Lerner, J. S. (2012). Leadership is associated with lower levels of stress.
Proceedings of the National Academy of Sciences, 109(44), 17903–17907.
43. Fenton-O’Creevy, M., Nicholson, N., Soane, E.,
& Willman, P. (2003). “Trading on illusions:
Unrealistic perceptions of control and trading
performance”. Journal of Occupational and
Organizational Psychology, 76(1), 53–68.
Imagine all the brokers trading on...
45. Software testing is relentless
learning, continuous
improvement and keeping
abreast of cutting-edge
technologies.
The research and development
team at Exactpro uses machine
learning methods of intellectual
data analysis to create next-
generation program analysis
tools.
We work in an agile environment
collaborating with a multinational
team of software developers.
R&D Case Study: Cradle Test Database
for Machine Learning
46. They told me I didn’t understand the
assignment and the test report does not
make anyone happy. I told them they
didn’t understand software testing.
Software testing is our life.
48. The complexity of post-trade
- There is a multitude of components in modern
complex post-trade infrastructures;
- Upstream and downstream system dependency;
- The participant structure is very complex;
- Trade/Position/Account life cycle;
- The number of Asset Classes may vary;
- The complexity of the Risk calculation process;
- Access via a set of API endpoints.
58. Exactpro Test Framework and Test Approach for
DLT-based system
3 steps on the ledger side:
1st - Initialisation of positions
2nd - Trade creation and participant
notification
3rd - Positions updates
Across each and every step the test
framework provides different methods
to validate the endpoints, inputs and
outputs.
The actual business flow steps and
validation check items are transformed
into ClearTH matrix after execution of
which the deviations from the expected
behaviour can be analysed.
59. ClearTH and Woodpecker for DLT
is an Exactpro test harness able to simultaneously
execute multiple end-to-end test scenarios in
batches.
● Supports multiple protocols (SWIFT, FIX,
Market Data, etc.)
● Automatically runs test scripts
● Executes concurrent automated tests
● Allows to create multiple-day test scenarios
● Has integrated scheduler and simulators
● Verifies each stage of the system daily life cycle
72. It is not called Georgia
• Territory: 69,700 sq km
• Population: ~4 million
• Capital: Tbilisi (1.5 million)
Five Cross Flag
ძალა ერთობაშია
Strength is in Unity
Sakartvelo საქართველო
73. Why Sakartvelo
To provide additional level of security to our Clients and Exactpro business, we
have decided to diversify our delivery locations by country.
Georgia is very well ranked in terms of economic freedom, ease of doing business
and corruption.
74. A Warm Location
19 September 2018
As part of executing our
strategy of global growth as
a company focused on
software testing and building
software to test our clients’
software, we launched
Exactpro Systems Georgia
and our new project delivery
center in Tbilisi.
77. Software Testing is Relentless Learning -
GeoSTQB
• Increase the awareness about ISTQB among
testers within different companies and universities
• Promote professionalism in Software Testing in
Georgia by implementing an internationally
recognized certified tester scheme
• Establish certification of IT professionals in Georgia
according to ISTQB standards
• Support talented and promising testers
22 February 2019
78. Software Testing is Relentless Learning -
Partnerships
Exactpro has recently joined the ISTQB® Partner Program as a platinum partner
alongside 74 other companies.
Joining the ISTQB® Partner Program allows us to play a bigger role in the
software testing community and to highlight our goal of providing accessible,
internationally relevant high-quality education to our staff
July 2019
79. Software Testing is Relentless Learning
The International Conference on Software
Testing, Machine Learning and Complex
Process Analysis (TMPA-2019) –
Tbilisi, Georgia on 7-9 November 2019.
Focus on the application of modern methods of
data science to the analysis of software quality.
www.tsu.ge
90. Changes in Software
• Extend regression libraries
• Reduce the gaps in the test coverage
• Build faster software to execute more tests
• Run tests concurrently and under load
• Repeat the same tests multiple times
• Introduce more diversity in tools and data sets
• Intensify Use of Chaos Engineering
93. Exactpro Test Cloud
• Cradle for the next generation of test tools
• Built-in test coverage and compliance reporting
• Pricing tailored to massive defect mining processes
• Located in UK, EU, US to satisfy data regulation
94. Exactpro Plans
• Cradle for the next generation of test tools
• Built-in test coverage and compliance reporting
• Pricing tailored to massive defect mining processes
• Located in UK, EU, US to satisfy data regulation
• Consider raising capital for it in 2021