The document discusses parameterized model checking of fault-tolerant distributed algorithms. It presents threshold-guarded distributed algorithms that use threshold guards like "if received message from at least t+1 processes". It describes abstracting such algorithms using parametric interval abstraction to reduce verification of infinite instances to a single finite model checking problem by abstracting data and counters. Spurious counterexamples found during model checking are then refined to prove correctness across all algorithm instances.
Dynamic Binary Analysis and Obfuscated Codes Jonathan Salwan
At this presentation we will talk about how a DBA (Dynamic Binary Analysis) may help a reverse engineer to reverse obfuscated code. We will first introduce some basic obfuscation techniques and then expose how it's possible to break some stuffs (using our open-source DBA framework - Triton) like detect opaque predicates, reconstruct CFG, find the original algorithm, isolate sensible data and many more... Then, we will conclude with a demo and few words about our future work.
How Triton can help to reverse virtual machine based software protectionsJonathan Salwan
The first part of the talk is going to be an introduction to the Triton framework to expose its components and to explain how they work together. Then, the second part will include demonstrations on how it's possible to reverse virtual machine based protections using taint analysis, symbolic execution, SMT simplifications and LLVM-IR optimizations.
Dynamic Binary Analysis and Obfuscated Codes Jonathan Salwan
At this presentation we will talk about how a DBA (Dynamic Binary Analysis) may help a reverse engineer to reverse obfuscated code. We will first introduce some basic obfuscation techniques and then expose how it's possible to break some stuffs (using our open-source DBA framework - Triton) like detect opaque predicates, reconstruct CFG, find the original algorithm, isolate sensible data and many more... Then, we will conclude with a demo and few words about our future work.
How Triton can help to reverse virtual machine based software protectionsJonathan Salwan
The first part of the talk is going to be an introduction to the Triton framework to expose its components and to explain how they work together. Then, the second part will include demonstrations on how it's possible to reverse virtual machine based protections using taint analysis, symbolic execution, SMT simplifications and LLVM-IR optimizations.
How to calculate the bus voltages using Power world Simulator with the help of Gauss-Seidel iteration Method. Also calculate the Mismatches and Y-bus. I explained each and every step.
"Good Luck"
An abstract of my final project in bachelor\'s degree in Mathematics: interpolation and approximation of curves and surfaces with B-Spline basis functions
Having been established in 1998 to serve in Ground and Foundation Engineering, Temeltaş has been performing qualified projects and moving the present, from its ground, to the future based on its 15 years of experience, its team consisting of professionals and its service understanding featuring high standards.
In accordance with financial and technological conditions, Temeltaş increases its mobility in changing environments and it produces right and economical solutions by its high-tech modern machinery and equipment from design to completion of projects.
Having achieved, since its establishment, the appreciation and satisfaction of domestic and foreign auditing companies in the works it carried out in Turkey and abroad, Temeltaş has adopted it as its principle to maintain its reliable, quality and economical applications with an increasing performance.
Event Stream Processing with Multiple ThreadsSylvain Hallé
We present an extension to the BeepBeep 3 event stream engine that allows the use of multiple threads during the evaluation of a query. Compared to the single-threaded version of BeepBeep, the allocation of just a few threads to specific portions of a query provides improvement in terms of throughput.
INTRUSION DETECTION SYSTEM USING DISCRETE FOURIER TRANSFORM WITH WINDOW FUNCTIONIJNSA Journal
An Intrusion Detection System (IDS) is countermeasureagainst network attack. There are mainly two typesof detections; signature-based and anomaly-based. And thereare two kinds of error; false negative and false positive. Indevelopment of IDS, establishment of a method to reduce suchfalse is a major issue. In this paper, we propose a new anomaly-baseddetection method using Discrete Fourier Transform (DFT)with window function. In our method, we assume fluctuation ofpayload in ordinary sessions as random. On the other hand, we cansee fluctuation in attack sessions have bias. From the viewpointof spectrum analysis based on such assumption, we can find outdifferent characteristic in spectrum of attack sessions. Using thecharacteristic, we can detect attack sessions. Example detectionagainst Kyoto2006+ dataset shows 12.0% of false positive at most,and 0.0% of false negative.
INTRUSION DETECTION SYSTEM USING DISCRETE FOURIER TRANSFORM WITH WINDOW FUNCTIONIJNSA Journal
An Intrusion Detection System (IDS) is countermeasureagainst network attack. There are mainly two
typesof detections; signature-based and anomaly-based. And thereare two kinds of error; false negative
and false positive. Indevelopment of IDS, establishment of a method to reduce suchfalse is a major issue.
In this paper, we propose a new anomaly-baseddetection method using Discrete Fourier Transform
(DFT)with window function. In our method, we assume fluctuation ofpayload in ordinary sessions as
random. On the other hand, we cansee fluctuation in attack sessions have bias. From the viewpointof
spectrum analysis based on such assumption, we can find outdifferent characteristic in spectrum of attack
sessions. Using thecharacteristic, we can detect attack sessions. Example detectionagainst Kyoto2006+
dataset shows 12.0% of false positive at most,and 0.0% of false negative.
Intrusion detection system using discrete fourier transform with window functionIJNSA Journal
An Intrusion Detection System (IDS) is counter measure against network attack. There are mainly two types of detections; signature-based and anomaly-based. And there are two kinds of error; false negative and false positive. In development of IDS, establishment of a method to reduce such false is a major issue. In this paper, we propose a new anomaly-based detection method using Discrete Fourier Transform (DFT)with window function. In our method, we assume fluctuation of payload in ordinary sessions as random. On the other hand, we can see fluctuation in attack sessions have bias. From the view point of spectrum analysis based on such assumption, we can find out different characteristic in spectrum of attack sessions. Using thecharacteristic, we can detect attack sessions. Example detection against Kyoto2006+
dataset shows 12.0% of false positive at most,and 0.0% of false negative.
Extended Kalman observer based sensor fault detectionIJECEIAES
This article discusses the Kalman observer based fault detection approach. The calculation of the residues can detect faults, but if there are noises, uncertainties become very important. To reduce the influence of these noises, a calculation of the instantaneous energy of the residues gave a better precision. The Kalman observer was used to estimate system performance and eliminate unknown noise and external disturbances. Instantaneous Power Calculation (IPCFD) based fault detection can detect potential sensor faults in hybrid systems. The effectiveness of the proposed approach is illustrated by the main application.
Similar to Parametrized Model Checking of Fault Tolerant Distributed Algorithms by Abstraction (part 1) (20)
Foundations of Software Testing Lecture 4Iosif Itkin
This lecture is a part of the online course on Software Testing for Complex Intelligent Systems and Autonomous Vehicles. The course lectures provide the theoretical basics of testing autonomous systems based on artificial intelligence.
The fourth lecture of the course entitled Foundations of Software Testing reviews the ‘absence-of-errors fallacy’ and other principles of software testing, as well as the types and levels of software testing. The lecture also provides a fuller picture of the understanding of test objectives and methodologies by different schools of thought within the software testing domain.
QA Financial Forum London 2021 - Automation in Software Testing. Humans and C...Iosif Itkin
Speaker: Iosif Itkin, co-CEO & co-founder, Exactpro Systems
9th November 2021
Hilton Canary Wharf
Exactpro is an independent software testing business focused on mission-critical financial market infrastructures, primarily exchanges and clearing houses. In his presentation, Iosif will give a brief overview of research on the concept of model-based testing and the principal challenges of its application while testing complex distributed systems. He will also outline the broader context of interaction between humans and complex computer models.
Exactpro FinTech Webinar - Global Exchanges Test OraclesIosif Itkin
Global Exchanges series webinar to discuss Test Oracles. A test oracle is a mechanism for determining whether a test has passed or failed. The use of oracles involves comparing the output(s) of the system under test for a given test-case input, to the output(s) that the oracle determines the product should have. We will review various types of test oracles using examples from Exactpro’s Global Exchanges division projects and protocol-based interactions in trading systems.
Exactpro FinTech Webinar - Global Exchanges FIX ProtocolIosif Itkin
Exactpro’s Global Exchanges Division training session on FIX Trading Protocol.
The essence of the FIX protocol and its place in the overall structure of network protocols, FIX message attributes and the internal data types of the protocol.
Operational Resilience in Financial Market InfrastructuresIosif Itkin
A4Q World Congress 13-16 April 2021
Iosif Itkin
Exactpro provides independent software testing services for mission critical technology that underpins global financial markets – exchanges and clearing houses. Half of the top 20 global exchange groups on all continents around the globe rely on processes, platforms and people from Exactpro to improve their quality and reliability. The company has spent the last 11 years studying operational resilience in this crucial sector. The presentation will outline the key principles for software testing of the systems that process hundreds of millions of orders per day with roundtrip latencies below one hundred microseconds.
20 Simple Questions from Exactpro for Your Enjoyment This Holiday SeasonIosif Itkin
Warmest wishes for a happy holiday season and a wonderful New Year!
We look forward to our continued collaboration in 2020. Thank you for your support.
QA-Financial Forum 2019 in New York
13 November
Iosif Itkin, CEO and co-founder
Elena Treshcheva, Business Development Manager and Researcher
An October 2019 survey by BoE and FCA found that ML in financial organizations has already passed an initial development phase, and the usage of live ML applications is about to dramatically increase over the next three years. Artificial Intelligence systems are used in market surveillance, they are providing intellectual analysis of news feeds, and they are an important part of the conversational agents facing users and helping them with their business needs from identity verification to trading and portfolio management. How to ensure that an AI-powered system is up to its task? And what would that mean from the software testing perspective?
EXTENT 2019: Exactpro Quality Assurance for Financial Market InfrastructuresIosif Itkin
On Complex Software Systems Testing — Alexey Zverev, co-CEO & co-founder, Exactpro
Software Testing and Machine Learning
Mind the Gap. Applying Process Mining
Learning from Failure is not just for Humans
Dancing with Whales. Adaptive Log Classification System
On Traceability and the Illusion of Control
Building Partnerships
Demystifying DLT Testing One Network at a Time
Get the MOST from FIX
Georgia on My Mind
Build Software to Test Software — Iosif Itkin, co-CEO & co-founder, Exactpro
ClearTH Test Automation Framework: Case Study in IRS & CDS Swaps Lifecycle Mo...Iosif Itkin
Synchronize Europe
18th June 2019
Iosif Itkin, co-CEO and co-founder, Exactpro
Using the ISDA CDM Swaps application, simultaneously execute multiple end-to-end scenarios for DAML applications in capital markets - validate with actual contract data on ledger.
EXTENT Talks 2019 Tbilisi: Failover and Recovery Test Automation - Ivan ShamraiIosif Itkin
Ivan Shamray, Senior NFT Analyst, Exactpro
20 April 2019 EXTENT Talks, Tbilisi, Georgia
Tbilisi QA Community
EXTENT Talks is a meeting place for IT specialists working in various industries and seeking professional growth, practitioners from IT firms, as well as Quality Assurance enthusiasts of all backgrounds interested in actively participating in local IT events.
EXTENT Talks QA Community Tbilisi 20 April 2019 - Conference OpenIosif Itkin
EXTENT Talks is a meeting place for IT specialists working in various industries and seeking professional growth, practitioners from IT firms, as well as Quality Assurance enthusiasts of all backgrounds interested in actively participating in local IT events. The first EXTENT Talks were held in Tbilisi on 22 February 2019, initiating the creation a QA Community in Tbilisi and laying a foundation for an international platform for exchanging experience and knowledge in the field of software testing, development and IT. The program of the inaugural event included presentations on ISTQB, Software Testing, and Agile methodology from senior specialists. The next EXTENT Talks in Tbilisi will take place on 20 April 2019.
User-Assisted Log Analysis for Quality Control of Distributed Fintech Applica...Iosif Itkin
The First IEEE International Conference On Artificial Intelligence Testing (2019 IEEE AITest)
Iosif Itkin, Anna Gromova, Anton Sitnikov, Elena Treshcheva, Rostislav Yavorskiy, Evgenii Tsymbalov, Andrey Novikov and Kirill Rudakov
1 Exactpro, UK, Georgia, USA, Russia
2 Skolkovo Institute of Science and Technology, Russia
3 Higher School of Economics, Russia
Speakers: Iosif Itkin, CEO and Co-Founder and Elena Treshcheva, Business Development Manager and Researcher - Exactpro
Exactpro provides software testing services for mission-critical technology that underpins global financial markets. Exactpro clients are regulated by FCA, Bank of England and their counterparts from other countries. During this session, Elena and Iosif will talk about end-to-end software testing for post-trade systems in financial market infrastructures. What are the key challenges in quality assurance at this scale? What kind of cognitive biases affect SDLC? How precise is the knowledge about the systems under test? What constitutes good test evidence? How to deal with complexity in regulated environments?
Behaviour Driven Development: Oltre i limiti del possibileIosif Itkin
The QA Financial Forum: Milan 2019
23 January at the Excelsior Hotel Gallia.
Anna-Maria Lukina, Exactpro Business Development Director
The QA Financial Forum: Milan is one of the leading fintech conferences in Italy. The event focuses on the latest achievements in software risk management and automation of software testing. The predominant theme of the Milan event will be Quality Assurance for the entire Software Development Life Cycle (SDLC).
The topics under discussion will feature:
- Technologies for Automation & AI
- DevOps & CI/CD
- Value Stream Management
- Test Data Management
- Regulatory Compliance
- App Security & DevSecOps
- Testing and quality assurance of Blockchain platforms
The official language of the event is Italian.
On 17th January 2018 Exactpro successfully completed a management buyout from London Stock Exchange Group (LSEG), signed a new multi-year master services agreement with LSEG, and opened its head office in London.
What else has happened in 2018?
I wanted to take the opportunity to reflect on what has been an unusual year for Exactpro.
Integration front to back - Mr. Custodian tear down that wall
The scope of the application level has been continuous extended over the years, albeit with a focus on the area of pre-trade and trade.
Recently, there has been an increased interest to move further into the area of post-trade which is predominantly driven by the ISO 20022 standard. Is there really a need for new FIX messages in areas such as payments and
what are the integration problems needing a resolution?
Panellists
- Iosif Itkin, CEO, Exactpro
- Jim Northey, Co-Chair Global Technical Committee, Americas Region, FIX Trading Community, Chair Elect, ISO TC68 Financial
Services Technical Committee, and Consultant and Industry Standards Liaison, Itiviti
- Barry Young, Director, Aladdin Product Manager, BlackRock
BDD. The Outer Limits. Iosif Itkin at Youcon (in Russian)Iosif Itkin
Exactpro is supporting the 3rd annual IT-conference YouCon to take place on 14th October in Saratov, Russia. Over 900 programmers, systems engineers and architects, software QA engineers, and marketing specialists will gather to discuss the latest trends in programming technology. It is the largest IT industry event in Saratov.
Iosif Itkin, CEO of Exactpro, part of London Stock Exchange Group, will deliver a "BDD. The Outer Limits" presentation named after Iosif's favorite Sci-Fi series.
The topics to be covered are:
Behavior Driven Development concepts
Applying BDD in trading and clearing systems
Specification by Example and using production data
Combining Model-based testing and BDD
The Outer Limits
There will be an opportunity to ask questions, share thoughts and expertise in BDD, or just chat with a representative at the Exactpro stand at any time during the event.
Don't miss out, stop by and ask how you can get your Exactpro souvenir :)
We look forward to meeting you there!
#Exactpro #Youconsaratov
Operation “Blue Star” is the only event in the history of Independent India where the state went into war with its own people. Even after about 40 years it is not clear if it was culmination of states anger over people of the region, a political game of power or start of dictatorial chapter in the democratic setup.
The people of Punjab felt alienated from main stream due to denial of their just demands during a long democratic struggle since independence. As it happen all over the word, it led to militant struggle with great loss of lives of military, police and civilian personnel. Killing of Indira Gandhi and massacre of innocent Sikhs in Delhi and other India cities was also associated with this movement.
Safalta Digital marketing institute in Noida, provide complete applications that encompass a huge range of virtual advertising and marketing additives, which includes search engine optimization, virtual communication advertising, pay-per-click on marketing, content material advertising, internet analytics, and greater. These university courses are designed for students who possess a comprehensive understanding of virtual marketing strategies and attributes.Safalta Digital Marketing Institute in Noida is a first choice for young individuals or students who are looking to start their careers in the field of digital advertising. The institute gives specialized courses designed and certification.
for beginners, providing thorough training in areas such as SEO, digital communication marketing, and PPC training in Noida. After finishing the program, students receive the certifications recognised by top different universitie, setting a strong foundation for a successful career in digital marketing.
A Strategic Approach: GenAI in EducationPeter Windle
Artificial Intelligence (AI) technologies such as Generative AI, Image Generators and Large Language Models have had a dramatic impact on teaching, learning and assessment over the past 18 months. The most immediate threat AI posed was to Academic Integrity with Higher Education Institutes (HEIs) focusing their efforts on combating the use of GenAI in assessment. Guidelines were developed for staff and students, policies put in place too. Innovative educators have forged paths in the use of Generative AI for teaching, learning and assessments leading to pockets of transformation springing up across HEIs, often with little or no top-down guidance, support or direction.
This Gasta posits a strategic approach to integrating AI into HEIs to prepare staff, students and the curriculum for an evolving world and workplace. We will highlight the advantages of working with these technologies beyond the realm of teaching, learning and assessment by considering prompt engineering skills, industry impact, curriculum changes, and the need for staff upskilling. In contrast, not engaging strategically with Generative AI poses risks, including falling behind peers, missed opportunities and failing to ensure our graduates remain employable. The rapid evolution of AI technologies necessitates a proactive and strategic approach if we are to remain relevant.
Francesca Gottschalk - How can education support child empowerment.pptxEduSkills OECD
Francesca Gottschalk from the OECD’s Centre for Educational Research and Innovation presents at the Ask an Expert Webinar: How can education support child empowerment?
Normal Labour/ Stages of Labour/ Mechanism of LabourWasim Ak
Normal labor is also termed spontaneous labor, defined as the natural physiological process through which the fetus, placenta, and membranes are expelled from the uterus through the birth canal at term (37 to 42 weeks
Acetabularia Information For Class 9 .docxvaibhavrinwa19
Acetabularia acetabulum is a single-celled green alga that in its vegetative state is morphologically differentiated into a basal rhizoid and an axially elongated stalk, which bears whorls of branching hairs. The single diploid nucleus resides in the rhizoid.
2024.06.01 Introducing a competency framework for languag learning materials ...Sandy Millin
http://sandymillin.wordpress.com/iateflwebinar2024
Published classroom materials form the basis of syllabuses, drive teacher professional development, and have a potentially huge influence on learners, teachers and education systems. All teachers also create their own materials, whether a few sentences on a blackboard, a highly-structured fully-realised online course, or anything in between. Despite this, the knowledge and skills needed to create effective language learning materials are rarely part of teacher training, and are mostly learnt by trial and error.
Knowledge and skills frameworks, generally called competency frameworks, for ELT teachers, trainers and managers have existed for a few years now. However, until I created one for my MA dissertation, there wasn’t one drawing together what we need to know and do to be able to effectively produce language learning materials.
This webinar will introduce you to my framework, highlighting the key competencies I identified from my research. It will also show how anybody involved in language teaching (any language, not just English!), teacher training, managing schools or developing language learning materials can benefit from using the framework.
Model Attribute Check Company Auto PropertyCeline George
In Odoo, the multi-company feature allows you to manage multiple companies within a single Odoo database instance. Each company can have its own configurations while still sharing common resources such as products, customers, and suppliers.
Biological screening of herbal drugs: Introduction and Need for
Phyto-Pharmacological Screening, New Strategies for evaluating
Natural Products, In vitro evaluation techniques for Antioxidants, Antimicrobial and Anticancer drugs. In vivo evaluation techniques
for Anti-inflammatory, Antiulcer, Anticancer, Wound healing, Antidiabetic, Hepatoprotective, Cardio protective, Diuretics and
Antifertility, Toxicity studies as per OECD guidelines
The French Revolution, which began in 1789, was a period of radical social and political upheaval in France. It marked the decline of absolute monarchies, the rise of secular and democratic republics, and the eventual rise of Napoleon Bonaparte. This revolutionary period is crucial in understanding the transition from feudalism to modernity in Europe.
For more information, visit-www.vavaclasses.com
Parametrized Model Checking of Fault Tolerant Distributed Algorithms by Abstraction (part 1)
1. Model Checking of Fault-Tolerant Distributed Algorithms
Part III: Parameterized Model Checking of Fault-tolerant Distributed
Algorithms by Abstraction
Annu Gmeiner Igor Konnov Ulrich Schmid
Helmut Veith Josef Widder
TMPA 2014, Kostroma, Russia
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 1 / 1
2. Fault-tolerant DAs: Model Checking Challenges
unbounded data types
counting how many messages have been received
parameterization in multiple parameters
among n processes f ≤ t are faulty with n > 3t
contrast to concurrent programs
fault tolerance against adverse environments
degrees of concurrency
many degrees of partial synchrony
continuous time
fault-tolerant clock synchronization
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 2 / 1
3. Distributed algorithms: computational model and faults
In previous parts, we considered algorithms operating
in the classic model by [Fischer, Lynch, Paterson’85]
Environment:
Asynchronous processes (interleaving semantics)
Reliable asynchronous message passing (non-blocking send and receive)
Faults:
crashes and clean crashes,
omission faults,
symmetric faults,
Byzantine faults
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 3 / 1
4. Model checking problem for fault-tolerant DA algorithms
Parameterized model checking problem:
given a distributed algorithm and spec. ϕ
show for all n, t, and f satisfying n > 3t ∧ t ≥ f ≥ 0
M(n, t, f ) |= ϕ
every M(n, t, f ) is a system of n − f correct processes
n
?
?
?
t
n
?
?
?
t f
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 4 / 1
5. Model checking problem for fault-tolerant DA algorithms
Parameterized model checking problem:
given a distributed algorithm and spec. ϕ
show for all n, t, and f satisfying resilience condition
M(n, t, f ) |= ϕ
every M(n, t, f ) is a system of N(n, f ) correct processes
n
?
?
?
t
n
?
?
?
t f
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 4 / 1
6. Properties in Linear Temporal Logic
Unforgeability (U). If vi = 0 for all correct processes i, then for all correct
processes j, acceptj remains 0 forever.
G
n−f
i=1
vi = 0 → G
n−f
j=1
acceptj = 0
Completeness (C). If vi = 1 for all correct processes i, then there is a correct
process j that eventually sets acceptj to 1.
G
n−f
i=1
vi = 1 → F
n−f
j=1
acceptj = 1
Relay (R). If a correct process i sets accepti to 1, then eventually all correct
processes j set acceptj to 1.
G
n−f
i=1
accepti = 1 → F
n−f
j=1
acceptj = 1
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 5 / 1
7. Properties in Linear Temporal Logic
Unforgeability (U). If vi = 0 for all correct processes i, then for all correct
processes j, acceptj remains 0 forever.
G
n−f
i=1
vi = 0 → G
n−f
j=1
acceptj = 0 Safety
Completeness (C). If vi = 1 for all correct processes i, then there is a correct
process j that eventually sets acceptj to 1.
G
n−f
i=1
vi = 1 → F
n−f
j=1
acceptj = 1 Liveness
Relay (R). If a correct process i sets accepti to 1, then eventually all correct
processes j set acceptj to 1.
G
n−f
i=1
accepti = 1 → F
n−f
j=1
acceptj = 1 Liveness
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 5 / 1
9. Threshold-guarded FTDAs
Fault-free construct: quantified guards (t=f=0)
Existential Guard
if received m from some process then ...
Universal Guard
if received m from all processes then ...
These guards allow one to treat the processes in a parameterized way
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 7 / 1
10. Threshold-guarded FTDAs
Fault-free construct: quantified guards (t=f=0)
Existential Guard
if received m from some process then ...
Universal Guard
if received m from all processes then ...
These guards allow one to treat the processes in a parameterized way
what if faults might occur?
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 7 / 1
11. Threshold-guarded FTDAs
Fault-free construct: quantified guards (t=f=0)
Existential Guard
if received m from some process then ...
Universal Guard
if received m from all processes then ...
These guards allow one to treat the processes in a parameterized way
what if faults might occur?
Fault-Tolerant Algorithms: n processes, at most t are Byzantine
Threshold Guard
if received m from n − t processes then ...
(the processes cannot refer to f!)
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 7 / 1
12. Threshold-based fault-tolerant distributed algorithms
The parameters (n, t, f ) are fixed in each run
Main loop with the body executed atomically
Processes are anonymous (no identifiers)
Receiving messages, counting them and comparing to thresholds, e.g.,
if received <ECHO> from t + 1 distinct processes
then ...
Sending messages to all processes, e.g.,
send <ECHO> to all
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 8 / 1
13. Control Flow Automata
Variables of process i
vi : {0 , 1} init with 0 or 1
accepti : {0 , 1} init with 0
An indivisible step:
i f vi = 1
then send ( echo ) to all ;
i f received (echo) from at l e a s t
t + 1 distinct processes
and not sent ( echo ) before
then send ( echo ) to all ;
i f received ( echo ) from at l e a s t
n - t distinct processes
then accepti := 1;
n − f copies of the process
qI
q0
q1
q2
q3
sv = V1
¬(sv = V1) inc nsnt
sv := SE
q4
q5
q6
q7
q8
qF
nrcvd := z where (nrcvd ≤ z ∧ z ≤ nsnt + f )
¬(t + 1 ≤ nrcvd)
t + 1 ≤ nrcvd
sv = V0
¬(sv = V0)
inc nsnt
n − t ≤ nrcvd
¬(n − t ≤ nrcvd)
sv := SE
sv := AC
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 9 / 1
14. Counting argument in threshold-guarded algorithms
n
t f
if received m from t + 1 processes then ...
t + 1
Correct processes count distinct incoming messages
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 10 / 1
15. Counting argument in threshold-guarded algorithms
n
t f
if received m from t + 1 processes then ...
t + 1
Correct processes count distinct incoming messages
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 10 / 1
16. Counting argument in threshold-guarded algorithms
n
t f
if received m from t + 1 processes then ...
t + 1
at least one non-faulty sent the message
Correct processes count distinct incoming messages
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 10 / 1
17. qI
q0
q1
q2
q3
sv = V1
¬(sv = V1) inc nsnt
sv := SE
q4
q5
q6
q7
q8
qF
nrcvd := z where (nrcvd ≤ z ∧ z ≤ nsnt + f )
¬(t + 1 ≤ nrcvd)
t + 1 ≤ nrcvd
sv = V0
¬(sv = V0)
inc nsnt
n − t ≤ nrcvd
¬(n − t ≤ nrcvd)
sv := SE
sv := AC
concrete values are not important
thresholds are essential:
0, 1, t + 1, n − t
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 11 / 1
18. qI
q0
q1
q2
q3
sv = V1
¬(sv = V1) inc nsnt
sv := SE
q4
q5
q6
q7
q8
qF
nrcvd := z where (nrcvd ≤ z ∧ z ≤ nsnt + f )
¬(t + 1 ≤ nrcvd)
t + 1 ≤ nrcvd
sv = V0
¬(sv = V0)
inc nsnt
n − t ≤ nrcvd
¬(n − t ≤ nrcvd)
sv := SE
sv := AC
concrete values are not important
thresholds are essential:
0, 1, t + 1, n − t
intervals with symbolic boundaries:
I0 = [0, 1)
I1 = [1, t + 1)
It+1 = [t + 1, n − t)
In−t = [n − t, ∞)
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 11 / 1
19. qI
q0
q1
q2
q3
sv = V1
¬(sv = V1) inc nsnt
sv := SE
q4
q5
q6
q7
q8
qF
nrcvd := z where (nrcvd ≤ z ∧ z ≤ nsnt + f )
¬(t + 1 ≤ nrcvd)
t + 1 ≤ nrcvd
sv = V0
¬(sv = V0)
inc nsnt
n − t ≤ nrcvd
¬(n − t ≤ nrcvd)
sv := SE
sv := AC
concrete values are not important
thresholds are essential:
0, 1, t + 1, n − t
intervals with symbolic boundaries:
I0 = [0, 1)
I1 = [1, t + 1)
It+1 = [t + 1, n − t)
In−t = [n − t, ∞)
Parameteric Interval Abstraction (PIA)
Similar to interval abstraction:
[t + 1, n − t) rather than [4, 10).
Total order: 0 < 1 < t + 1 < n − t for
all parameters satisfying RC:
n > 3t, t ≥ f ≥ 0.
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 11 / 1
20. Technical challenges
We have to reduce the verification of an infinite number of instances
where
1 the process code is parameterized
2 the number of processes is parameterized
to one finite state model checking instance
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 12 / 1
21. Technical challenges
We have to reduce the verification of an infinite number of instances
where
1 the process code is parameterized
2 the number of processes is parameterized
to one finite state model checking instance
We do that by:
1 PIA data abstraction
2 PIA counter abstraction
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 12 / 1
22. Technical challenges
We have to reduce the verification of an infinite number of instances
where
1 the process code is parameterized
2 the number of processes is parameterized
to one finite state model checking instance
We do that by:
1 PIA data abstraction
2 PIA counter abstraction
abstraction is an over approximation ⇒ possible abstract behavior that
does not correspond to a concrete behavior.
3 Refining spurious counter-examples
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 12 / 1
23. Abstraction overview
Parameterized family
M(n, t, f ) = P(n, t, f ) · · · P(n, t, f )
N(n,t,f ) processes
: n > 3t, t ≥ f , f ≥ 0} extract
Parametric Interval Domain D
parametric interval
data abstraction
Uniform parameterized family
ˆM(n, t, f ) = ˆP · · · ˆP
N(n,t,f ) processes
: n > 3t, t ≥ f , f ≥ 0}
P does not depend on n, t, f
P simulates P(n, t, f )
change representation
Counter representation
parametric interval
counter abstraction
one abstract system A that
simulates for every n, t, f
the behavior of M(n, t, f )
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 13 / 1
24. Abstraction overview
Parameterized family
M(n, t, f ) = P(n, t, f ) · · · P(n, t, f )
N(n,t,f ) processes
: n > 3t, t ≥ f , f ≥ 0} extract
Parametric Interval Domain D
parametric interval
data abstraction
Uniform parameterized family
ˆM(n, t, f ) = ˆP · · · ˆP
N(n,t,f ) processes
: n > 3t, t ≥ f , f ≥ 0}
P does not depend on n, t, f
P simulates P(n, t, f )
change representation
Counter representation
parametric interval
counter abstraction
one abstract system A that
simulates for every n, t, f
the behavior of M(n, t, f )
finite-state model checkin
replay the counter-example
refine the system
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 13 / 1
25. Data abstraction
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 14 / 1
26. qI
q0
q1
q2
q3
sv = V1
¬(sv = V1) inc nsnt
sv := SE
q4
q5
q6
q7
q8
qF
nrcvd := z where (nrcvd ≤ z ∧ z ≤ nsnt + f )
¬(t + 1 ≤ nrcvd)
t + 1 ≤ nrcvd
sv = V0
¬(sv = V0)
inc nsnt
n − t ≤ nrcvd
¬(n − t ≤ nrcvd)
sv := SE
sv := AC
concrete values are not important
thresholds are essential:
0, 1, t + 1, n − t
intervals with symbolic boundaries:
I0 = [0, 1)
I1 = [1, t + 1)
It+1 = [t + 1, n − t)
In−t = [n − t, ∞)
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 15 / 1
27. Abstract operations
Concrete:
Abstract:
0 1 t + 1 n − t above
· · ·
I0 I1 It+1 In−t
Concrete t + 1 ≤ x
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 16 / 1
28. Abstract operations
Concrete:
Abstract:
0 1 t + 1 n − t above
· · ·
I0 I1 It+1 In−t
Concrete t + 1 ≤ x is abstracted as x = It+1 ∨ x = In−t.
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 16 / 1
29. Abstract operations
Concrete:
Abstract:
0 1 t + 1 n − t above
· · ·
I0 I1 It+1 In−t
Concrete t + 1 ≤ x is abstracted as x = It+1 ∨ x = In−t.
Concrete x = x + 1,
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 16 / 1
30. Abstract operations
Concrete:
Abstract:
0 1 t + 1 n − t above
· · ·
I0 I1I0 I1 It+1 In−t
Concrete t + 1 ≤ x is abstracted as x = It+1 ∨ x = In−t.
Concrete x = x + 1, is abstracted as:
x = I0 ∧ x = I1 . . .
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 16 / 1
31. Abstract operations
Concrete:
Abstract:
0 1 t + 1 n − t above
· · ·
I0 I1I0 I1 It+1 In−t
Concrete t + 1 ≤ x is abstracted as x = It+1 ∨ x = In−t.
Concrete x = x + 1, is abstracted as:
x = I0 ∧ x = I1
∨x = I1 ∧ (x = I1 ∨ x = It+1) . . .
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 16 / 1
32. Abstract operations
Concrete:
Abstract:
0 1 t + 1 n − t above
· · ·
I0 I1I0 I1 It+1 In−t
Concrete t + 1 ≤ x is abstracted as x = It+1 ∨ x = In−t.
Concrete x = x + 1, is abstracted as:
x = I0 ∧ x = I1
∨x = I1 ∧ (x = I1 ∨ x = It+1)
∨x = It+1 ∧ (x = It+1 ∨ x = In−t) . . .
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 16 / 1
33. Abstract operations
Concrete:
Abstract:
0 1 t + 1 n − t above
· · ·
I0 I1I0 I1 It+1 In−t
Concrete t + 1 ≤ x is abstracted as x = It+1 ∨ x = In−t.
Concrete x = x + 1, is abstracted as:
x = I0 ∧ x = I1
∨x = I1 ∧ (x = I1 ∨ x = It+1)
∨x = It+1 ∧ (x = It+1 ∨ x = In−t)
∨x = In−t ∧ x = In−t
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 16 / 1
34. Abstract operations
Concrete:
Abstract:
0 1 t + 1 n − t above
· · ·
I0 I1
Concrete t + 1 ≤ x is abstracted as x = It+1 ∨ x = In−t.
Concrete x = x + 1, is abstracted as:
x = I0 ∧ x = I1
∨x = I1 ∧ (x = I1 ∨ x = It+1)
∨x = It+1 ∧ (x = It+1 ∨ x = In−t)
∨x = In−t ∧ x = In−t
abstract increase may keep the same value!
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 16 / 1
35. Abstract CFA
qI
q0
q1
q2
q3
sv = V1
¬(sv = V1) inc nsnt
sv := SE
q4
q5
q6
q7
nrcvd := z where (nrcvd ≤ z ∧ z ≤ nsnt + f )
¬(t + 1 ≤ nrcvd)
t + 1 ≤ nrcvd
sv = V0
¬(sv = V0)
inc nsnt
n − t ≤ nrcvd
¬(n − t ≤ nrcvd)
sv := AC
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 17 / 1
36. Abstract CFA
qI
q0
q1
q2
q3
sv = V1
¬(sv = V1) inc nsnt
sv := SE
q4
q5
q6
q7
nrcvd := z where (nrcvd ≤ z ∧ z ≤ nsnt + f )
¬(t + 1 ≤ nrcvd)
t + 1 ≤ nrcvd
sv = V0
¬(sv = V0)
inc nsnt
n − t ≤ nrcvd
¬(n − t ≤ nrcvd)
sv := AC
qI
q0
q1
q2
q3
sv = V1
¬(sv = V1) inc nsnt
sv := SE
q4
q5
q6
q7
nrcvd = I0 ∧ nsnt = I0 ∧ (nrcvd = I0 ∨ nrcvd = I1) ∨ . . .
¬(t + 1 ≤ nrcvd)
nrcvd = It+1 ∨ nrcvd = In−t
sv = V0
¬(sv = V0)
nsnt = I1 ∧ (nsnt = I1 ∨ nsnt = It+1) ∨ . . .
n − t ≤ nrcvd
¬(n − t ≤ nrcvd)
sv := AC
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 17 / 1
37. Abstraction overview
Parameterized family
M(n, t, f ) = P(n, t, f ) · · · P(n, t, f )
N(n,t,f ) processes
: n > 3t, t ≥ f , f ≥ 0} extract
Parametric Interval Domain D
parametric interval
data abstraction
Uniform parameterized family
ˆM(n, t, f ) = ˆP · · · ˆP
N(n,t,f ) processes
: n > 3t, t ≥ f , f ≥ 0}
P does not depend on n, t, f
P simulates P(n, t, f )
change representation
Counter representation
parametric interval
counter abstraction
one abstract system A that
simulates for every n, t, f
the behavior of M(n, t, f )
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 18 / 1
39. Classic (0, 1, ∞)-counter abstraction
Pnueli, Xu, and Zuck (2001) introduced (0, 1, ∞)-counter abstraction:
finitely many local states,
e.g., {N, T, C}.
based on counter representation:
for each local states count how many processes are in it
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 20 / 1
40. Classic (0, 1, ∞)-counter abstraction
Pnueli, Xu, and Zuck (2001) introduced (0, 1, ∞)-counter abstraction:
finitely many local states,
e.g., {N, T, C}.
based on counter representation:
for each local states count how many processes are in it
abstract the number of processes in every state,
e.g., K : C → 0, T → 1, N → “many”.
perfectly reflects mutual exclusion properties
e.g., G (K(C) = 0 ∨ K(C) = 1).
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 20 / 1
41. Limits of (0, 1, ∞)-counter abstraction
Our parametric data + counter abstraction:
we require finer counting of processes:
t + 1 processes in a specific state can force global progress,
t processes cannot
mapping t, t + 1, and n − t to “many” is too coarse.
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 21 / 1
42. Limits of (0, 1, ∞)-counter abstraction
Our parametric data + counter abstraction:
we require finer counting of processes:
t + 1 processes in a specific state can force global progress,
t processes cannot
mapping t, t + 1, and n − t to “many” is too coarse.
starting point of our approach...
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 21 / 1
43. Data + counter abstraction over parametric intervals
n = 6, t = 1, f = 1
t + 1 = 2, n − t = 5
nr. processes (counters)
received received
sent accepted
•
0
•
0
•
1
•
1
•
2
•
2
•
3
•
3
•
4
•
4
•
5
•
5
•
6
•
6
•0
•1
•2
•3
•4
•5
•6
•
Local state is (sv, nrcvd),
where sv ∈ {sent, accepted} and 0 ≤ rcvd ≤ n
3 processes at (sent, received=3)
1 process at (accepted, received=5)
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 22 / 1
44. Data + counter abstraction over parametric intervals
n = 6, t = 1, f = 1
t + 1 = 2, n − t = 5
nr. processes (counters)
received received
sent accepted
•
0
•
0
•
1
•
1
•
2
•
2
•
3
•
3
•
4
•
4
•
5
•
5
•
6
•
6
•0
•1
•2
•3
•4
•5
•6
•
Local state is (sv, nrcvd),
where sv ∈ {sent, accepted} and 0 ≤ rcvd ≤ n
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 22 / 1
45. Data + counter abstraction over parametric intervals
n = 6, t = 1, f = 1
t + 1 = 2, n − t = 5
nr. processes (counters)
received received
sent accepted
•
0
•
0
•
1
•
1
•
2
•
2
•
3
•
3
•
4
•
4
•
5
•
5
•
6
•
6
•0
•1
•2
•3
•4
•5
•6
•
Local state is (sv, nrcvd),
where sv ∈ {sent, accepted} and 0 ≤ rcvd ≤ n
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 22 / 1
46. Data + counter abstraction over parametric intervals
XXXXXXn = 6, XXXXXXt = 1, XXXXXXf = 1
n 3 · t ∧ t ≥ f
Parametricintervals:
I0 = [0, 1) I1 = [1, t + 1)
It+1 = [t + 1, n − t)
In−t = [n − t, ∞)
nr. processes (counters)
received received
sent accepted
• • • •
I0 I1 It+1 In−t
• • • •
I0 I1 It+1 In−t
•
•
•
•
I0
I1
It+1
In−t
A local state is (sv, nrcvd),
where sv ∈ {sent, accepted} and nrcvd ∈ {I0, I1, It+1, In−t}
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 22 / 1
47. Data + counter abstraction over parametric intervals
n 3 · t ∧ t ≥ f
Parametricintervals:
I0 = [0, 1) I1 = [1, t + 1)
It+1 = [t + 1, n − t)
In−t = [n − t, ∞)
nr. processes (counters)
received received
sent accepted
• • • •
I0 I1 It+1 In−t
• • • •
I0 I1 It+1 In−t
•
•
•
•
I0
I1
It+1
In−t
when all correct processes accepted,
all non-zero counters are in this area
A local state is (sv, nrcvd),
where sv ∈ {sent, accepted} and nrcvd ∈ {I0, I1, It+1, In−t}
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 22 / 1
49. Spurious behavior
abstraction adds behaviors (e.g., x’=x+1 may lead to x’ being equal to x)
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 24 / 1
50. Spurious behavior
abstraction adds behaviors (e.g., x’=x+1 may lead to x’ being equal to x)
⇒ specs that hold in concrete system may be violated in abstract system
spurious counterexamples
we have to reduce the behaviors of the abstract system
make it more concrete
. . . based on the counterexamples = CEGAR
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 24 / 1
51. Spurious behavior
abstraction adds behaviors (e.g., x’=x+1 may lead to x’ being equal to x)
⇒ specs that hold in concrete system may be violated in abstract system
spurious counterexamples
we have to reduce the behaviors of the abstract system
make it more concrete
. . . based on the counterexamples = CEGAR
Three sources of spurious behavior
# processes decreasing or increasing
# messages sent = # processes which have sent a message
unfair loops
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 24 / 1
52. Spurious behavior
abstraction adds behaviors (e.g., x’=x+1 may lead to x’ being equal to x)
⇒ specs that hold in concrete system may be violated in abstract system
spurious counterexamples
we have to reduce the behaviors of the abstract system
make it more concrete
. . . based on the counterexamples = CEGAR
Three sources of spurious behavior
# processes decreasing or increasing
# messages sent = # processes which have sent a message
unfair loops
. . . and a new abstraction phenomenon
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 24 / 1
57. CEGAR — automated workflow
Model Checking
correct
Abstraction refinement
using SMT
counterexample
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 26 / 1
58. CEGAR — automated workflow
Model Checking
correct
Abstraction refinement
using SMT
counterexample
CE feasible: bug
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 26 / 1
59. CEGAR — automated workflow
Model Checking
correct
Abstraction refinement
using SMT
counterexample
CE feasible: bug
CE spurious:
refined abstraction
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 26 / 1
60. What is SMT?
recall SAT:
given a Boolean formula, e.g., (¬a ∨ ¬b ∨ c) ∧ (¬a ∨ b ∨ d ∨ e)
is there an assignment of true and false to variables a, b, c, d, e
such that the formula evaluates to true?
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 27 / 1
61. What is SMT?
recall SAT:
given a Boolean formula, e.g., (¬a ∨ ¬b ∨ c) ∧ (¬a ∨ b ∨ d ∨ e)
is there an assignment of true and false to variables a, b, c, d, e
such that the formula evaluates to true?
Satisfiability Modulo Theories (SMT) :
here just linear arithmetics
given a formula, e.g.,
x = y ∧ y = z ∧ u = x ∧ (x + y ≤ 1 ∧ 2x + y = 1) ∨ 3x + 2y ≥ 3
is there an assignment of values to u, x, y, z such that formula
evaluates to true?
practically efficient tools: Yices, Z3
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 27 / 1
62. Counter example: losing processes
Output of data abstraction: 16 local states: L = {(sv, ˆnrcvd)
with sv ∈ {v0, v1, sent, accepted} and ˆrcvd ∈ {I0, I1, It+1, In−t}}
An abstract global state is (ˆk, ˆnsnt),
where ˆnsnt ∈ {I0, I1, It+1, In−t} and ˆk : L → {I0, I1, It+1, In−t}
Consider an abstract trace:
ˆnsnt1 = I0
ˆk1( ) =
In−t , if = (v1, I0)
I0, otherwise
ˆnsnt2 = I1
ˆk2( ) =
In−t , if = (v1, I0)
I1, if = (sent, I0)
I0, otherwise
ˆnsnt3 = It+1
ˆk3( ) =
In−t , if = (v1, I0)
It+1, if = (sent, I0)
I0, otherwise
Encode the last state in SMT as a conjunction T of the constraints:
resilience condition n 3t ∧ t ≥ f ∧ f ≥ 0
zero counters (i = 4 ∧ i = 8) → 0 ≤ k3[i] 1
non-zero counters n − t ≤ k3[4] ∧ t + 1 ≤ k3[8] n − t
system size n − f = k3[0] + k3[1] + · · · + k3[15]
UNSAT
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 28 / 1
63. Remove transitions
We ask the SMT solver:
is there a satisfiable assignment for T?
if yes,
then the state is OK, may be part of a real counterexample
if not, then the state is spurious
remove transitions to that state in the abstract system
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 29 / 1
64. Liveness
distributed algorithm requires reliable communication
every message sent is eventually received
¬in transit ≡ [∀i. nrcvdi ≥ nsnt]
fairness F G ¬in transit necessary to verify liveness,
e.g., F G ¬in transit → G ([∀i. svi = v1] → F [∀i. svi = accept])
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 30 / 1
65. Liveness
distributed algorithm requires reliable communication
every message sent is eventually received
¬in transit ≡ [∀i. nrcvdi ≥ nsnt]
fairness F G ¬in transit necessary to verify liveness,
e.g., F G ¬in transit → G ([∀i. svi = v1] → F [∀i. svi = accept])
counter example (lasso):
¬in transit
¬in transit
¬in transit
¬in transit
¬in transit
¬in transit
¬in transit
s1 ¬in transit
s2
sk
s3
· · ·
· · ·
· · ·
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 30 / 1
66. Liveness — fairness suppression
¬in transit
¬in transit
¬in transit
¬in transit
¬in transit
¬in transit
¬in transit
s1 ¬in transit
s2
sk
s3
· · ·
· · ·
· · ·
if there is a spurious sj (all its concretizations violate ¬in transit),
then the loop is spurious.
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 31 / 1
67. Liveness — fairness suppression
¬in transit
¬in transit
¬in transit
¬in transit
¬in transit
¬in transit
¬in transit
s1 ¬in transit
s2
sk
s3
· · ·
· · ·
· · ·
if there is a spurious sj (all its concretizations violate ¬in transit),
then the loop is spurious.
refine fairness to F G ¬in transit ∧ G F
1≤j≤k
“out of sj
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 31 / 1
68. Liveness — fairness suppression
¬in transit
¬in transit
¬in transit
¬in transit
¬in transit
¬in transit
¬in transit
s1 ¬in transit
s2
sk
s3
· · ·
· · ·
· · ·
if there is a spurious sj (all its concretizations violate ¬in transit),
then the loop is spurious.
refine fairness to F G ¬in transit ∧ G F
1≤j≤k
“out of sj
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 31 / 1
70. Concrete vs. parameterized (Byzantine case)
Time to check relay (sec, logscale) Memory to check relay (MB, logscale)
Parameterized model checking performs well (the red line).
Experiments for fixed parameters quickly degrade
(n = 9 runs out of memory).
We found counter-examples for the cases n = 3t and f t,
where the resilience condition is violated.
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 33 / 1
71. Experimental results at a glance
Algorithm Fault Resilience Property Valid? #Refinements Time
ST87 Byz n 3t U 0 4 sec.
ST87 Byz n 3t C 10 32 sec.
ST87 Byz n 3t R 10 24 sec.
ST87 Symm n 2t U 0 1 sec.
ST87 Symm n 2t C 2 3 sec.
ST87 Symm n 2t R 12 16 sec.
ST87 Omit n 2t U 0 1 sec.
ST87 Omit n 2t C 5 6 sec.
ST87 Omit n 2t R 5 10 sec.
ST87 Clean n t U 0 2 sec.
ST87 Clean n t C 4 8 sec.
ST87 Clean n t R 13 31 sec.
CT96 Clean n t U 0 1 sec.
CT96 Clean n t A 0 1 sec.
CT96 Clean n t R 0 1 sec.
CT96 Clean n t C 0 1 sec.
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 34 / 1
72. When resilience condition is wrong...
Algorithm Fault Resilience Property Valid? #Refinements Time
ST87 Byz n 3t ∧ f ≤ t+1 U 9 56 sec.
ST87 Byz n 3t ∧ f ≤ t+1 C 11 52 sec.
ST87 Byz n 3t ∧ f ≤ t+1 R 10 17 sec.
ST87 Byz n ≥ 3t ∧ f ≤ t U 0 5 sec.
ST87 Byz n ≥ 3t ∧ f ≤ t C 9 32 sec.
ST87 Byz n ≥ 3t ∧ f ≤ t R 30 78 sec.
ST87 Symm n 2t ∧ f ≤ t+1 U 0 2 sec.
ST87 Symm n 2t ∧ f ≤ t+1 C 2 4 sec.
ST87 Symm n 2t ∧ f ≤ t+1 R 8 12 sec.
ST87 Omit n ≥ 2t ∧ f ≤ t U 0 1 sec.
ST87 Omit n ≥ 2t ∧ f ≤ t C 0 2 sec.
ST87 Omit n ≥ 2t ∧ f ≤ t R 0 2 sec.
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 35 / 1
73. Summary of results
Abstraction tailored for distributed algorithms
threshold-based
fault-tolerant
allows to express different fault assumptions
Verification of threshold-based fault-tolerant algorithms
with threshold guards that are widely used
Byzantine faults (and other)
for all system sizes
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 36 / 1
74. Related work: non-parameterized
Model checking of the small size instances:
clock synchronization [Steiner, Rushby, Sorea, Pfeifer 2004]
consensus [Tsuchiya, Schiper 2011]
asynchronous agreement, folklore broadcast, condition-based
consensus [John, Konnov, Schmid, Veith, Widder 2013]
and more...
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 37 / 1
75. Related work: parameterized case
Regular model checking of fault-tolerant distributed protocols:
[Fisman, Kupferman, Lustig 2008]
“First-shot” theoretical framework.
No guards like x ≥ t + 1, only x ≥ 1.
No implementation.
Manual analysis applied to folklore broadcast (crash faults).
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 38 / 1
76. Related work: parameterized case
Regular model checking of fault-tolerant distributed protocols:
[Fisman, Kupferman, Lustig 2008]
“First-shot” theoretical framework.
No guards like x ≥ t + 1, only x ≥ 1.
No implementation.
Manual analysis applied to folklore broadcast (crash faults).
Backward reachability using SMT with arrays:
[Alberti, Ghilardi, Pagani, Ranise, Rossi 2010-2012]
Implementation.
Experiments on Chandra-Toueg 1990.
No resilience conditions like n 3t.
Safety only.
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 38 / 1
81. Tool Chain: ByMC
Parametric Promela code static analysis + Yices
Parametric Interval Domain D
Parametric data abstraction
with Yices
Parametric Promela code
Parametric counter ab-
straction with Yices
normal
Promela code
Spin
property holds
counterexample
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 42 / 1
82. Tool Chain: ByMC
Parametric Promela code static analysis + Yices
Parametric Interval Domain D
Parametric data abstraction
with Yices
Parametric Promela code
Parametric counter ab-
straction with Yices
normal
Promela code
Spin
property holds
counterexample
Refine
Concrete counter
representation (VASS)
SMT formula
Yices
counterexample feasible
unsat
sat
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 42 / 1
83. Tool Chain: ByMC
Parametric Promela code static analysis + Yices
Parametric Interval Domain D
Parametric data abstraction
with Yices
Parametric Promela code
Parametric counter ab-
straction with Yices
normal
Promela code
Spin
property holds
counterexample
Refine
Concrete counter
representation (VASS)
SMT formula
Yices
counterexample feasible
invariant candidates (by the user)
unsat
sat
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 42 / 1
84. Experimental setup
The tool (source code in OCaml),
the code of the distributed algorithms in Parametric Promela,
and a virtual machine with full setup
are available at: http://forsyte.at/software/bymc
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 43 / 1
85. Running the tool — concrete case
user specifies parameter value
useful to check whether the code behaves as expected
$bymc/verifyco-spin N=4,T=1,F=1 bcast-byz.pml relay
model checking problem in directory
“./x/spin-bcast-byz-relay-N=4,T=1,F=1”
in concrete.prm
parameters are replaced by numbers
process prototype is replaced with N − F = 3 active processes
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 44 / 1
86. Running the tool — parameterized model checking
PIA data and counter abstraction
finite-state model checking on abstract model
$bymc/verifypa-spin bcast-omit.pml relay
model checking problem in directory
“./x/bcast-byz-relay-yymmdd-HHMM.*”
directory contains
abs-interval.prm: result of the data abstraction;
abs-counter.prm: result of the counter abstraction;
abs-vass.prm: auxiliary abstraction for abstraction refinement;
mc.out: the last output by Spin;
cex.trace: the counterexample (if there is one);
yices.log: communication log with Yices.
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 45 / 1
87. Fairness, Refinement, and Invariants
In the Byzantine case we have in transit : ∀i. (nrcvdi ≥ nsnt) and
G F ¬in transit.
In this case communication fairness implies computation fairness.
But in the abstract version nsnt can deviate from the number of
processes who sent the echo message.
In this case the user formulates a simple state invariant candidate,
e.g., nsnt = K([sv = SE ∨ sv = AC]) (on the level of the original
concrete system).
The tool checks automatically, whether the candidate is actually a
state invariant.
After the abstraction the abstract version of the invariant restricts the
behavior of the abstract transition system.
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 46 / 1
88. Parametric abstraction refinement — justice suppression
justice G F ¬in transit necessary to verify liveness
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 47 / 1
89. Parametric abstraction refinement — justice suppression
justice G F ¬in transit necessary to verify liveness
counter example:
in transit
in transit
in transit
in transit
in transit
in transit
in transit
s1 in transit
s2
sk
s3
· · ·
· · ·
· · ·
if ∀j all concretizations of sj violate ¬in transit, then CE is spurious.
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 47 / 1
90. Parametric abstraction refinement — justice suppression
justice G F ¬in transit necessary to verify liveness
counter example:
in transit
in transit
in transit
in transit
in transit
in transit
in transit
s1 in transit
s2
sk
s3
· · ·
· · ·
· · ·
if ∀j all concretizations of sj violate ¬in transit, then CE is spurious.
refine justice to G F ¬in transit ∧ G F
1≤j≤k
¬at(sj )
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 47 / 1
91. Parametric abstraction refinement — justice suppression
justice G F ¬in transit necessary to verify liveness
counter example:
in transit
in transit
in transit
in transit
in transit
in transit
in transit
s1 in transit
s2
sk
s3
· · ·
· · ·
· · ·
if ∀j all concretizations of sj violate ¬in transit, then CE is spurious.
refine justice to G F ¬in transit ∧ G F
1≤j≤k
¬at(sj )
. . . we use unsat cores to refine several loops at once
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 47 / 1
92. Parametric abstraction refinement — justice suppression
justice G F ¬in transit necessary to verify liveness
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 48 / 1
93. Parametric abstraction refinement — justice suppression
justice G F ¬in transit necessary to verify liveness
counter example:
in transit
in transit
in transit
in transit
in transit
in transit
in transit
s1 in transit
s2
sk
s3
· · ·
· · ·
· · ·
if ∀j all concretizations of sj violate ¬in transit, then CE is spurious.
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 48 / 1
94. Parametric abstraction refinement — justice suppression
justice G F ¬in transit necessary to verify liveness
counter example:
in transit
in transit
in transit
in transit
in transit
in transit
in transit
s1 in transit
s2
sk
s3
· · ·
· · ·
· · ·
if ∀j all concretizations of sj violate ¬in transit, then CE is spurious.
refine justice to G F ¬in transit ∧ G F
1≤j≤k
¬at(sj )
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 48 / 1
95. Parametric abstraction refinement — justice suppression
justice G F ¬in transit necessary to verify liveness
counter example:
in transit
in transit
in transit
in transit
in transit
in transit
in transit
s1 in transit
s2
sk
s3
· · ·
· · ·
· · ·
if ∀j all concretizations of sj violate ¬in transit, then CE is spurious.
refine justice to G F ¬in transit ∧ G F
1≤j≤k
¬at(sj )
. . . we use unsat cores to refine several loops at once
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 48 / 1
96. asynchronous reliable broadcast (srikanth toueg 1987)
the core of the classic broadcast algorithm from the da literature.
it solves an agreement problem depending on the inputs vi .
Variables of process i
vi : {0 , 1} init with 0 or 1
accepti : {0 , 1} init with 0
An indivisible step:
i f vi = 1
then send ( echo ) to all ;
i f received (echo) from at l e a s t
t + 1 distinct processes
and not sent ( echo ) before
then send ( echo ) to all ;
i f received ( echo ) from at l e a s t
n - t distinct processes
then accepti := 1;
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 49 / 1
97. asynchronous reliable broadcast (srikanth toueg 1987)
the core of the classic broadcast algorithm from the da literature.
it solves an agreement problem depending on the inputs vi .
Variables of process i
vi : {0 , 1} init with 0 or 1
accepti : {0 , 1} init with 0
An indivisible step:
i f vi = 1
then send ( echo ) to all ;
i f received (echo) from at l e a s t
t + 1 distinct processes
and not sent ( echo ) before
then send ( echo ) to all ;
i f received ( echo ) from at l e a s t
n - t distinct processes
then accepti := 1;
asynchronous
t byzantine faults
correct if n 3t
resilience condition rc
parameterized process
skeleton p(n, t)
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 49 / 1
98. Abstract CFA
qI
q0
q1
q2
q3
sv = V1
¬(sv = V1) inc nsnt
sv := SE
q4
q5
q6
q7
nrcvd := z where (nrcvd ≤ z ∧ z ≤ nsnt + f )
¬(t + 1 ≤ nrcvd)
t + 1 ≤ nrcvd
sv = V0
¬(sv = V0)
inc nsnt
n − t ≤ nrcvd
¬(n − t ≤ nrcvd)
sv := AC
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 50 / 1
99. Abstract CFA
qI
q0
q1
q2
q3
sv = V1
¬(sv = V1) inc nsnt
sv := SE
q4
q5
q6
q7
nrcvd := z where (nrcvd ≤ z ∧ z ≤ nsnt + f )
¬(t + 1 ≤ nrcvd)
t + 1 ≤ nrcvd
sv = V0
¬(sv = V0)
inc nsnt
n − t ≤ nrcvd
¬(n − t ≤ nrcvd)
sv := AC
qI
q0
q1
q2
q3
sv = V1
¬(sv = V1) inc nsnt
sv := SE
q4
q5
q6
q7
nrcvd = I0 ∧ nsnt = I0 ∧ (nrcvd = I0 ∨ nrcvd = I1) ∨ . . .
¬(t + 1 ≤ nrcvd)
nrcvd = It+1 ∨ nrcvd = In−t
sv = V0
¬(sv = V0)
nsnt = I1 ∧ (nsnt = I1 ∨ nsnt = It+1) ∨ . . .
n − t ≤ nrcvd
¬(n − t ≤ nrcvd)
sv := AC
Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 50 / 1