Your SlideShare is downloading.
×

- 1. Python metaprogramming in linear time language for automated runtime verification with graph neural networks Diploma thesis by: Dimitrios Karageorgiou (SRN: 8420) soulrain@outlook.com Supervisors: • Andreas Symeonidis (Associate Professor) • Emmanouil Krasanakis (PhD Candidate) Faculty of Engineering School of Electrical And Computers Engineering Department of Electronics and Computer Engineering Intelligent Systems and Software Engineering Labgroup Friday, 12th November 2021
- 2. In other words… Lovpy is a runtime logic verification library for Python. 2 Lovpy Logic verification becomes mainstream! Lovpy by Dimitrios Karageorgiou
- 3. What is logic verification? Runtime verification approach at its heart. ► E.g. a deadlock can be detected only at runtime. Extends verification techniques outside of strict software verification domain. E.g. enforcing best practices to the users of a library is also a logic problem. 3 Instrumentation Python System Monitor Specifications Verdicts Events Specification Violated or Specification Holds e.g. Lovpy by Dimitrios Karageorgiou
- 4. Not another runtime verification library... Only complex and application specific runtime verification libraries existed for Python. Design goals: Minimal user effort to enable verification, without required code modifications. Specifications in an easy-to-learn and intuitive language. Never report a violation that does not exist (0% false-negatives). Report violations before they happen (prevent side-effects). Report the last provably correct line of code (all specifications hold). 4 Lovpy by Dimitrios Karageorgiou
- 5. No code modifications required! Enable verification by just executing the library: python –m lovpy <script.py> 5 Lovpy by Dimitrios Karageorgiou ► Lovpy AOT Preprocessor handles the rest: Python Code Lovpy AOT Preprocessor Augmented Python Code Python Runtime Python Objects Augmented Objects Lovpy Execution
- 6. Augmented Python Objects Each Python object is augmented to hold its execution state: 6 Current System State Augmented Object 1 Object 1 State 1 Augmented Object 2 Object 2 State 2 Augmented Object N Object N State N Objects can live anywhere (multiple threads, processes etc.) Parallelization of original system is retained. Verification on per-object basis. Lovpy by Dimitrios Karageorgiou
- 7. Specifications in Gherkin 7 Gherkin is: Simple Easy-to-learn Intuitive 1. SCENARIO: 2. WHEN call acquire 3. THEN SHOULD NOT locked 4. AND locked 5. 6. SCENARIO: 7. GIVEN locked 8. WHEN call release 9. THEN NOT locked Lovpy by Dimitrios Karageorgiou
- 8. Everything is mathematically proved! Monitor utilizes an Automated Theorem Prover: 8 Monitor Current System State Theorems Automated Theorem Prover System State Builder Specifications Parser Properties to Prove Execution Events Specifications Verdicts Lovpy by Dimitrios Karageorgiou
- 9. Automated Theorem Proving 9 Lovpy by Dimitrios Karageorgiou
- 10. Everything is a Temporal Graph Execution States, Theorems and Properties are converted to Temporal Graphs. Temporal Graph: o A kind of Abstract Syntax Graph. o Nodes are either logical operators or predicates. o Edges contain timestamps. o Each timestamp is the most recent moment the subgraph holds. o Timestamps can be relative or absolute. o Definition of mathematically proved logic algorithms (logic graph removal/addition, graph modus ponens, etc.) 10 Lovpy by Dimitrios Karageorgiou
- 11. Everything is a Temporal Graph Execution States, Theorems and Properties are converted to Temporal Graphs. 11 … lock = threading.Lock() … lock.acquire() … lock.release() … lock.acquire() … Lovpy by Dimitrios Karageorgiou
- 12. Everything is a Temporal Graph Execution States, Theorems and Properties are converted to Temporal Graphs. 12 1. SCENARIO: 2. GIVEN locked 3. WHEN call release 4. THEN NOT locked Lovpy by Dimitrios Karageorgiou
- 13. Everything is a Temporal Graph Execution States, Theorems and Properties are converted to Timed Graphs. 13 1. SCENARIO: 2. WHEN call acquire 3. THEN SHOULD NOT locked 4. AND locked Part of conclusion that refers to the same time moment with assumption, is always proved and becomes a theorem. Lovpy by Dimitrios Karageorgiou
- 14. Proving process initialization 14 Lovpy by Dimitrios Karageorgiou Property to Prove Execution State Graph
- 15. Theorem application #1 15 Lovpy by Dimitrios Karageorgiou Applied Theorem #1 Execution State Graph #1
- 16. Theorem application #2 16 Lovpy by Dimitrios Karageorgiou Applied Theorem #2 Execution State Graph #2
- 17. Property proved! 17 Lovpy by Dimitrios Karageorgiou Proved property Final Execution State Graph
- 18. Violation detected! 18 Lovpy by Dimitrios Karageorgiou ► Last correct line reported too!
- 19. Improve theorem proving capability Deterministic next theorem selection: Next theorem to apply is the one whose assumption uses the oldest predicates. х Problem: Oldest theorem is not always the appropriate one to apply. Solution: Deep Learning and Graph Neural Networks ► Next theorem to apply is selected using a deep neural model. Use Graph Neural Networks to embed graph. Train model using synthetic theorems generated by: Lovpy Synthetic Theorems Generator 19 Lovpy by Dimitrios Karageorgiou
- 20. Deep Neural Architecture Overview 20 Lovpy by Dimitrios Karageorgiou Concatenation Current State Theorem Instance Goal Property Theorem N Theorem 2 Theorem 1 Score N Score 2 Score 1 Current State Graph Encoder Theorem Instance Graph Encoder Goal Property Graph Encoder Current State Graph Theorem Graph Goal Graph
- 21. Neural Graph Encoder 21 Lovpy by Dimitrios Karageorgiou
- 22. Evaluation 22 Evaluated five different architectures on proving 2.5k synthetic theorems. Correct Proofs Heuristic 70.7% MLP 59.25 MLP + Heuristic 70.7% GNN 61.6% GNN + Heuristic 73.8% Lovpy by Dimitrios Karageorgiou ► Detected common bugs of 5 different domains, in 20 erroneous python programs. Detected two bugs in popular open-source projects: Django Web Framework Keras
- 23. Future possibilities ► Public Lovpy’s Repository: Community written specifications for specific domains (e.g. best practices for using Tensorflow) Community trained neural models for theorem selection. Specifications mining for eliminating the need for hand-written specifications. Natural language support in Gherkin rules. 23 Lovpy by Dimitrios Karageorgiou
- 24. Questions??? 24 Thanks for watching! Lovpy by Dimitrios Karageorgiou
- 25. Slides Graveyard 25 Lovpy by Dimitrios Karageorgiou
- 26. Lovpy is available for everyone! ► Lovpy is available at PyPI: ► Also available as an open-source project on Github: 26 Lovpy by Dimitrios Karageorgiou python –m pip install lovpy https://github.com/dkarageo/lovpy
- 27. Detecting code violations 27 Detected two bugs in popular open-source projects: Django Web Framework Keras Violations Detected Threads Data Neural Math Common Total Heuristic 3 1 2 2 6 14 MLP 1 0 1 2 3 7 MLP + Heuristic 3 1 2 2 6 14 GNN 4 1 1 1 3 12 GNN + Heuristic 5 2 3 2 6 18 ► Detected common bugs of 5 different domains, in 20 erroneous python programs. Lovpy by Dimitrios Karageorgiou
- 28. Synthetic sample example 28 ► Generated by Lovpy Synthetic Theorems Generator. Lovpy by Dimitrios Karageorgiou