Методы формальной   верификации
Методы формальной   верификации        … в Java 8
Методы формальной   верификации        … в Java 8          … да и вообще...
Pол                                  евы                                игр   е                                    ы    Вл...
Program testing can be used to show thepresence of bugs, but never to show theirabsence.[“Structured programming”, Dahl O....
Testing                              isRunning the tested software   –   in different environment   –   with different dat...
Fundamental Test TheoremJust a few years after “Structured programming” ...We prove … that properly structured tests areca...
Fundamental Test Theorem              Program F(d) for domain D              Requirements: OUT(d, F(d)) = OK(d)           ...
But wait! Its not over yet!I hope to have convinced you that by its very natureresponsible system design and development m...
But wait! Its not over yet!"Arrogance in computer science is measured in               nano-Dijkstras."                   ...
But wait! Its not over yet!"Arrogance in computer science is measured in               nano-Dijkstras."                   ...
Dynamic         Testing                              isRunning the tested software   –   in different environment   –   wi...
Static testing                              isAnalysis of artifacts   –   source code   –   binaries   –   data filesin an...
Static testing                             includes●   Using static analyzers    –   Big number of false positives●   Code...
What defects          could by found by dynamic testing                Any defect! You just need to invent enough test :) ...
What defects              could by found by static testing                  Any defect!You just need to look on the whole ...
What defects                  are hard to find by dynamic testing●   Intermittent problems    –   You may just missed it● ...
What defects                   are hard to find by static analysis●   Bugs in deep and wide class inheritance    –   Virtu...
Formal verification                        isFormal verification is the act of proving ordisproving the correctness of int...
Formal verification vs Testing                                 is also●   Testing    –   Upper bound for program quality  ...
Formal verification                           requires●   Correctness of    –   Language    –   Compiler    –   “Core” of ...
Formal verification             appliedboolean isPowerOfTwo(int a) {    return (a&(a-1)) == 0;}
Formal verification                          applied                                                        n∀0< a∈ N : a ...
Formal verification                        isFormal verification is the act of proving ordisproving the correctness of int...
Deductive Verification                         Theorem proving●   Four color theorem (proved in 1976)●   Curry-Howard isom...
Using tools                           how about ...●   We create a program    –   Is capable of proving something about an...
Formal verification                     compiler is a prover on its own●   Formal verification for Java is performed by   ...
Annotations in Java@Stateless @LocalBeanpublic class GalleryFacade {  @EJB  private GalleryEAO galleryEAO;  @TransactionAt...
Annotations in Java●   Introduced in Java 5●   Metadata●   May be reflective    –   SOURCE, CLASS, RUNTIME●   Standard (e....
Annotations: pre-Java 8●   Allowed on declarations only    –   Class declaration          @A public class Test {          ...
Annotations: pre-Java 8●   Allowed on declarations only    –   Field declaration          @A public class Test {          ...
Annotations: pre-Java 8●   Allowed on declarations only    –   Method declaration          @A public class Test {         ...
Annotations: pre-Java 8●   Allowed on declarations only    –   Method parameter declaration          @A public class Test ...
Annotations: pre-Java 8●   Allowed on declarations only    –   Local variable declaration          @A public class Test { ...
Limitations●   Consider @NonNull annotation●   How to declare a Map with non-null keys and    values?
Limitations●   Consider @NonNull annotation●   How to declare a Map with non-null keys and    values?               @NonNu...
Limitations●   Consider @NonNull annotation●   How to declare a Map with non-null keys and    values?               @NonNu...
Limitations●   Consider @NonNull annotation●   How to declare a Map with non-null keys and    values?                 @Non...
Type annotations in Java 8●   Allowed anywhere you would write a type      … including generics and casts      … for array...
Type annotations in Java 8:               Examples●   Class inheritance      class UnmodifiableList<T>           implement...
Type annotations in Java 8:               Examples●   Generics      List<@Interned String> messages;●   Type parameter bou...
Pluggable types●   User-defined (pluggable) type system●   Extend built-in type system    – express extra information abou...
Checker Framework●   Collection of compiler plugins (“checkers”)●   Relies on Pluggable types and Type    Annotations in J...
Example: Nullness Checker●   Annotations    –   @NonNull    –   @Nullable●   Partial type hierarchy
Example: Nullness Checker●   Annotations    –   @NonNull    –   @Nullable●   Example:        @Nullable Object o1; // might...
Example: Nullness Checker●   Annotations    –   @NonNull    –   @Nullable●   Example:          public <@NonNull T> T proce...
Example: Tainting Checker●   Use case:    –   Trusted vs untrusted data    –   Verify before use●   Examples    –   SQL in...
Example: Tainting Checker●   Annotations    –   @Untainted        ●   A type that includes only untainted, trusted values ...
Example: Tainting Checker●   Annotations    –   @Untainted    –   @Tainted●   Example        void execute(@Untainted Strin...
Credit card number                 Annotation@Documented@Retention(RetentionPolicy.RUNTIME)@Target({ElementType.TYPE_USE, ...
Credit card number                Checker@TypeQualifiers(CreditCard.class)@SuppressWarningsKey("credit.card")public class ...
Credit card number                      Usagepublic class Account { private final @CreditCard String cardNumber; public Ac...
Credit card number                      Sources@SuppressWarnings("credit.card")@CreditCard String convert(String input) { ...
Credit card number                        Conclusion●   A card number in an account is always validated●   That is guarant...
More real life examplesString getProperty(@PropertyKey String key);HashMap <@Adult Person, @NonNull Address>    findSobuty...
Checkers Framework:                 Advanced features●   Linear checker    –   Implements linear types (based on linear lo...
How to start using●   No need to wait Java 8 release    –   modified compiler already available●   Incremental program ann...
Links●   Type Annotations Specification (JSR-308)      http://types.cs.washington.edu/jsr308/specification/java-●   Checke...
Q&A
Владимир Ивановvladimir.x.ivanov@oracle.com Александр Ильин alexandre.iline@oracle.com
Алло, мы ищем таланты!  Приходите к нам работать!   alexandre.iline@oracle.co                m
Upcoming SlideShare
Loading in …5
×

Формальная верификация как средство тестирования (в Java)

1,463 views
1,312 views

Published on

Доклад Владимира Иванова, Александра Ильина на конференции SQA Days-12, 30 ноября-1 декабря, Минск

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,463
On SlideShare
0
From Embeds
0
Number of Embeds
188
Actions
Shares
0
Downloads
23
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • A static type system helps programmers to detect and prevent errors. However, a language’s built-in type system does not help to detect and prevent enough errors, because it cannot express certain important invariants. A user-defined, or pluggable, type system enriches the built-in type system by expressing extra information about types via type qualifiers. Pluggable types permit more expressive compile-time checking and guarantee the absence of additional errors.
  • @GuardedBy(&quot;MyClass.myLock&quot;) Object myMethod() { ... } // reassignments without holding the lock are OK. @GuardedBy(&quot;MyClass.myLock&quot;) Object x = myMethod(); @GuardedBy(&quot;MyClass.myLock&quot;) Object y = x; Object z = x; // ILLEGAL (assuming no lock inference), // because z can be freely accessed. x.toString() // ILLEGAL because the lock is not held synchronized(MyClass.myLock) { y.toString(); // OK: the lock is held } =================================================== void helper1(@GuardedBy(&quot;MyClass.myLock&quot;) Object a) { a.toString(); // ILLEGAL: the lock is not held synchronized(MyClass.myLock) { a.toString(); // OK: the lock is held } } @Holding(&quot;MyClass.myLock&quot;) void helper2(@GuardedBy(&quot;MyClass.myLock&quot;) Object b) { b.toString(); // OK: the lock is held } void helper3(Object c) { helper1(c); // OK: passing a subtype in place of a the @GuardedBy supertype c.toString(); // OK: no lock constraints } void helper4(@GuardedBy(&quot;MyClass.myLock&quot;) Object d) { d.toString(); // ILLEGAL: the lock is not held } void myMethod2(@GuardedBy(&quot;MyClass.myLock&quot;) Object e) { helper1(e); // OK to pass to another routine without holding the lock e.toString(); // ILLEGAL: the lock is not held synchronized (MyClass.myLock) { helper2(e); helper3(e); helper4(e); // OK, but helper4&apos;s body still does not type-check } }
  • @GuardedBy(&quot;MyClass.myLock&quot;) Object myMethod() { ... } // reassignments without holding the lock are OK. @GuardedBy(&quot;MyClass.myLock&quot;) Object x = myMethod(); @GuardedBy(&quot;MyClass.myLock&quot;) Object y = x; Object z = x; // ILLEGAL (assuming no lock inference), // because z can be freely accessed. x.toString() // ILLEGAL because the lock is not held synchronized(MyClass.myLock) { y.toString(); // OK: the lock is held } =================================================== void helper1(@GuardedBy(&quot;MyClass.myLock&quot;) Object a) { a.toString(); // ILLEGAL: the lock is not held synchronized(MyClass.myLock) { a.toString(); // OK: the lock is held } } @Holding(&quot;MyClass.myLock&quot;) void helper2(@GuardedBy(&quot;MyClass.myLock&quot;) Object b) { b.toString(); // OK: the lock is held } void helper3(Object c) { helper1(c); // OK: passing a subtype in place of a the @GuardedBy supertype c.toString(); // OK: no lock constraints } void helper4(@GuardedBy(&quot;MyClass.myLock&quot;) Object d) { d.toString(); // ILLEGAL: the lock is not held } void myMethod2(@GuardedBy(&quot;MyClass.myLock&quot;) Object e) { helper1(e); // OK to pass to another routine without holding the lock e.toString(); // ILLEGAL: the lock is not held synchronized (MyClass.myLock) { helper2(e); helper3(e); helper4(e); // OK, but helper4&apos;s body still does not type-check } }
  • @GuardedBy(&quot;MyClass.myLock&quot;) Object myMethod() { ... } // reassignments without holding the lock are OK. @GuardedBy(&quot;MyClass.myLock&quot;) Object x = myMethod(); @GuardedBy(&quot;MyClass.myLock&quot;) Object y = x; Object z = x; // ILLEGAL (assuming no lock inference), // because z can be freely accessed. x.toString() // ILLEGAL because the lock is not held synchronized(MyClass.myLock) { y.toString(); // OK: the lock is held } =================================================== void helper1(@GuardedBy(&quot;MyClass.myLock&quot;) Object a) { a.toString(); // ILLEGAL: the lock is not held synchronized(MyClass.myLock) { a.toString(); // OK: the lock is held } } @Holding(&quot;MyClass.myLock&quot;) void helper2(@GuardedBy(&quot;MyClass.myLock&quot;) Object b) { b.toString(); // OK: the lock is held } void helper3(Object c) { helper1(c); // OK: passing a subtype in place of a the @GuardedBy supertype c.toString(); // OK: no lock constraints } void helper4(@GuardedBy(&quot;MyClass.myLock&quot;) Object d) { d.toString(); // ILLEGAL: the lock is not held } void myMethod2(@GuardedBy(&quot;MyClass.myLock&quot;) Object e) { helper1(e); // OK to pass to another routine without holding the lock e.toString(); // ILLEGAL: the lock is not held synchronized (MyClass.myLock) { helper2(e); helper3(e); helper4(e); // OK, but helper4&apos;s body still does not type-check } }
  • Формальная верификация как средство тестирования (в Java)

    1. 1. Методы формальной верификации
    2. 2. Методы формальной верификации … в Java 8
    3. 3. Методы формальной верификации … в Java 8 … да и вообще...
    4. 4. Pол евы игр е ы Владимир Иванов Разработчик HotSpot JVM Хардкорный девелопер Александр ИльинАрхитектор тестирования Oracle JDK Тролль из отдела тестирования
    5. 5. Program testing can be used to show thepresence of bugs, but never to show theirabsence.[“Structured programming”, Dahl O.J., Dijkstra E.W. and Hoare C.A.R.] (1972)[When] you have given the proof of [aprograms] correctness, … [you] candispense with testing altogether.[“Software engineering”, Naur P., Randell B.] (1969)
    6. 6. Testing isRunning the tested software – in different environment – with different datain an attempt to – Certify conformance – Prove program correctness – Prove incorrectness
    7. 7. Fundamental Test TheoremJust a few years after “Structured programming” ...We prove … that properly structured tests arecapable of demonstrating the absence oferrors in a program.[“Toward a Theory of Test Data Selection”, John B. Goodenough, Susan L.Gerhart] (1975)
    8. 8. Fundamental Test Theorem Program F(d) for domain D Requirements: OUT(d, F(d)) = OK(d) Data selection criteria: CCOMPLETE (T ,C )=(∀d ∈T OK (d )⇒∀d ∈ D OK (d ))∨(∀d ∈T ¬OK (d )⇒∀d ∈ D ¬OK (d )) SUCCESSFUL(T )=∀t ∈T OK (t) RELIABLE (C )=(∀T1 ,T2⊂ D)COMLPETE (T1 ,C )∧COMPLETE (T2 , C )⇒ (SUCCESSFUL(T1)≡SUCCESSFUL(T2)) VALID(C )=∀d ∈ D ¬OK (d )⇒(∃T ⊆ D)(COMPLETE (T ,C )∧¬SUCCESSFUL(T )) ∃T ⊆ D ,∃C (COMPLETE (T ,C )∧RELIABLE (C )∧VALID(C )∧SUCCESSFUL(T ))⇒ ∀d ∈ D OK (d )
    9. 9. But wait! Its not over yet!I hope to have convinced you that by its very natureresponsible system design and development mustbe an activity of an undeniably mathematical nature.… programming became an industrial activity at amoment that the American manager was extremelyfearful of relying on the education and theintelligence of his companys employees. Andmanagement tried to organize the industrialprogramming task in such a way that eachindividual programmer would need to think as littleas possible.[“Why correctness must be a mathematical concern” E. W Dijkstra] (1979)
    10. 10. But wait! Its not over yet!"Arrogance in computer science is measured in nano-Dijkstras." Alan Kay
    11. 11. But wait! Its not over yet!"Arrogance in computer science is measured in nano-Dijkstras." Alan Kay "… and micro-Kays". Unknown source ;-)
    12. 12. Dynamic Testing isRunning the tested software – in different environment – with different datain an attempt to – Certify conformance – Prove program correctness (requires formal proof) – Prove program incorrectness (practically)
    13. 13. Static testing isAnalysis of artifacts – source code – binaries – data filesin an attempt to – discover errors – identify suspicious patterns – verify conformance of the artifacts
    14. 14. Static testing includes● Using static analyzers – Big number of false positives● Code reviews – Tedious manual work – Many errors missed – Non formalizable
    15. 15. What defects could by found by dynamic testing Any defect! You just need to invent enough test :) only ...It may take an indefinite amount of tests So, the testing is, effectively, endless
    16. 16. What defects could by found by static testing Any defect!You just need to look on the whole source long enough only ...You can not know which ones you are detecting and You never know how many are left
    17. 17. What defects are hard to find by dynamic testing● Intermittent problems – You may just missed it● Platform/environment specific problem – You just may not have the environment
    18. 18. What defects are hard to find by static analysis● Bugs in deep and wide class inheritance – Virtual methods are resolved in runtime● Bugs in modular system – Many modules implement the same features, modules are registered somewhere, etc. – Same reason – modules are registered as runtime
    19. 19. Formal verification isFormal verification is the act of proving ordisproving the correctness of intended algorithmsunderlying a system with respect to a certainformal specification or property, usingformal methods of mathematics.
    20. 20. Formal verification vs Testing is also● Testing – Upper bound for program quality ● Passed test says nothing about quality ● What matters is when test fails● Formal verification – Lower bound for program quality ● Passed test guarantees absence of some type of failures in a program
    21. 21. Formal verification requires● Correctness of – Language – Compiler – “Core” of the program● The specification is self-consistent
    22. 22. Formal verification appliedboolean isPowerOfTwo(int a) { return (a&(a-1)) == 0;}
    23. 23. Formal verification applied n∀0< a∈ N : a &(a−1)=0⇔∃ n∈ N : a=2a > 0 => binary presentation of a has a least one 1 bit m >= 0 Lets take a binary representation of a: (a1…ak)10....0 m m a-1 = (a1…ak)01....1 => a&(a-1) = (a1…ak)00....0 a&(a-1) = 0 => a1,...,ak = 0 => a = 2m a = 2n => m=n, a1,...,ak = 0 => a&(a-1) = 0
    24. 24. Formal verification isFormal verification is the act of proving ordisproving the correctness of intended algorithmsunderlying a system with respect to a certainformal specification or property, usingformal methods of mathematics.Another approach is deductive verification. Itconsists of generating from the system and itsspecifications (and possibly other annotations) acollection of mathematical proof obligations,the truth of which imply conformance of thesystem to its specification.
    25. 25. Deductive Verification Theorem proving● Four color theorem (proved in 1976)● Curry-Howard isomorphism – (Theorem, Proof) <=> (Type, Program)● Theorem provers – Interactive environments for constructing proofs – Coq, Agda, Isabelle, HOL● Real-world example – COMPCERT: C Verified Compiler
    26. 26. Using tools how about ...● We create a program – Is capable of proving something about another program – Is itself proven (yeah, yeah, a recursion)● Use the program to prove something about another program● Lets call it a “prover” Is this still a formal verification? Sure!
    27. 27. Formal verification compiler is a prover on its own● Formal verification for Java is performed by Java compiler – Types – Uninitialized variable – Missing of return statement – Uncaught exceptions – etc. – etc.
    28. 28. Annotations in Java@Stateless @LocalBeanpublic class GalleryFacade { @EJB private GalleryEAO galleryEAO; @TransactionAttribute(SUPPORTS) public Gallery findById(Long id) { ... } @TransactionAttribute(REQUIRED) public void create(String name) { … }
    29. 29. Annotations in Java● Introduced in Java 5● Metadata● May be reflective – SOURCE, CLASS, RUNTIME● Standard (e.g. @Override) & custom annotations● Extensively used nowadays – JavaEE 6, IoC containers, test harnesses, etc
    30. 30. Annotations: pre-Java 8● Allowed on declarations only – Class declaration @A public class Test { @B private int a = 0; @C public void m(@D Object o) { @E int a = 1; ... } }
    31. 31. Annotations: pre-Java 8● Allowed on declarations only – Field declaration @A public class Test { @B private int a = 0; @C public void m(@D Object o) { @E int a = 1; ... } }
    32. 32. Annotations: pre-Java 8● Allowed on declarations only – Method declaration @A public class Test { @B private int a = 0; @C public void m(@D Object o) { @E int a = 1; ... } }
    33. 33. Annotations: pre-Java 8● Allowed on declarations only – Method parameter declaration @A public class Test { @B private int a = 0; @C public void m(@D Object o) { @E int a = 1; ... } }
    34. 34. Annotations: pre-Java 8● Allowed on declarations only – Local variable declaration @A public class Test { @B private int a = 0; @C public void m(@D Object o) { @E int a = 1; ... } }
    35. 35. Limitations● Consider @NonNull annotation● How to declare a Map with non-null keys and values?
    36. 36. Limitations● Consider @NonNull annotation● How to declare a Map with non-null keys and values? @NonNull Map<K,V>?
    37. 37. Limitations● Consider @NonNull annotation● How to declare a Map with non-null keys and values? @NonNull Map<K,V>? NO!
    38. 38. Limitations● Consider @NonNull annotation● How to declare a Map with non-null keys and values? @NonNull Map<K,V>? NO!● Map<@NonNull K, @NonNull V> … but incorrect in Java 7 and earlier Type annotations in Java 8 for the rescue!
    39. 39. Type annotations in Java 8● Allowed anywhere you would write a type … including generics and casts … for array levels and receivers
    40. 40. Type annotations in Java 8: Examples● Class inheritance class UnmodifiableList<T> implements @ReadOnly List<T> { ... }● Casts myDate = (@ReadOnly Date) roDate;● Type tests myString instanceof @NonNull String;● Arrays String @NonNull [] messages;
    41. 41. Type annotations in Java 8: Examples● Generics List<@Interned String> messages;● Type parameter bounds Collection<? super @Exists File>● Generic type arguments in a generic method o.<@NonNull String>toString("...");
    42. 42. Pluggable types● User-defined (pluggable) type system● Extend built-in type system – express extra information about types via type qualifiers● Permit more expressive compile-time checking and guarantee the absence of additional errors
    43. 43. Checker Framework● Collection of compiler plugins (“checkers”)● Relies on Pluggable types and Type Annotations in Java 8● Find different sorts of bugs or verify their absence – 14 checkers are already provided● Supports custom compiler plugins (provides API) – 5 third-party checkers
    44. 44. Example: Nullness Checker● Annotations – @NonNull – @Nullable● Partial type hierarchy
    45. 45. Example: Nullness Checker● Annotations – @NonNull – @Nullable● Example: @Nullable Object o1; // might be null @NonNull Object o2; // never null o1.toString(); // warning o2 = o1; // warning if (o2 == null) // warning: redundant test
    46. 46. Example: Nullness Checker● Annotations – @NonNull – @Nullable● Example: public <@NonNull T> T process(T);
    47. 47. Example: Tainting Checker● Use case: – Trusted vs untrusted data – Verify before use● Examples – SQL injection attack ● validate SQL query before executing it – information leakage ● secret data vs data displayed to a user
    48. 48. Example: Tainting Checker● Annotations – @Untainted ● A type that includes only untainted, trusted values – @Tainted ● A type that includes only tainted, untrusted values
    49. 49. Example: Tainting Checker● Annotations – @Untainted – @Tainted● Example void execute(@Untainted String sql) throws SQLException; @Untainted String validate(@Tainted String) throws SQLException;
    50. 50. Credit card number Annotation@Documented@Retention(RetentionPolicy.RUNTIME)@Target({ElementType.TYPE_USE, ElementType.TYPE_PARAMETER})@TypeQualifier@SubtypeOf(Unqualified.class)public @interface CreditCard {}
    51. 51. Credit card number Checker@TypeQualifiers(CreditCard.class)@SuppressWarningsKey("credit.card")public class CreditCardChecker extends BaseTypeChecker {…}
    52. 52. Credit card number Usagepublic class Account { private final @CreditCard String cardNumber; public Account(@CreditCard String number) { this.cardNumber = number; } public @CreditCard String getCardNumber() { return cardNumber; }
    53. 53. Credit card number Sources@SuppressWarnings("credit.card")@CreditCard String convert(String input) { if(checkLuhn(input)) return input; else throw IllegalArgumentException("...")}new Account("4111111111111111");new Account("4111111111111110");
    54. 54. Credit card number Conclusion● A card number in an account is always validated● That is guaranteed at compile time● You do not need to test with invalid numbers● You do need to test – All @SuppressWarnings("credit.card") – checkLuhn(String cardNum)● Better all … prove it!
    55. 55. More real life examplesString getProperty(@PropertyKey String key);HashMap <@Adult Person, @NonNull Address> findSobutylnik(@NonNull Location);void monitorTemperature() throws @Critical TemperatureException;
    56. 56. Checkers Framework: Advanced features● Linear checker – Implements linear types (based on linear logic) – control aliasing and prevent re-use – Single ownership abstraction ● Prevents absence of ownership and multiple owners● Dependent types – @Dependent annotation – Changes the type depending on qualified type of the receiver (this) – Example List[N] – list with its length encoded into its type
    57. 57. How to start using● No need to wait Java 8 release – modified compiler already available● Incremental program annotation – Partial program checking – Warnings during compilation – Easily convertible into compilation errors ● -Werror flag to javac – Default annotations for types w/o annotations● Ability to annotate external libraries
    58. 58. Links● Type Annotations Specification (JSR-308) http://types.cs.washington.edu/jsr308/specification/java-● Checker Framework http://types.cs.washington.edu/checker-framework/curre
    59. 59. Q&A
    60. 60. Владимир Ивановvladimir.x.ivanov@oracle.com Александр Ильин alexandre.iline@oracle.com
    61. 61. Алло, мы ищем таланты! Приходите к нам работать! alexandre.iline@oracle.co m

    ×