import org.scalacheck._  BeScala – February 2012         Gilles Scouvart
Once upon a time…• Three scientists (an astrophysicist, a physicist and a  mathematician) are in a bus to a conference in ...
The moral of the story• We’re often relying on a limited set of simple  examples to validate our assumptions• But our impl...
How can we be sure?• We would like to check our assumptions for a « general  case » (intension)   – This can only be done ...
Main ideaIn ScalaCheck, we check properties on random         datasets created by generators.
Properties• Logical statements that the function must satisfy, e.g.                 Math                                  ...
Generators• org.scalacheck.Gen• Basically a function Params => Option[T]   – Params: size + random number generator• Built...
Generator example• Suppose we have   case class Person(name:String,age:Int)• We can construct a Gen for Person   val perso...
Generator example (cont’d)• Now if we only want young people  val youngPeople = personGen.filter(_.age<31)• We can use it ...
Complex failures• Suppose you have a property that relies on  complex data (e.g. a list)   (l:List[Int]) => l.size==l.dist...
How did this happen?• ScalaCheck applies shrinking strategies to  gradually simplify the problem to a simplest  failing ca...
Integration with Scala frameworks• ScalaTest  – prop.Checkers trait    • check method      (but cannot use the should/must...
Example• Scampi               class AlgebraTest extends FeatureSpec with Checkers with Algebra {  – Library for         //...
Other nice stuff• collect to store generated values• classify to make histograms• Stateful testing  – Finite State Machine...
Pros• Enhance readability  – Very declarative, appeals to mathematically-inclined  – Domain experts can read and comment• ...
Cons• It can be difficult to    – Find « good » properties        • Not trivially satisfied        Reflection on the domai...
Credits• Inspiration   – Haskell library QuickCheck (Koen Claessen, John Hughes)• Main contributors   –   Ricky Nilsson   ...
Thank you!
Upcoming SlideShare
Loading in …5
×

ScalaCheck

4,160 views

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
4,160
On SlideShare
0
From Embeds
0
Number of Embeds
1,038
Actions
Shares
0
Downloads
32
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

ScalaCheck

  1. 1. import org.scalacheck._ BeScala – February 2012 Gilles Scouvart
  2. 2. Once upon a time…• Three scientists (an astrophysicist, a physicist and a mathematician) are in a bus to a conference in Scotland• Shortly after the border, they see a black sheep on a hillside.• The astrophysicist declares: « In Scotland, all sheep are black! »• The physicist then says: « Hm, all we can say is that in Scotland, some sheep are black. »• The mathematician sighs and concludes: « All we can tell is that in Scotland, there exists one sheep with at least one black side! »
  3. 3. The moral of the story• We’re often relying on a limited set of simple examples to validate our assumptions• But our implementation might totally overfit our test set (particularly in TDD)• And edge cases might go completely unnoticed… Can we do better?
  4. 4. How can we be sure?• We would like to check our assumptions for a « general case » (intension) – This can only be done through formal verification of the code – This cannot be automatized for Turing-complete languages (cf. halting problem)• The alternative is then to generate explicitly all possible configurations (extension) and test each of them – But the number of configurations can be huge or infinite• Now if our code passes the test on 100 randomly chosen examples, we might still get some good level of confidence in our implementation – This is just what ScalaCheck proposes
  5. 5. Main ideaIn ScalaCheck, we check properties on random datasets created by generators.
  6. 6. Properties• Logical statements that the function must satisfy, e.g. Math Scala i  , 2*i == i+i forAll((i:Int) => 2*i == i+i) exists((i:Int) => 2*i == 2) i  , such that 2*i==2 forAll{(i:Int,j:Int)=> i,j  , (i*j) == 0  i == 0 || j == 0 (i*j)==0 ==> (i==0 || j==0)} Try those, you might have some surprises…• Elements – Quantifier: forall, exists, atLeast, iff, imply… – Assertion: boolean function• org.scalacheck.Prop
  7. 7. Generators• org.scalacheck.Gen• Basically a function Params => Option[T] – Params: size + random number generator• Built-in generators – choose: integer in range – oneOf : element from sequence of T or Gen[T] – frequency : oneOf taking weights into account – arbitrary : arbitrary instance (incl. edge cases)• Gen is a Monad! – map, flatMap, filter
  8. 8. Generator example• Suppose we have case class Person(name:String,age:Int)• We can construct a Gen for Person val personGen = for (n<-arbitrary[String];a<- choose(1,125)) yield Person(n,a)• We can test it using the sample function personGen.sample Not «?», but Unicode! res: Option[Person] = Some(Person(???????????????,85))• We can use it to test properties on Person Prop.forAll(personGen)((p:Person) => p.age>0)• If we want to use it implicitly we can declare it as an arbitrary generator implicit val arbPerson = Arbitrary(personGen) Prop.forAll((p:Person) => p.age>0)
  9. 9. Generator example (cont’d)• Now if we only want young people val youngPeople = personGen.filter(_.age<31)• We can use it for some specific tests Prop.forAll(youngPeople)((p:Person) => p.age<50)• If we want a list of 3 young people, we can write val trio = Gen.listOfN(3,youngPeople) trio.sample res: Option[List[Person]] = Some(List(Person(????????????????????????,4), Person( ???????????????,9), Person(???????????,20))))
  10. 10. Complex failures• Suppose you have a property that relies on complex data (e.g. a list) (l:List[Int]) => l.size==l.distinct.size• Now this obviously fails for list with duplicates• ScalaCheck could give the first failure ! Falsified after 5 passed tests. > ARG_0: List("-2147483648", "1", "1933529754", "-726958561", "- 2147483648”, "750300922", "841716922", "- 2147483648", "1671473995")• But it gives instead ! Falsified after 8 passed tests. > ARG_0: List("-1", "-1")
  11. 11. How did this happen?• ScalaCheck applies shrinking strategies to gradually simplify the problem to a simplest failing case• Standard shrinking strategies are provided for most common cases – List – Tuple – String – Int – Container
  12. 12. Integration with Scala frameworks• ScalaTest – prop.Checkers trait • check method (but cannot use the should/must matcher syntax)• Specs2 – check function
  13. 13. Example• Scampi class AlgebraTest extends FeatureSpec with Checkers with Algebra { – Library for // [Generators for E…] def associativity(op: (E, E) => E) = (a: E, b: E, c: E) => simplify(op(op(a, b), c)) == simplify(op(a, op(b, c))) Operations Research def commutativity(op: (E, E) => E) = (a: E, b: E) => simplify(op(a, b)) == simplify(op(b, a)) – Abstract algebra feature ("Addition") { using GADTs val addition = (e1: E, e2: E) => e1 + e2 scenario ("scalars") { – Very } check((i: Int, j: Int) => Const(i) + Const(j) == Const(i + j)) mathematical scenario ("associativity") { check(associativity(addition)) – Perfect fit } scenario ("commutativity") { check(commutativity(addition)) } } //… } hg clone https://bitbucket.org/pschaus/scampi
  14. 14. Other nice stuff• collect to store generated values• classify to make histograms• Stateful testing – Finite State Machines – Types • State • Command
  15. 15. Pros• Enhance readability – Very declarative, appeals to mathematically-inclined – Domain experts can read and comment• Focuses on properties of input and output – Instead of concrete examples• Automatically explores edge cases – Instead of tedious, incomplete and error-prone code• Shrinks to simplest failing examples Domain knowledge Active documentation
  16. 16. Cons• It can be difficult to – Find « good » properties • Not trivially satisfied Reflection on the domain • Not too complex to observe – Create « good » generators • Relevant edge cases Reflection on the specifications – Write checking functions • Implementing the check might be as error-prone as writing the function! Focus on variations/set of properties• Requires functional code Good programming practice• Beware of shrinking – If your property relies on the size of your instances
  17. 17. Credits• Inspiration – Haskell library QuickCheck (Koen Claessen, John Hughes)• Main contributors – Ricky Nilsson – Tony Morris – Paul Phillips – Yuvi Masonry• Links – GitHub: https://github.com/rickynils/scalacheck – User guide: http://code.google.com/p/scalacheck/wiki/UserGuide – Articles: http://code.google.com/p/scalacheck/wiki/ScalaCheckArticles
  18. 18. Thank you!

×