Property-based testing involves generating random inputs and checking that the output of a function satisfies desired properties or laws. It works by specifying properties as assertions that should hold true for all inputs. Testing libraries like ScalaCheck, QuickCheck, and Hypothesis can automatically generate random test values and attempt to falsify the properties. This helps test a wide range of inputs and edge cases. The document provides examples of using property tests to check if a sorting algorithm produces a sorted output and to validate data formats meet constraints. It also discusses generating customized input models and common properties like order relationships.

1. — Ruslan Shevchenko [iov42; codespace]
// @rssh1, ruslan@shevchenko.kiev.ua
PROPERTY BASED TESTING
”
“

2. WHAT IS IT ?
PROPERTY BASED TESTING
Eng: property
властивість
свойство
Other : example
приклад
пример
for all x: Int x < (x+1)
3 < (3 + 1)

3. SHOW ME THE CODE/SCALA
import org.scalacheck.Properties
import org.scalacheck.Prop.forAll
object IntSpecification extends Properties("Int")
{
property("x < x+1") =
forAll{ (x:Int) => x < x+1 }
}

4. SHOW ME THE CODE/SCALA
import org.scalacheck.Properties
import org.scalacheck.Prop.forAll
object IntSpecification extends Properties("Int")
{
property(“x < x+1") =
forAll{ (x:Int) => x < x+1 }
}
[info] ! Int.x > x+1: Falsified after 3 passed tests.
[info] > ARG_0: 2147483647
[info] Failed: Total 1, Failed 1, Errors 0, Passed 0
[error] Failed tests:
[error] IntSpecification
[error] (test:test) sbt.TestsFailedException: Tests unsuccessfu

5. SHOW ME THE CODE/JAVA
…..
@RunWith(JUnitQuickcheck.class) public class IntPropertyTest {
@Property public void xLessIncr(int x)
{
assert(x < x+1);
}
}

6. SHOW ME THE CODE/JAVA
…..
@RunWith(JUnitQuickcheck.class) public class IntPropertyTest {
@Property public void xLessIncr(int x)
{
assert(x < x+1);
}
}
[info] Running com.github.rssh.IntPropertyTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0,
Time elapsed: 0.75 sec
// Problem not found.
// Generation of random values are differ ;)

7. SHOW ME THE CODE/JAVASCRIPT
var jsc = require("jsverify")
describe('double ceiling', function() {
it ('x < x + 1e-15',function() {
expect(
jsc.checkForall(jsc.number,function(x){
return x < x+1e-15
})
).toBe(true)
});
});
// explore the limits of the floating point ceiling.

8. SHOW ME THE CODE/JAVASCRIPT
var jsc = require("jsverify")
describe('double ceiling', function() {
it ('x < x + 1e-15',function() {
expect(
jsc.checkForall(jsc.number,function(x){
return x < x+1e-15
})
).toBe(true)
});
});
// explore the limits of the floating point ceiling.
Failed after 8 tests and 1 shrinks. rngState:
0879d03f2e184b6c5a; Counterexample: 19.7693584933877;
[ 19.7693584933877 ]
F
Failures:
1) double ceiling x < x + 1e-15
1.1) Expected Object({ counterexample: [ 19.7693584933877 ],
counterexamplestr: '19.7693584933877', shrinks: 1, exc:
false, tests: 8, rngState: '0879d03f2e184b6c5a' }) to be
true.

9. MONKEY TESTING
PROPERTY-BASED TESTING
We generate random values.
Pass one to our function or service to test
Expect, that result of this function satisfy some properties
laws (classical property-based testing); crash
The Infinite Monkey Theorem

10. HISTORY
QuickCheck. Haskell. 1999
• https://en.wikipedia.org/wiki/QuickCheck
• http://www.cse.chalmers.se/~rjmh/QuickCheck/
John Hughes founded QuivQ
• commercial version of QuickCheck for Erlang
• extensions/interfaces for
• C/C++
• WebServices
• Industry specific standards … (appl. Volvo cars)

11. IMPLEMENTATIONS FOR POPULAR LANGUAGES
Scala: scalacheck
http://www.scalacheck.org
Java: junit-quickcheck
http://pholser.github.io/junit-quickcheck/
JavaScript: jsverify
http://jsverify.github.io/
Python: Hypothesis
http://hypothesis.works/
PHP: eris; phpquickchek
https://github.com/giorgiosironi/eris
https://github.com/steos/php-quickcheck
…………………… practically any language.

12. TYPICAL USAGE
Thinks and describe laws
(dependencies between input & output)
Tune input generators
Check such properties in output
Demos:
Programming tasks (sorting, serialization)
Form validation
State checks

13. LET’S CHECK SORTED ALGORITHM
def sort(x:IndexedSeq[T]):IndexedSeq[T]
invariant: length
definition of sorted:
idempotency:

14. LET’S CHECK SORTED ALGORITHM
def sort(x:Vector[T]): Vector[T]
invariant: length
property(“sorted array must the same size of origin”) =
forAll{ (x: Vector[Int]) => x.size == sort(x).size }

15. LET’S CHECK SORTED ALGORITHM
We need to generate:
• n - max number of entries
• sorted vector of the length n
• i < n
• j: i < j < n

16. CUSTOM GENERATOR
val orderedVectorGen = for {
n <- Gen.choose(2,100)
x <- Gen.containerOfN[Vector,Int]
i <- Gen.choose(0,n-2)
j <- Gen.choose(i,n-1)
} yield (n, sort(x), i, j)

17. CUSTOM GENERATOR
val orderedVectorGen = for {
n <- Gen.choose(2,100)
x <- Gen.containerOfN[Vector,Int]
i <- Gen.choose(0,n-2)
j <- Gen.choose(i,n-1)
} yield (n, sort(x), i, j)
property(“sorted array must reflect ordered indexes”) =
forAll(orderedVectorGen){ case (n, x, i, j) => x(i) <= x(j) }
property test can serve as documentation for input and output.

18. LAWS
Ordering[T]:
• reflexivity:
forall{ x:T => x <= x}
• anitsimmetry:
forall{ x:T, y:T =>
x <= y & y <= x ==> x==y }
• transitivity:
• forall{ x:T, y:T, z: T =>
x <= y & y <= z ==> x <= z }
// Laws = set of properties.
// Can be used as specification for type argument.

19. EXAMPLE: DATA VALIDATION
case class Subsriber(id:Long, // must be unique
firstName: String, // max 20 sym, no spaces.
lastName:String // max 20 sym, no space
)
val correctSubscriberGen = for {
id <- arbitrary[Long]
firstName <- Gen.alphaStr suchThat (_.length <= FIRST_NAME_LEN)
lastName <- Gen.alphaStr suchThat (_.length <= LAST_NAME_LEN)
} yield Subscriber(id,firstName, lastName)
val incorrectSubscriberGen = for {
id <- arbitrary[Long]
firstName <- incorrectNameGen(FIRST_NAME_LEN)
………

20. GENERAL QUESTIONS
Is property-based testing useful itself ?
— definitely yes.
— force to think hight-level
Are we steel need example-based testing ?
— probably yes.
• What to manual construct thing, which
• explore some type of bug.
• Dimension Problem.
• N probes can be very tiny part.

21. QUESTIONS ?
THANKS FOR LISTENING
Full example code is available on github:
https://github.com/rssh/property-based-testing-talk
@rssh1
Ruslan Shevchenko
<ruslan@shevchenko.kiev.ua>