The document discusses property-based testing and how it can be used to test code more effectively than example-based testing. It introduces the concepts of testing properties like commutativity, associativity, and identity to specify requirements at a deeper level. QuickCheck is presented as a tool that generates random test values to check properties, finding bugs more quickly than testing by hand. JUnit Quickcheck and JSVerify are given as libraries for implementing property tests in Java and JavaScript. Potential applications of property testing like testing SDKs and data processing frameworks are outlined.

1. Property-based Testing
for Everyone
Dmitriy Morozov / Zoomdata Tech Lunch

2. Why is testing hard?

3. Example-based
Testing
• set up one input scenario
• run the code under test
• check the output
(and any other effects the code is supposed to have)

4. Imagine we needed to
implement a custom ADD
function

5. @Test
public void testSum() {
assertThat(sum(1, 2), equalTo(3));
}

6. private int sum(int a, int b) {
if (a == 1 && b == 2) {
return 3;
} else {
return 0;
}
}

7. @Test
public void testSum() {
assertThat(sum(2, 2), equalTo(4));
}

8. private int sum(int a, int b) {
if (a == 1 && b == 2) {
return 3;
} else if (a == 2 && b == 2) {
return 4;
} else {
return 0;
}
}

9. Enterprise Developer

10. @Test
public void testSum3() {
// When I add two numbers I expect their sum
for (int i = 0; i < 100; i++) {
int x = randomInt();
int y = randomInt();
assertThat(sum(x, y), equalTo(x + y));
}
}

11. @Test
public void testSum3() {
// When I add two numbers I expect their sum
for (int i = 0; i < 100; i++) {
int x = randomInt();
int y = randomInt();
assertThat(sum(x, y), equalTo(x + y));
}
}

12. @Test
public void testSumUsingCommutativityProperty() {
// Changing the order of arguments
// shouldn’t change their sum
for (int i = 0; i < 100; i++) {
int x = randomInt();
int y = randomInt();
assertThat(sum(x, y), equalTo(sum(y, x)));
}
}

13. // unfortunately that's not enough
private int sum(int a, int b) {
return a * b;
}

14. @Test
// `x + 2` is the same as `x + 1 + 1`
public void testSumWithAdditiveProperty() {
for (int i = 0; i < 100; i++) {
int x = randomInt();
int result1 = sum(sum(x , 1), 1);
int result2 = sum(x, 2);
assertEquals(result1, result2);
}
}

15. // WTF?!
private int sum(int a, int b) {
return 0;
}

16. @Test
// `x + 1` = `x`
public void testSumWithIdentityProperty() {
for (int i = 0; i < 100; i++) {
int x = randomInt();
assertEquals(x, sum(x, 0));
}
}

17. Testing Specification with properties
• Commutativity property
• Associativity property
• Identity property

18. Testing Specification with properties
By using specifications, we have understood the
requirements in a deeper way

19. QuickCheck

21. QuickCheck in a nutshell

22. John Hughes - Testing the Hard Stuff and Staying Sane

24. Benefits
• Less time spent writing test code
- One property replaces many tests
• Better testing
- Lots of combinations you’d never test by hand
• Less time spent on diagnosis
- Failures minimized automatically

25. 3,000 pages of specifications
20,000 lines of QuickCheck
1,000,000 LoC from 6 suppliers
200 bugs
100 bugs in the standard

26. junit-quickcheck

27. @RunWith(JUnitQuickcheck.class)
public class AdditionPropertyTest {
@Property
public void commutativityProperty(int x, int y) {
assertThat(sum(x, y), equalTo(sum(y, x)));
}
@Property
public void additiveProperty(int x) {
int result1 = sum(sum(x , 1), 1);
int result2 = sum(x, 2);
assertEquals(result1, result2); }
@Property
public void identityProperty(int x) {
assertEquals(x, sum(x, 0));
}
}

28. java.lang.AssertionError: Property identityProperty
falsified via shrinking: expected:<1> but was:<0>
Shrunken args: [1]
Original failure message: [expected:<98> but was:<0>]
Original args: [98]
Seeds: [7329045779044418918]

29. @RunWith(JUnitQuickcheck.class)
public class MoneyChangerTest {
@Property
public void sumOfCoinsEqualsAmount(
@InRange(min = "0", max = "500") int amountToChange,
Set<@InRange(min = "1", max = "100") Integer> denominations) {
assumeThat(denominations.size(), greaterThan(0));
denominations.add(1);
MoneyChanger moneyChanger = new MoneyChanger();
List<Integer> coins = moneyChanger.change(amountToChange, denominations);
int sum = coins.stream().mapToInt(coin -> coin).sum();
assertThat(sum, equalTo(amountToChange));
}
}

30. JSVerify
npm install jsverify

31. describe("sort", function () {
jsc.property(
"idempotent", "array nat", function (arr) {
return _.isEqual(sort(sort(arr)), sort(arr));
});
});

32. How to handle state?

33. How to handle state?
• Generate actions changing the state
• Apply each action to the state, if possible
• After each application, verify the model

34. Where can we benefit
from
property-based tests?

35. Spark Proxy on DataFrame
API
• Generate random DataReadRequest
• Execute request with RDD-based code
• Execute same request with DataFrame based code
• Compare the results

36. SDK
• Send START_VIS
• Generate random request, change metric, group-
by, filter, sort, etc
• Validate responses based on the test model
• Send STOP_VIS

37. • QuickCheck
• Testing Telecoms Software with Quviq QuickCheck
• Testing the hard stuff and staying sane
• The Mysteries of Dropbox
• Jepsen IV: Hope Springs Eternal
• Java Examples from this talk