This talk was given in underscore meetup in Tel Aviv in 24/06 ------- There is no doubt that Unit tests are an essential part of the development process today. However, writing unit tests is a tedious and repetitive job that is often unrewarding since bugs slip away. Property-based testing addresses these issues by providing a way to define laws and properties on your data types and generate tests to verify them. Dor Sever is a software developer at bigpanda.io

1. Killing the Unit test
dor sever @ bigpanda
Killing the unit test - @dor_sever 1

2. The missing
Unit Test problem
2

3. Solving the missing unit test problem
Real life example: from Unit Test to Property Based Tests.
3

4. 4

5. 5

6. 6

7. 7

8. 8

9. Implement counters for folders
9

10. 10

11. sealed trait Folder
case object Active extends Folder
case object Shared extends Folder
case object Resolved extends Folder
11

12. 12

13. case class Alert(data: String, folders: Set[Folder])
13

14. The requirements
» Add Alerts to current folder counters.
» Remove Alerts from folder counters.
» Direct access to a folder's count.
14

15. type Counters = Map[Folder, Int]
trait CountersApi {
def addToCounters(current: Counters,
alert: Alert): Counters
def removeFromCounters(current: Counters,
alert: Alert): Counters
}
15

16. type Counters = Map[Folder, Int]
trait CountersApi {
def addToCounters(current: Counters,
alert: Alert): Counters
def removeFromCounters(current: Counters,
alert: Alert): Counters
}
16

17. 17

18. Imports
import org.scalatest._
trait TestSpec extends WordSpecLike with Matchers
import CountersApiV1Instance.addToCounters
18

19. class FolderCountersSpec extends TestSpec {
"addToCounters" should {
"increment the Active count by 1 when adding an Active alert" in {}
"increment the Shared count by 1 when adding a Shared alert" in {}
"increment the Resolved count by 1 when adding a Resolved alert" in {}
"increment all folders counts by 1 when adding alert that sits on all folders" in {}
}
"removeFromCounters" should {
"decrement the Active count by 1 when removing an Active alert" in {}
"decrement the Shared count by 1 when removing a Shared alert" in {}
"decrement all counts by 1 when removing an alert that sits on all folders" in {}
}
}
19

20. class FolderCountersSpec extends TestSpec {
val emptyCounters = Map[Folder,Int]()
"addToCounters" should {
"increment the Active count by 1 when adding an Active alert" in {
val activeAlert =
Alert("High CPU in elastic 11", Set(Active))
addToCounters(emptyCounters, activeAlert) shouldEqual Map(Active -> 1)
}
}
}
20

21. 21

22. 22

23. 23

24. The bug
24

25. 25

26. The original test
26

27. 27

28. The missing unit test problem
28

29. 29

30. Property Based Tests
» Property based tests work by defining a property
30

31. Property Based Tests
» Property based tests work by defining a property
which is a high-level specification of behavior
31

32. Property Based Tests
» Property based tests work by defining a property,
which is a high-level specification of behavior
that should hold for all values of the specific type.
32

33. Property Based Tests
» Property based tests work by defining a property,
which is a high-level specification of behavior
that should hold for all values of the specific type.
» ∀ value : T => property(value) == true
33

34. The white lie
» We can't really test all values.
» We generate 100 random inputs, and verify the property on
them.
34

35. my first property
» For all lists, if we reverse them twice, we should get the
original list.
35

36. List("my","first","property").reverse.reverse
36

37. scala> List("my","first","property").reverse.reverse
res1: List[String] = List(my, first, property)
37

38. List(1,1,2,3,5,8,13,21).reverse.reverse
38

39. scala> List(1,1,2,3,5,8,13,21).reverse.reverse
res3: List[Int] = List(1, 1, 2, 3, 5, 8, 13, 21)
39

40. Property based tests - The mechanics
» Scala check
» Generators
» Properties
40

41. trait Generator[A] {
def sample: A
}
41

42. Int generator
object intGen extends Generator[Int] {
override def sample: Int = scala.util.Random.nextInt
}
scala> intGen.sample
res4: Int = -1668959352
scala> intGen.sample
res5: Int = 309764036
scala> intGen.sample
res6: Int = -1278707761
42

43. Let's Generate Counters
scala> type Counters = Map[Folder, Int]
defined type alias Counters
43

44. import org.scalacheck.Gen
val folderToCounterGen =
for {
folder <- Gen.oneOf(Active, Resolved, Shared)
counter <- Gen.posNum[Int]
} yield folder -> counter
implicit val countersGen: Gen[Map[Folder, Int]] =
Gen.listOf(folderToCounterGen).map(_.toMap)
44

45. scala> countersGen.sample
res7: Option[Map[Folder,Int]] = Some(Map(Active -> 42, Shared -> 84, Resolved -> 33))
scala> countersGen.sample
res8: Option[Map[Folder,Int]] = Some(Map(Resolved -> 38, Active -> 11, Shared -> 36))
45

46. Properties
46

47. Property = Assertion ?
47

48. What is a property ?
» Properties are assertions with special requirements
» They must hold for all values!
48

49. Imports
import org.scalatest.prop.GeneratorDrivenPropertyChecks
trait FolderPropertiesSpec extends WordSpecLike
with Matchers
with GeneratorDrivenPropertyChecks
import CountersApiV1Instance.addToCounters
49

50. The first property
class FolderCountersPropertySpec extends FolderPropertiesSpec {
"increment the alert folders" in forAll {
(counters: Map[Folder, Int], alert: Alert) =>
addToCounters(counters, alert) shouldEqual ???
}
}
50

51. class FolderCountersPropertySpec extends FolderPropertiesSpec {
"increment the alert folders" in forAll {
(counters: Map[Folder, Int], alert: Alert) =>
addToCounters(counters, alert) shouldEqual ???
}
}
51

52. class FolderCountersPropertySpec extends FolderPropertiesSpec {
"increment the alert folders" in forAll {
(counters: Map[Folder, Int], alert: Alert) =>
addToCounters(counters, alert) shouldEqual ???
}
}
52

53. How do we define properties?
53

54. 54

55. Empty alert Property
55

56. 56

57. class FolderCountersPropertySpec extends FolderPropertiesSpec {
"adding the empty alert should not change the counters" in forAll {
counters: Map[Folder, Int] =>
addToCounters(counters, Alert("",Set())) shouldEqual counters
}
}
57

58. 58

59. Victory ?
59

60. Remove From Counters
» Repeat the process
60

61. The missing properties problem
How to find properties?
trait CountersApi {
def addToCounters(current: Counters, alert: Alert): Counters
def removeFromCounters(current: Counters, alert: Alert): Counters
}
61

62. Killing the unit test - @dor_sever 62

63. Remove and Add are related!
» Removing and then adding an alert should keep the original
counters
class FolderCountersPropertySpecWorking extends FolderPropertiesSpec {
"Removing and then adding an alert should keep the original counters" in forAll {
(counters: Map[Folder, Int], alert: Alert) =>
addToCounters(removeFromCounters(counters, alert), alert) shouldEqual counters
}
}
63

64. class FolderCountersPropertySpecWorking extends FolderPropertiesSpec {
"Removing and then adding an alert should keep the original counters" in forAll {
(counters: Map[Folder, Int], alert: Alert) =>
addToCounters(removeFromCounters(counters, alert), alert) shouldEqual counters
}
}
64

65. class FolderCountersPropertySpecWorking extends FolderPropertiesSpec {
"Removing and then adding an alert should keep the original counters" in forAll {
(counters: Map[Folder, Int], alert: Alert) =>
addToCounters(removeFromCounters(counters, alert), alert) shouldEqual counters
}
}
65

66. 66

67. 67

68. Making it fail consistently
class FolderCountersPropertySpecWorking extends FolderPropertiesSpec {
"Removing and adding an alert should keep the original counters" in forAll(minSuccessful(1000)) {
(counters: Map[Folder, Int], alert: Alert) =>
addToCounters(removeFromCounters(counters, alert), alert) shouldEqual counters
}
}
68

69. Let's define
val sharedAlert = Alert("",Set(Shared))
val emptyCounters = Map[Folder,Int]()
69

70. The BUG in our property
scala> val removedCounters = removeFromCounters(emptyCounters,sharedAlert)
removedCounters: Counters = Map()
70

71. The BUG in our property
scala> val removedCounters = removeFromCounters(emptyCounters,sharedAlert)
removedCounters: Counters = Map()
scala> val result = addToCounters(removedCounters,sharedAlert)
result: Counters = Map(Shared -> 1)
71

72. The BUG in our property
scala> val removedCounters = removeFromCounters(emptyCounters,sharedAlert)
removedCounters: Counters = Map()
scala> val result = addToCounters(removedCounters,sharedAlert)
result: Counters = Map(Shared -> 1)
scala> result == emptyCounters
res0: Boolean = false
72

73. The BUG in our property
scala> val removedCounters = removeFromCounters(emptyCounters,sharedAlert)
removedCounters: Counters = Map()
scala> val result = addToCounters(removedCounters,sharedAlert)
result: Counters = Map(Shared -> 1)
scala> Map(Shared->1) == Map()
res1: Boolean = false
73

74. The Good property?
class FolderCountersPropertySpecWorking extends FolderPropertiesSpec {
"Adding and then removing an alert should keep the original counters" in forAll{
(counters: Map[Folder, Int], alert: Alert) =>
removeFromCounters(addToCounters(counters,alert),alert) shouldEqual counters
}
}
74

75. A different bug
scala> val addedAlerts = addToCounters(emptyCounters,sharedAlert)
addedAlerts: Counters = Map(Shared -> 1)
75

76. A different bug
scala> val addedAlerts = addToCounters(emptyCounters,sharedAlert)
addedAlerts: Counters = Map(Shared -> 1)
scala> val result = removeFromCounters(addedAlerts,sharedAlert)
result: Counters = Map(Shared -> 0)
76

77. A different bug
scala> val addedAlerts = addToCounters(emptyCounters,sharedAlert)
addedAlerts: Counters = Map(Shared -> 1)
scala> val result = removeFromCounters(addedAlerts,sharedAlert)
result: Counters = Map(Shared -> 0)
scala> result == emptyCounters
res2: Boolean = false
77

78. A different bug
scala> val addedAlerts = addToCounters(emptyCounters,sharedAlert)
addedAlerts: Counters = Map(Shared -> 1)
scala> val result = removeFromCounters(addedAlerts,sharedAlert)
result: Counters = Map(Shared -> 0)
scala> Map(Shared->0) == Map()
res3: Boolean = false
78

79. Let's fix it?
79

80. Victory?
80

81. Group to the rescue
» Disclaimer:
» this is going to require a leap of faith
» this will not work at all times
» This is going to be fast and fun
81

82. The Plan
82

83. 83

84. trait myGroup[A] {
def add(x: A, y: A): A
def remove(x: A, y: A): A
def inverse(x: A): A
def identity: A
}
84

85. trait CountersApi {
def addToCounters(current: Counters, alert: Alert): Counters
def removeFromCounters(current: Counters, alert: Alert): Counters
}
85

86. Getting rid of the alert data type
86

87. Getting rid of the alert data type
def toCounters(alert:Alert):Counters = ???
87

88. trait CountersApi {
def addToCounters(current: Counters, added: Counters): Counters
def removeFromCounters(current: Counters, removed: Counters): Counters
}
88

89. Identity Element
89

90. trait CountersApi {
def addToCounters(current: Counters, added: Counters): Counters
def removeFromCounters(current: Counters, removed: Counters): Counters
def identity : Counters
}
90

91. Making the API polymorphic
trait CountersApi[A] {
def addToCounters(current: A, added: A): A
def removeFromCounters(current: A, removed: A): A
def identity : A
}
91

92. The inverse function
92

93. trait CountersApi[A] {
def addToCounters(current: A, added: A): A
def removeFromCounters(current: A, removed: A): A
def inverse(x: A) : A
def identity : A
}
93

94. trait CountersApi[A] {
def combine(current: A, added: A): A
def remove(current: A, removed: A): A =
combine(current,inverse(removed))
def inverse(x:A): A
def identity : A
}
94

95. Implementation
95

96. The tests!
import cats.laws.discipline._
import cats.kernel.laws.discipline.GroupTests
import org.scalatest.FunSuiteLike
import org.typelevel.discipline.scalatest.Discipline
class GroupsTests extends Discipline with FunSuiteLike {
checkAll("CountersGroupTests", GroupTests[Counters].group)
}
96

97. 97

98. 98

99. 99

100. Summary
» The missing unit test problem
» Property based tests
» Generators
» Properties, the missing properties problem
» Math to the rescue!
Killing the unit test - @dor_sever 100

101. Takeaways
» Try this at home !
» Huge payoff, and it's super fun
Killing the unit test - @dor_sever 101

102. Thank you
» https://github.com/dorsev/Killing-the-unit-test-talk
» https://medium.com/free-code-camp/an-introduction-to-
law-testing-in-scala-4243d72272f9
Killing the unit test - @dor_sever 102