This document explains Property Based Testing – a method beyond unit testing to help detect defects in codes during the development phase.

1. This document is confidential and contains proprietary information, including trade secrets of CitiusTech. Neither the document nor any of the information
contained in it may be reproduced or disclosed to any unauthorized person under any circumstances without the express written permission of CitiusTech.
Building Efficient Software with Property
Based Testing
19 February, 2019 | Author: Gayatri Himthani | Technical Lead
CitiusTech Thought
Leadership

2. 2
Agenda
▪ Introduction to Property Based Testing
▪ Failures due to lack of Testing
▪ Methods to Define Properties
▪ Property Based Testing of a Web Service
▪ Property Based Testing of a Stateful System
▪ Property Based Testing in Healthcare
▪ Available Property Based Testing Frameworks
▪ Conclusion

3. 3
Introduction: Property Based Testing
Beyond Unit Testing
Unit tests are written with a predefined input in mind. For any
scenario, it’s unfeasible to test functionality with all input
variations, to make unit tests and code coverage inadequate
indicators of a thoroughly tested code.
Property Based
Testing
In Property Based Testing, a System Under Test (SUT) runs
frequently with all possible valid inputs that can break the
code.
Finding the Edge Case
/ Shrinking the Input
After the test is broken, a PBT framework shrinks the input to
find the smallest input that falsifies the test. This helps to find
the edge case inputs – empty strings, Unicode characters,
special characters, empty list, negative numbers, lengthy lists,
etc.
Focus on the Property
Assertions / specifications written in SUT test the property of
the functionality being coded. E.g. A DateTimeValue should
have same Date when expressed in milliseconds and
nanoseconds.

4. 4
1. Sorting problem in JDK with Tim Sort Algorithm
▪ This problem occurred in JDK 7u76, 8 and 9. When a custom comparison logic is written, the Tim
Sort algorithm presents an exception – “ArrayIndexOutOfBoundsException” for worst-case long
arrays
▪ This bug has been resolved in later versions of JDK 9 (Details can be found at -
https://bugs.openjdk.java.net/browse/JDK-8072909)
▪ This is an edge case scenario as the algorithm breaks for particular input length collection. It is
highly possible that unit test was missed for this case
2. Heartbleed Open SSL bug of 2014
▪ Heartbleed bug arose because a critical check was missing – the receiver of the Heartbeat
request never checked the length it claimed
▪ From Martin Fowler’s blog:
o “Heartbleed was a similarly heartbreaking case of untested security-critical code which
appeared as part of the ubiquitous OpenSSL library”
o “The change introducing the bug was code reviewed; it is apparent that the reviewer did not
insist that the change include unit tests”
▪ When the logic becomes complex, it becomes exponentially difficult to write tests with all input
variations
Failures due to lack of Testing

5. 5
Methods to Define Properties
1. Inversing
Applying an inverse function to a function should result in original input.
e.g. For a given input x in a defined range of inputs, the property
“deserialize (serialize(x)) == x”
should always hold true
OR
For a given input x in a defined range of inputs, the property
“decode (encode(x)) == x”
should always hold true
2. Idempotency
Multiple application of the same function to an input doesn’t change the result,
e.g. ‘Sorting’ or ‘Ordering’ a collection multiple times should present the same
result
3. Invariance
Certain properties should never change after the code has run against the input
multiple times. E.g., the hash value of keys of a collection should never change
after any operation; sorting should never change the size of a collection

6. 6
Methods to Define Properties
4. Commutativity
The change in order of applying the function should not change the result.
E.g., Sorting a List, negating, then reversing it == negating the list, then
sorting it
5. Alternative Implementation
There is also an alternate implementation available for the functionality
developed. This approach can be used to test the correctness of the code.
E.g., Sorting a collection with a new algorithm == sorting a collection with
Bubble sort
6. Business Rule as a Property
In the real world, we are guided by business rules to write software, and
are reliable to define properties for testing. Business rules not only deal
with primitive types, but with complex objects too

7. 7
▪ Web services and APIs are an inherent part of software development, and It’s crucial
to have a break-proof implementation, as it may be consumed by variety of clients
with various purposes. To achieve this, ensure:
o The API doesn’t break for any valid input and generates a valid output under the
defined schema / rules
o Property of the API functionality works as per the definition for all valid inputs
▪ As illustrated below, a typical Property Based Testing framework can be used to test
the API. PBT will generate inputs based on the schema; test the specification written
against the API implementation; and validate the output based on schema
Property Based Testing of a Web Service
PBT
(Test the
property of
an API)
Input
………….
Output
………….
Input &
Output
Schema
Result
(Pass / Fail)
Client
Upon failure, tests are run again with shrunken input to identify the edge case

8. 8
▪ State cannot be avoided in OOP world
▪ A correct sequence of states defines a property of a
stateful system
▪ Property Based Testing can help test a stateful system by
generating random sequence of actions and verifying
their output against a defined property
▪ From the adjoining workflow, consider a scenario where
patient encounter has defined states
o An event / action will trigger the change in state
o A typical PBT framework will generate a random
sequence of actions and test the next state along
with any defined conditions
o Upon failure, the framework will shrink the
sequence of action commands to give the smallest
set of actions that falsifies the property
Property Based Testing of a Stateful System
Examination
Diagnosis
Treatment
Evaluation
Completed
States of Patient Encounter

9. 9
▪ Healthcare technology systems are getting more complex to support various
healthcare workflows. It’s critical to ensure that the quality of software isn’t
compromised for patient safety
▪ For feature development, compliance with regulatory (HIPAA) standards and
interoperability rules of DICOM, HL7, CDA etc. are considered along with complex
functional / business rules
▪ Testing should ensure that a feature is penetrated by all possible variations of inputs
and it doesn’t break the software at any particular input
▪ Properties / specifications can be defined for features of various healthcare systems
(claim systems, healthcare CRM, EHR portals etc.) with the help of business and
compliance rules and extensive domain knowledge
▪ For complex software like claims systems, there’s a high frequency of modification,
which makes the feature vulnerable to bugs if not tested against various possible
inputs
▪ Property Based Testing helps to generate a wide array of possible inputs, and
identifies the edge case for a failed scenario
Property Based Testing in Healthcare
Correct a feature during development to improve the quality of codes

10. 10
Available Property Based Testing Frameworks
Programming Language PBT Framework
Java junit-Quickcheck, QuickTheories, jqwik
C#, F#, VB FsCheck
Python Hypothesis, pytest quickcheck
Haskell HaskellQuickcheck
Erlang, Elixir Quviq Quickcheck
Javascript QC.js, jsverify
Scala ScalaCheck
C Theft
C++ Quickcheck++
Clojure ClojureCheck
QuickChek for Haskell is the first Property Based Testing tool
developed for testing Haskell programs

11. 11
▪ Property Based Testing is not difficult but thoughtful
▪ Property Based Testing should not be implemented to achieve code
coverage, but to test critical functionality of an implemented feature
▪ The code to implement functionality shouldn’t be re-written to test a
property, as it violates the abstraction and encapsulation principles of
programming
▪ Patterns mentioned in previous sections should be employed to
define properties
Conclusion

12. 12
▪ Quviq Quickcheck: http://www.quviq.com/products/erlang-quickcheck/
▪ FsCheck Docs: https://fscheck.github.io/FsCheck/index.html
▪ Jqwik: https://jqwik.net/
▪ Junit-Quickcheck: https://github.com/pholser/junit-quickcheck
▪ HeartBleed Vulnerability: https://martinfowler.com/articles/testing-culture.html
▪ Choosing properties for PBT:
o https://fsharpforfunandprofit.com/posts/property-based-testing-2/
o https://www.ibm.com/developerworks/community/files/form/anonymous/api/library/382
18957-7195-4fe9-812a-10b7869e4a87/document/ab12b05b-9f07-4146-8514-
18e22bd5408c/media
References

13. 13
Thank You
Authors:
Gayatri Himthani
thoughtleaders@citiustech.com
About CitiusTech
3,500+
healthcare IT professionals worldwide
100%
healthcare industry focus
30%+
CAGR over last 5 years
110+
healthcare customers
• Healthcare technology companies
• Hospitals, IDNs & medical groups
• Payers and health plans
• ACOs, MCOs, HIEs, HIXs, NHINs
• Pharma & Life Sciences companies