1. Testing Business Logic
Using DSLs in Clojure
Mayank Jain
Test Engineer

2. A TALK, IN 6 PARTS
1. What is the problem?
2. Real World Example.
3. Demo of the actual code.
4. Other features tested using same ideas.
5. Advantages/Disadvantages of writing DSLs for testing.
6. QA.

3. WHAT IS THE PROBLEM?
Testing real world stateful business logic is hard

4. Microwave Oven State Machine
Business logic is Stateful

5. Microwave Oven State Machine
Large number transition states

6. Difficult to enumerate all possible cases
Microwave Oven State Machine

7. Tests become unreadable
Microwave Oven State Machine

8. HELPSHIFT
• Embeddable support desk for native apps
• Main Features:
• Frequently Asked Questions which
customers can search
• File issues/tickets from within the app

9. Helpshift
Supercell
Clash of Clans
Gaana
General Billing
Domain
App
Section
FAQ 1 FAQ 2FAQs
Boom Beach Gaana App
Translations English Content Hindi Content
….
….
….
….
….
Top Down View of FAQs

10. Example of Gaana FAQ Page
Customers can search FAQs
Available FAQs FAQ Sections
Domain
App

11. FAQ Title
FAQ Body
FAQ is Visible?

12. PROBLEM
You cannot share FAQs across apps.
Big Customers have multiple apps which have same
FAQ translations content example “Privacy Policy”

13. FEATURE: LINKED FAQS

14. {:published? false
:id “faq-id-1”
:app_id “app-1”
:section_id “section-1”
:translations
{:en {:published? false
:stags []
:body “Privacy Body”
:title “Privacy Title”}}
:linked_faq_ids [“faq-id-2”]
:publish_id “1”}
App-1
{:published? false
:id “faq-id-2”
:app_id “app-2”
:section_id “section-2”
:translations
{:en {:published? false
:stags []
:body “Privacy Body"
:title “Privacy Title"}}
:linked_faq_ids [“faq-id-1”]
:publish_id “2”}
App-2
faq 1
(Sync faq-1 to App-2)General
faq 2
General

15. {:published? false
:id “faq-id-1”
:app_id “app-1”
:section_id “section-1”
:translations
{:en {:published? false
:stags []
:body “Update Body”
:title “Privacy Title”}}
:linked_faq_ids [“faq-id-2”]
:publish_id “1”}
{:published? false
:id “faq-id-2”
:app_id “app-2”
:section_id “section-2”
:translations
{:en {:published? false
:stags []
:body “Privacy Body"
:title “Privacy Title"}}
:linked_faq_ids [“faq-id-1”]
:publish_id “2”}
faq 1
Sync
faq 2
Update FAQ 1’s Body

16. {:published? false
:id “faq-id-1”
:app_id “app-1”
:section_id “section-1”
:translations
{:en {:published? false
:stags []
:body “Update Body”
:title “Privacy Title”}}
:linked_faq_ids [“faq-id-2”]
:publish_id “1”}
{:published? false
:id “faq-id-2”
:app_id “app-2”
:section_id “section-2”
:translations
{:en {:published? false
:stags []
:body “Update Body"
:title “Privacy Title"}}
:linked_faq_ids [“faq-id-1”]
:publish_id “2”}
faq 1
Sync
faq 2
Update FAQ 1’s Body

17. EXAMPLE TEST CASE

18. 1. Add 1st App with only English languages.
2. Add 2nd App with only English languages
3. Add 1st FAQ under 1st App
4. Link 1st FAQ to 2nd App to create 2nd FAQ
5. Check -> translations of 2nd FAQ == 1st FAQ
6. Update English title of FAQ-1
7. Check -> translations of 2nd FAQ == 1st FAQ
8. Delete FAQ-1
9. Check if 1st FAQ is deleted in DB
10.Assert 2nd FAQ should remain as it is in database.
Simulation Verification

19. PARTS OF EACH ACTION
“Add APP-1 with only English language”
1. Type of Action Add an App
2. Names App-1
3. Arguments English Language
4. Expected Result As per Spec/Modal of the system
5. Actual Result Database

20. EXPRESS ACTIONS AS
CLOJURE DATA
“Add app APP-1 with English language”
:add-app :app-1 {:langs-config [:en]}

21. ONE UNIT OF ACTION
[:add-app :app-1 {:langs-config [:en]}]

22. SERIES OF ACTIONS
[[:add-app :app-1 {:langs-config [:en]]
[:add-app :app-2 {:langs-config nil}]
[:add-faq :faq-1 {:app-var :app-1 ...}]
[:link-faq :faq-2 {:faq-var :faq-1 :app-var :app-2 ...}]
[:update-faq nil {:app-var :app-1 :faq var :faq-1 ...}]
[:update-faq nil {:app-var :app-2 :faq-var :faq-2 ...}]
[:delete-faq nil {:faq-var :faq-1 ...}]]

23. REDUCE ON ACTIONS
Reduce
{}
[:add-app :app-1 {:langs-config nil}])
{:env {:apps {:app-1 {…}}}
:result [{result-1…}]}
…..}
Reduce
[:add-app :app-2 {:langs-config nil}])
{:env {:apps {:app-1 {…}
:app-2 {…}}}
:result [{result-1…}
{result-2…}]}
…..}
{:env {:apps {:app-1 {…}}}
:result [{result-1…}]}
…..}
And so on…

24. RESULT HASH-MAP COMPRISES OF
{:env {:apps {:app-1 {:app-id “970"}
:app-2 {:app-id “142”}}}
:faqs {:faq-1 {:faq-id “112”}}}}
:result [{:action-type :add-faq
:expected {….}
:actual {….}}
{…more}]
“112” {…state…}
…more}
Environment (:env) - Contains bindings of vars

25. RESULT HASH-MAP COMPRISES OF
{:env {:apps {:app-1 {:app-id “970"}
:app-2 {:app-id “142”}}}
:faqs {:faq-1 {:faq-id “112”}}}}
:result [{:action-type :add-faq
:expected {….}
:actual {….}}
{…more}]
“112” {…state…}
…more}
Result (:result) - Contains Actual And Expected Result

26. RESULT HASH-MAP COMPRISES OF
{:env {:apps {:app-1 {:app-id “970"}
:app-2 {:app-id “142”}}}
:faqs {:faq-1 {:faq-id “112”}}}}
:result [{:action-type :add-faq
:expected {….}
:actual {….}}
{…more}]
“112” {…state…}
…more}
Current Generated Expected State

27. Dispatch On Action Type
Update The Variables in
arguments to its bindings
Call args with relevant
function
Bind the result to the given
var in Global Data
Store Expected current
state
Store Actual current
database state
What
happens
inside
the
reducer?

28. Dispatch On Action Type
Update The Variables in
arguments to its bindings
Call args with relevant
function
Bind the result to the given
var in Global Data
Store Expected current
state
Store Actual current
database state
[:add-app :app-1 {:langs-config nil}])
[:add-app :app-1 {:langs-config nil}])
(add-app {:langs-config nil})
{:env {:apps {:app-1 {:app-id "970"}}}}

29. Dispatch On Action Type
Update The Variables in
arguments to its bindings
Call args with relevant
function
Bind the result to the given
var in Global Data
Store Expected current
state
Store Actual current
database state
[:add-app :app-2 {:langs-config nil}])
[:add-app :app-2 {:langs-config nil}])
(add-app {:langs-config nil})
{:env {:apps {:app-1 {:app-id “970"}
:app-2 {:app-id “142"}}}}

30. Dispatch On Action Type
Update The Variables in
arguments to its bindings
[:add-faq :faq-1 {:app-var :app-1 ….}])
{:env {:apps {:app-1 {:app-id “970"}
:app-2 {:app-id “142”}}}

31. Dispatch On Action Type
Update The Variables in
arguments to its bindings
[:add-faq :faq-1 {:app-var :app-1 ….}])
[:add-faq :faq-1 {:app-id “970” …}])
{:env {:apps {:app-1 {:app-id “970"}
:app-2 {:app-id “142”}}}

32. Dispatch On Action Type
Update The Variables in
arguments to its bindings
Call args with relevant
function
Bind the result to the given
var in Global Data
[:add-faq :faq-1 {:app-var :app-1 ….}])
[:add-faq :faq-1 {:app-id “970” …}])
(add-faq {:app-id “970” ….})

33. Dispatch On Action Type
Update The Variables in
arguments to its bindings
Call args with relevant
function
Bind the result to the given
var in Global Data
[:add-faq :faq-1 {:app-var :app-1 ….}])
[:add-faq :faq-1 {:app-id “970” …}])
(add-faq {:app-id “970” ….})
{:env {:apps {:app-1 {:app-id “970"}
:app-2 {:app-id “142”}}
:faqs {:faq-1 {:faq-id “112”}}}}

34. Dispatch On Action Type
Update The Variables in
arguments to its bindings
Call args with relevant
function
Bind the result to the given
var in Global Data
Store Expected current
state
[:add-faq :faq-1 {:app-var :app-1 ….}])
[:add-faq :faq-1 {:app-id “970” …}])
(add-faq {:app-id “970” ….})
{:env {:apps {:app-1 {:app-id “970"}
:app-2 {:app-id “142”}}
:faqs {:faq-1 {:faq-id “112”}}}}
{:env {:apps {:app-1 {:app-id “970"}
:app-2 {:app-id “142”}}}
:faqs {:faq-1 {:faq-id “112”}}}}
:result [{:action-type :add-faq
:expected {….}}

35. Dispatch On Action Type
Update The Variables in
arguments to its bindings
Call args with relevant
function
Bind the result to the given
var in Global Data
Store Expected current
state
Store Actual current
database state
[:add-faq :faq-1 {:app-var :app-1 ….}])
[:add-faq :faq-1 {:app-id “970” …}])
(add-faq {:app-id “970” ….})
{:env {:apps {:app-1 {:app-id “970"}
:app-2 {:app-id “142”}}
:faqs {:faq-1 {:faq-id “112”}}}}
{:env {:apps {:app-1 {:app-id “970"}
:app-2 {:app-id “142”}}}
:faqs {:faq-1 {:faq-id “112”}}}}
:result [{:action-type :add-faq
:expected {….}
:actual {….}}

36. VERIFICATION:
COMPARE EXPECTED VS ACTUAL FOR EACH STEP

37. COMPARE RESULT FOR
FIRST ACTION
{"112"
{:translations
{:en
{:published? false,
:stags [],
:body "Temp body",
:title “Title 1"}},
:linked-faq-ids #{}}}
Actual Data in DB
Expected Data
from my Generated result
{"112"
{:translations
{:en
{:published? false,
:stags [],
:body "Temp body",
:title “Title 1"}},
:linked-faq-ids #{}}}
==
PASS

38. :title "Title 1"
Actual Data in DB
Expected Data
from my Generated result
:title "Title 2"
==
FAIL
COMPARE RESULT FOR
SECOND ACTION

39. DEMO TIME

40. APPROACHES TO VERIFY
Actions : Hardcoded
Expected Output : Hardcoded
Actions : Simulated
Expected Output : Hardcoded
Actions : Simulated
Expected Output : Generated
Actions : Generated
Expected Output : Generated

41. Hardcoded Actions,
Hardcoded Expected Output
ADVANTAGES DISADVANTAGES
Requires no extra knowledge to
understand
Very cumbersome to
enumerate
Anyone can add/edit tests Modification is hard
False negatives are not possible

42. Simulate list of Actions,
Hardcoded Final Expected Output
ADVANTAGES DISADVANTAGES
Easy to enumerate
Requires knowledge of the
DSL
Very readable
Maintenance overhead of
DSL
Shareable with dev to simulate
bugs
Does not check intermediate
state
False negatives are not possible
Expected output may have
data which is available only
at runtime like faq-ids

43. Simulate list of actions,
Generate Expected Output
ADVANTAGES DISADVANTAGES
Easy to enumerate
Requires knowledge of the
DSL
Very readable
Maintenance overhead of
DSL
Shareable with dev to simulate
bugs
Maintenance overhead of
Expected Modal
Checks intermediate state False negatives are possible
Runtime data is available like
faq-ids

44. Generate Actions,
Generate Expected Output
ADVANTAGES DISADVANTAGES
Possible to generate large
number of tests
Requires knowledge of the
DSL
Possible to Shrink failed test
case using test.check library
Maintenance overhead of
DSL
Shareable with dev to simulate
bugs
Maintenance overhead of
Expected Modal
Checks intermediate state False negatives are possible
Maintenance overhead of
generative code for list of
actions.

45. Generate Actions,
Generate Expected Output
Feature - Issue Audit Trail

46. Issue Audit Trail
• Maintains a log of
• Who Took an action
• What Action they took

47. Who - Types of Users
App User Support User
Agents Admins
Helpshift

48. What - Types of Actions
App User
• Create Issue
• Reply Issue
• etc…
Admin User
• Create Issue
• Reply Issue
• Resolve Issue
• Edit Tags
• etc…
Agent User
• Create Issue
• Reply Issue
• Resolve Issue
• Edit Tags
• etc…

49. App User
Admin
“Mayank”
Agent
“Agent-2”
Example Issue

50. Example Issue
Issue
Audit
Logs

51. DEMO - ISSUE AUDIT TRAIL
Generate Actions,
Generate Expected Output

52. ADVANTAGES OF WRITING DSL
Discovering very hard to find bugs, for example we found:
Duplicate issue messages being rendered only if the
number of messages were "just right".
Finding ordering bugs.
Increase in developer/tester productivity

53. DISADVANTAGES OF WRITING DSL
Cost of maintaining DSLs is high.
If your DSL is just data being evaluated at run time,
changes in feature code will not throw any compile time
errors like function parameters being changed in DSL
code.
You have to educate your team members to learn your
specific DSL for that feature to be able to understand the
tests.

54. CONCLUSION
DSL can be used as a workflow for writing tests.
Separating simulation vs verification of tests.
A step towards ability to generate tests instead of writing
them.

55. Further Resources
• Clojure Made Simple - Rich Hickey
• Growing a DSL with Clojure
• Jeanine Adkisson - Design and Prototype a Language
In Clojure
• John Hughes - Testing the Hard Stuff and Staying Sane
• Reid Draper - Powerful Testing with test.check
• Clojure Tutorials on DSLs with Tim Baldridge (Paid)

56. Any Questions?
@firesofmay
facebook.com/firesofmay/
firesofmay@gmail.com