Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Testing Business Logic
Using DSLs in Clojure
Mayank Jain
Test Engineer
A TALK, IN 6 PARTS
1. What is the problem?
2. Real World Example.
3. Demo of the actual code.
4. Other features tested usi...
WHAT IS THE PROBLEM?
Testing real world stateful business logic is hard
Microwave Oven State Machine
Business logic is Stateful
Microwave Oven State Machine
Large number transition states
Difficult to enumerate all possible cases
Microwave Oven State Machine
Tests become unreadable
Microwave Oven State Machine
HELPSHIFT
• Embeddable support desk for native apps
• Main Features:
• Frequently Asked Questions which
customers can sear...
Helpshift
Supercell
Clash of Clans
Gaana
General Billing
Domain
App
Section
FAQ 1 FAQ 2FAQs
Boom Beach Gaana App
Translati...
Example of Gaana FAQ Page
Customers can search FAQs
Available FAQs FAQ Sections
Domain
App
FAQ Title
FAQ Body
FAQ is Visible?
PROBLEM
You cannot share FAQs across apps.
Big Customers have multiple apps which have same
FAQ translations content examp...
FEATURE: LINKED FAQS
{:published? false
:id “faq-id-1”
:app_id “app-1”
:section_id “section-1”
:translations
{:en {:published? false
:stags []
...
{:published? false
:id “faq-id-1”
:app_id “app-1”
:section_id “section-1”
:translations
{:en {:published? false
:stags []
...
{:published? false
:id “faq-id-1”
:app_id “app-1”
:section_id “section-1”
:translations
{:en {:published? false
:stags []
...
EXAMPLE TEST CASE
1. Add 1st App with only English languages.
2. Add 2nd App with only English languages
3. Add 1st FAQ under 1st App
4. Lin...
PARTS OF EACH ACTION
“Add APP-1 with only English language”
1. Type of Action Add an App
2. Names App-1
3. Arguments Engli...
EXPRESS ACTIONS AS
CLOJURE DATA
“Add app APP-1 with English language”
:add-app :app-1 {:langs-config [:en]}
ONE UNIT OF ACTION
[:add-app :app-1 {:langs-config [:en]}]
SERIES OF ACTIONS
[[:add-app :app-1 {:langs-config [:en]]
[:add-app :app-2 {:langs-config nil}]
[:add-faq :faq-1 {:app-var...
REDUCE ON ACTIONS
Reduce
{}
[:add-app :app-1 {:langs-config nil}])
{:env {:apps {:app-1 {…}}}
:result [{result-1…}]}
…..}
...
RESULT HASH-MAP COMPRISES OF
{:env {:apps {:app-1 {:app-id “970"}
:app-2 {:app-id “142”}}}
:faqs {:faq-1 {:faq-id “112”}}}...
RESULT HASH-MAP COMPRISES OF
{:env {:apps {:app-1 {:app-id “970"}
:app-2 {:app-id “142”}}}
:faqs {:faq-1 {:faq-id “112”}}}...
RESULT HASH-MAP COMPRISES OF
{:env {:apps {:app-1 {:app-id “970"}
:app-2 {:app-id “142”}}}
:faqs {:faq-1 {:faq-id “112”}}}...
Dispatch On Action Type
Update The Variables in
arguments to its bindings
Call args with relevant
function
Bind the result...
Dispatch On Action Type
Update The Variables in
arguments to its bindings
Call args with relevant
function
Bind the result...
Dispatch On Action Type
Update The Variables in
arguments to its bindings
Call args with relevant
function
Bind the result...
Dispatch On Action Type
Update The Variables in
arguments to its bindings
[:add-faq :faq-1 {:app-var :app-1 ….}])
{:env {:...
Dispatch On Action Type
Update The Variables in
arguments to its bindings
[:add-faq :faq-1 {:app-var :app-1 ….}])
[:add-fa...
Dispatch On Action Type
Update The Variables in
arguments to its bindings
Call args with relevant
function
Bind the result...
Dispatch On Action Type
Update The Variables in
arguments to its bindings
Call args with relevant
function
Bind the result...
Dispatch On Action Type
Update The Variables in
arguments to its bindings
Call args with relevant
function
Bind the result...
Dispatch On Action Type
Update The Variables in
arguments to its bindings
Call args with relevant
function
Bind the result...
VERIFICATION:
COMPARE EXPECTED VS ACTUAL FOR EACH STEP
COMPARE RESULT FOR
FIRST ACTION
{"112"
{:translations
{:en
{:published? false,
:stags [],
:body "Temp body",
:title “Title...
:title "Title 1"
Actual Data in DB
Expected Data
from my Generated result
:title "Title 2"
==
FAIL
COMPARE RESULT FOR
SECO...
DEMO TIME
APPROACHES TO VERIFY
Actions : Hardcoded
Expected Output : Hardcoded
Actions : Simulated
Expected Output : Hardcoded
Actio...
Hardcoded Actions,
Hardcoded Expected Output
ADVANTAGES DISADVANTAGES
Requires no extra knowledge to
understand
Very cumbe...
Simulate list of Actions,
Hardcoded Final Expected Output
ADVANTAGES DISADVANTAGES
Easy to enumerate
Requires knowledge of...
Simulate list of actions,
Generate Expected Output
ADVANTAGES DISADVANTAGES
Easy to enumerate
Requires knowledge of the
DS...
Generate Actions,
Generate Expected Output
ADVANTAGES DISADVANTAGES
Possible to generate large
number of tests
Requires kn...
Generate Actions,
Generate Expected Output
Feature - Issue Audit Trail
Issue Audit Trail
• Maintains a log of
• Who Took an action
• What Action they took
Who - Types of Users
App User Support User
Agents Admins
Helpshift
What - Types of Actions
App User
• Create Issue
• Reply Issue
• etc…
Admin User
• Create Issue
• Reply Issue
• Resolve Iss...
App User
Admin
“Mayank”
Agent
“Agent-2”
Example Issue
Example Issue
Issue
Audit
Logs
DEMO - ISSUE AUDIT TRAIL
Generate Actions,
Generate Expected Output
ADVANTAGES OF WRITING DSL
Discovering very hard to find bugs, for example we found:
Duplicate issue messages being rendere...
DISADVANTAGES OF WRITING DSL
Cost of maintaining DSLs is high.
If your DSL is just data being evaluated at run time,
chang...
CONCLUSION
DSL can be used as a workflow for writing tests.
Separating simulation vs verification of tests.
A step towards...
Further Resources
• Clojure Made Simple - Rich Hickey
• Growing a DSL with Clojure
• Jeanine Adkisson - Design and Prototy...
Any Questions?
@firesofmay
facebook.com/firesofmay/
firesofmay@gmail.com
Upcoming SlideShare
Loading in …5
×

Testing business-logic-in-dsls

698 views

Published on

Slides for my talk at techjam.
Link - http://techjam.org/#dsl_clojure
Video - https://www.youtube.com/watch?v=6l3VMc8rd5o

Published in: Software
  • Be the first to comment

  • Be the first to like this

Testing business-logic-in-dsls

  1. 1. Testing Business Logic Using DSLs in Clojure Mayank Jain Test Engineer
  2. 2. A TALK, IN 6 PARTS 1. What is the problem? 2. Real World Example. 3. Demo of the actual code. 4. Other features tested using same ideas. 5. Advantages/Disadvantages of writing DSLs for testing. 6. QA.
  3. 3. WHAT IS THE PROBLEM? Testing real world stateful business logic is hard
  4. 4. Microwave Oven State Machine Business logic is Stateful
  5. 5. Microwave Oven State Machine Large number transition states
  6. 6. Difficult to enumerate all possible cases Microwave Oven State Machine
  7. 7. Tests become unreadable Microwave Oven State Machine
  8. 8. HELPSHIFT • Embeddable support desk for native apps • Main Features: • Frequently Asked Questions which customers can search • File issues/tickets from within the app
  9. 9. Helpshift Supercell Clash of Clans Gaana General Billing Domain App Section FAQ 1 FAQ 2FAQs Boom Beach Gaana App Translations English Content Hindi Content …. …. …. …. …. Top Down View of FAQs
  10. 10. Example of Gaana FAQ Page Customers can search FAQs Available FAQs FAQ Sections Domain App
  11. 11. FAQ Title FAQ Body FAQ is Visible?
  12. 12. PROBLEM You cannot share FAQs across apps. Big Customers have multiple apps which have same FAQ translations content example “Privacy Policy”
  13. 13. FEATURE: LINKED FAQS
  14. 14. {:published? false :id “faq-id-1” :app_id “app-1” :section_id “section-1” :translations {:en {:published? false :stags [] :body “Privacy Body” :title “Privacy Title”}} :linked_faq_ids [“faq-id-2”] :publish_id “1”} App-1 {:published? false :id “faq-id-2” :app_id “app-2” :section_id “section-2” :translations {:en {:published? false :stags [] :body “Privacy Body" :title “Privacy Title"}} :linked_faq_ids [“faq-id-1”] :publish_id “2”} App-2 faq 1 (Sync faq-1 to App-2)General faq 2 General
  15. 15. {:published? false :id “faq-id-1” :app_id “app-1” :section_id “section-1” :translations {:en {:published? false :stags [] :body “Update Body” :title “Privacy Title”}} :linked_faq_ids [“faq-id-2”] :publish_id “1”} {:published? false :id “faq-id-2” :app_id “app-2” :section_id “section-2” :translations {:en {:published? false :stags [] :body “Privacy Body" :title “Privacy Title"}} :linked_faq_ids [“faq-id-1”] :publish_id “2”} faq 1 Sync faq 2 Update FAQ 1’s Body
  16. 16. {:published? false :id “faq-id-1” :app_id “app-1” :section_id “section-1” :translations {:en {:published? false :stags [] :body “Update Body” :title “Privacy Title”}} :linked_faq_ids [“faq-id-2”] :publish_id “1”} {:published? false :id “faq-id-2” :app_id “app-2” :section_id “section-2” :translations {:en {:published? false :stags [] :body “Update Body" :title “Privacy Title"}} :linked_faq_ids [“faq-id-1”] :publish_id “2”} faq 1 Sync faq 2 Update FAQ 1’s Body
  17. 17. EXAMPLE TEST CASE
  18. 18. 1. Add 1st App with only English languages. 2. Add 2nd App with only English languages 3. Add 1st FAQ under 1st App 4. Link 1st FAQ to 2nd App to create 2nd FAQ 5. Check -> translations of 2nd FAQ == 1st FAQ 6. Update English title of FAQ-1 7. Check -> translations of 2nd FAQ == 1st FAQ 8. Delete FAQ-1 9. Check if 1st FAQ is deleted in DB 10.Assert 2nd FAQ should remain as it is in database. Simulation Verification
  19. 19. PARTS OF EACH ACTION “Add APP-1 with only English language” 1. Type of Action Add an App 2. Names App-1 3. Arguments English Language 4. Expected Result As per Spec/Modal of the system 5. Actual Result Database
  20. 20. EXPRESS ACTIONS AS CLOJURE DATA “Add app APP-1 with English language” :add-app :app-1 {:langs-config [:en]}
  21. 21. ONE UNIT OF ACTION [:add-app :app-1 {:langs-config [:en]}]
  22. 22. SERIES OF ACTIONS [[:add-app :app-1 {:langs-config [:en]] [:add-app :app-2 {:langs-config nil}] [:add-faq :faq-1 {:app-var :app-1 ...}] [:link-faq :faq-2 {:faq-var :faq-1 :app-var :app-2 ...}] [:update-faq nil {:app-var :app-1 :faq var :faq-1 ...}] [:update-faq nil {:app-var :app-2 :faq-var :faq-2 ...}] [:delete-faq nil {:faq-var :faq-1 ...}]]
  23. 23. REDUCE ON ACTIONS Reduce {} [:add-app :app-1 {:langs-config nil}]) {:env {:apps {:app-1 {…}}} :result [{result-1…}]} …..} Reduce [:add-app :app-2 {:langs-config nil}]) {:env {:apps {:app-1 {…} :app-2 {…}}} :result [{result-1…} {result-2…}]} …..} {:env {:apps {:app-1 {…}}} :result [{result-1…}]} …..} And so on…
  24. 24. RESULT HASH-MAP COMPRISES OF {:env {:apps {:app-1 {:app-id “970"} :app-2 {:app-id “142”}}} :faqs {:faq-1 {:faq-id “112”}}}} :result [{:action-type :add-faq :expected {….} :actual {….}} {…more}] “112” {…state…} …more} Environment (:env) - Contains bindings of vars
  25. 25. RESULT HASH-MAP COMPRISES OF {:env {:apps {:app-1 {:app-id “970"} :app-2 {:app-id “142”}}} :faqs {:faq-1 {:faq-id “112”}}}} :result [{:action-type :add-faq :expected {….} :actual {….}} {…more}] “112” {…state…} …more} Result (:result) - Contains Actual And Expected Result
  26. 26. RESULT HASH-MAP COMPRISES OF {:env {:apps {:app-1 {:app-id “970"} :app-2 {:app-id “142”}}} :faqs {:faq-1 {:faq-id “112”}}}} :result [{:action-type :add-faq :expected {….} :actual {….}} {…more}] “112” {…state…} …more} Current Generated Expected State
  27. 27. Dispatch On Action Type Update The Variables in arguments to its bindings Call args with relevant function Bind the result to the given var in Global Data Store Expected current state Store Actual current database state What happens inside the reducer?
  28. 28. Dispatch On Action Type Update The Variables in arguments to its bindings Call args with relevant function Bind the result to the given var in Global Data Store Expected current state Store Actual current database state [:add-app :app-1 {:langs-config nil}]) [:add-app :app-1 {:langs-config nil}]) (add-app {:langs-config nil}) {:env {:apps {:app-1 {:app-id "970"}}}}
  29. 29. Dispatch On Action Type Update The Variables in arguments to its bindings Call args with relevant function Bind the result to the given var in Global Data Store Expected current state Store Actual current database state [:add-app :app-2 {:langs-config nil}]) [:add-app :app-2 {:langs-config nil}]) (add-app {:langs-config nil}) {:env {:apps {:app-1 {:app-id “970"} :app-2 {:app-id “142"}}}}
  30. 30. Dispatch On Action Type Update The Variables in arguments to its bindings [:add-faq :faq-1 {:app-var :app-1 ….}]) {:env {:apps {:app-1 {:app-id “970"} :app-2 {:app-id “142”}}}
  31. 31. Dispatch On Action Type Update The Variables in arguments to its bindings [:add-faq :faq-1 {:app-var :app-1 ….}]) [:add-faq :faq-1 {:app-id “970” …}]) {:env {:apps {:app-1 {:app-id “970"} :app-2 {:app-id “142”}}}
  32. 32. Dispatch On Action Type Update The Variables in arguments to its bindings Call args with relevant function Bind the result to the given var in Global Data [:add-faq :faq-1 {:app-var :app-1 ….}]) [:add-faq :faq-1 {:app-id “970” …}]) (add-faq {:app-id “970” ….})
  33. 33. Dispatch On Action Type Update The Variables in arguments to its bindings Call args with relevant function Bind the result to the given var in Global Data [:add-faq :faq-1 {:app-var :app-1 ….}]) [:add-faq :faq-1 {:app-id “970” …}]) (add-faq {:app-id “970” ….}) {:env {:apps {:app-1 {:app-id “970"} :app-2 {:app-id “142”}} :faqs {:faq-1 {:faq-id “112”}}}}
  34. 34. Dispatch On Action Type Update The Variables in arguments to its bindings Call args with relevant function Bind the result to the given var in Global Data Store Expected current state [:add-faq :faq-1 {:app-var :app-1 ….}]) [:add-faq :faq-1 {:app-id “970” …}]) (add-faq {:app-id “970” ….}) {:env {:apps {:app-1 {:app-id “970"} :app-2 {:app-id “142”}} :faqs {:faq-1 {:faq-id “112”}}}} {:env {:apps {:app-1 {:app-id “970"} :app-2 {:app-id “142”}}} :faqs {:faq-1 {:faq-id “112”}}}} :result [{:action-type :add-faq :expected {….}}
  35. 35. Dispatch On Action Type Update The Variables in arguments to its bindings Call args with relevant function Bind the result to the given var in Global Data Store Expected current state Store Actual current database state [:add-faq :faq-1 {:app-var :app-1 ….}]) [:add-faq :faq-1 {:app-id “970” …}]) (add-faq {:app-id “970” ….}) {:env {:apps {:app-1 {:app-id “970"} :app-2 {:app-id “142”}} :faqs {:faq-1 {:faq-id “112”}}}} {:env {:apps {:app-1 {:app-id “970"} :app-2 {:app-id “142”}}} :faqs {:faq-1 {:faq-id “112”}}}} :result [{:action-type :add-faq :expected {….} :actual {….}}
  36. 36. VERIFICATION: COMPARE EXPECTED VS ACTUAL FOR EACH STEP
  37. 37. COMPARE RESULT FOR FIRST ACTION {"112" {:translations {:en {:published? false, :stags [], :body "Temp body", :title “Title 1"}}, :linked-faq-ids #{}}} Actual Data in DB Expected Data from my Generated result {"112" {:translations {:en {:published? false, :stags [], :body "Temp body", :title “Title 1"}}, :linked-faq-ids #{}}} == PASS
  38. 38. :title "Title 1" Actual Data in DB Expected Data from my Generated result :title "Title 2" == FAIL COMPARE RESULT FOR SECOND ACTION
  39. 39. DEMO TIME
  40. 40. APPROACHES TO VERIFY Actions : Hardcoded Expected Output : Hardcoded Actions : Simulated Expected Output : Hardcoded Actions : Simulated Expected Output : Generated Actions : Generated Expected Output : Generated
  41. 41. Hardcoded Actions, Hardcoded Expected Output ADVANTAGES DISADVANTAGES Requires no extra knowledge to understand Very cumbersome to enumerate Anyone can add/edit tests Modification is hard False negatives are not possible
  42. 42. Simulate list of Actions, Hardcoded Final Expected Output ADVANTAGES DISADVANTAGES Easy to enumerate Requires knowledge of the DSL Very readable Maintenance overhead of DSL Shareable with dev to simulate bugs Does not check intermediate state False negatives are not possible Expected output may have data which is available only at runtime like faq-ids
  43. 43. Simulate list of actions, Generate Expected Output ADVANTAGES DISADVANTAGES Easy to enumerate Requires knowledge of the DSL Very readable Maintenance overhead of DSL Shareable with dev to simulate bugs Maintenance overhead of Expected Modal Checks intermediate state False negatives are possible Runtime data is available like faq-ids
  44. 44. Generate Actions, Generate Expected Output ADVANTAGES DISADVANTAGES Possible to generate large number of tests Requires knowledge of the DSL Possible to Shrink failed test case using test.check library Maintenance overhead of DSL Shareable with dev to simulate bugs Maintenance overhead of Expected Modal Checks intermediate state False negatives are possible Maintenance overhead of generative code for list of actions.
  45. 45. Generate Actions, Generate Expected Output Feature - Issue Audit Trail
  46. 46. Issue Audit Trail • Maintains a log of • Who Took an action • What Action they took
  47. 47. Who - Types of Users App User Support User Agents Admins Helpshift
  48. 48. What - Types of Actions App User • Create Issue • Reply Issue • etc… Admin User • Create Issue • Reply Issue • Resolve Issue • Edit Tags • etc… Agent User • Create Issue • Reply Issue • Resolve Issue • Edit Tags • etc…
  49. 49. App User Admin “Mayank” Agent “Agent-2” Example Issue
  50. 50. Example Issue Issue Audit Logs
  51. 51. DEMO - ISSUE AUDIT TRAIL Generate Actions, Generate Expected Output
  52. 52. ADVANTAGES OF WRITING DSL Discovering very hard to find bugs, for example we found: Duplicate issue messages being rendered only if the number of messages were "just right". Finding ordering bugs. Increase in developer/tester productivity
  53. 53. DISADVANTAGES OF WRITING DSL Cost of maintaining DSLs is high. If your DSL is just data being evaluated at run time, changes in feature code will not throw any compile time errors like function parameters being changed in DSL code. You have to educate your team members to learn your specific DSL for that feature to be able to understand the tests.
  54. 54. CONCLUSION DSL can be used as a workflow for writing tests. Separating simulation vs verification of tests. A step towards ability to generate tests instead of writing them.
  55. 55. Further Resources • Clojure Made Simple - Rich Hickey • Growing a DSL with Clojure • Jeanine Adkisson - Design and Prototype a Language In Clojure • John Hughes - Testing the Hard Stuff and Staying Sane • Reid Draper - Powerful Testing with test.check • Clojure Tutorials on DSLs with Tim Baldridge (Paid)
  56. 56. Any Questions? @firesofmay facebook.com/firesofmay/ firesofmay@gmail.com

×