Integration testing in enterprises using TaaS (Test as a Service)

1,415 views

Published on

If a product technology stack cannot be unified in an organization, then why FORCE the Test Automation technology stack to be the same across the same organization?

"TaaS - Test as a Service" is an open source framework that allows teams within large organisations to use tools and technologies that will best test individual products, and then use TaaS to do the end-2-end automated integration testing across these products.

Integration testing in enterprises using TaaS (Test as a Service)

  1. 1. Integra(on  tes(ng  in  Enterprises  using  TaaS   Anand  Bagmar   So#ware  Quality  Evangelist   Principal  Consultant,  ThoughtWorks  
  2. 2. Organiza?on  Objec?ve  
  3. 3. Business  spread  across  the  globe  
  4. 4. Test  Automa?on  
  5. 5. Is  Test  Automa?on  treated  as  1st  class  ci?zen  in  your  organiza?on?  •  Value  •  Quality  
  6. 6. Quick  survey  •  Part  of  a  large  organiza?on?  •  #  of  products  in  the  porLolio?  •  #  of  projects  across  all  these  products?  •  Technology  stack  of  these  products?  •  Part  of  merged  /  acquired  companies?     –  Same  technology  of  “new”  products?  •  Distributed  teams?  •  Common  Test  Automa?on  framework?  
  7. 7. Case  Study  
  8. 8. Outlook  Web   Outlook  Sync   Access  (OWA)   (Android)   (Browser)   Outlook  (Mac)   …   Outlook  (Windows)   Outlook   …  
  9. 9. How  will  you  automate  these?  
  10. 10. The  solu?on  Outlook  for   Outlook  for   Outlook  Web   Outlook  Sync  Windows   Mac   Access   for  Android  •  White  /  QTP   •  Automator   (OWA)   •  Robo?um  /   •  Selenium  /   Java   Ruby  
  11. 11. For  a  user  who  has  installed  Outlook  on   –  Windows  OS  –  desktop  version,  and,   –  Mac  OS  –  desktop  version   –  Android  device  –  na?ve  mobile  app     How  will  you  test  ….     An  email  dra7ed  in  one  product  is  reflected  in   the  other  products    
  12. 12. Can  you  use  the  same  technology   stack  for  tes(ng  each  variant  of   Outlook?  
  13. 13. On  Windows  •  Create  an  email  and  save  it  as  dra#   In  Browser  (OWA)   •  Verify  dra#  email  is  seen   On  Mac   •  Verify  dra#  email  is  seen   On  Android   •  Verify  dra#  email  is  seen   …  
  14. 14. …   In  OWA   •  Change  the  dra#  email  content   On  Windows   •  Verify  updated  email  content  is  seen   On  Mac   •  Verify  updated  email  content  is  seen   On  Android   •  Verify  updated  email  content  is  seen  
  15. 15. The  problem  Outlook  for   Outlook  for   Outlook  Web   Outlook  Sync  Windows   Mac   Access   for  Android  •  White  /  QTP   •  Automator   (OWA)   •  Robo?um  /   •  Selenium  /   Java   Ruby  
  16. 16. The  solu(on  
  17. 17. Create  &  Save   (Test   as  DraE   Framework   for)     1   Outlook  on   Windows     4   Verify  DraE   End-­‐2-­‐End   Integra-on   Test   Verify  DraE     2   present   (Test   Framework   for)   3   Outlook  Orchestrator   Sync  on   Modify  DraE   Android    
  18. 18. Create  &  Save   Test   as  DraE   Framework   TaaS   for     1   Server   Outlook  on   Windows     4   Verify  DraE   End-­‐2-­‐End  Integra-on   TaaS   Service  Providers   Test   Client  Framework   Verify  DraE     2   present   Test   Framework   3   TaaS   for  Outlook   Orchestrator   Server   Sync  on   Modify  DraE   Android    
  19. 19. TaaS  Server  •  Specify  the  contract  •  Implement  the  contract  •  Return  the  results  •  Run  TaaS  Server  (REST  service)  •  Serve  TaaS  Client  requests   –  As  separate  processes  
  20. 20. The  contract  
  21. 21. Implement  the  contract  •  Update  tests  to  -­‐   –  Handle  input  parameters   •  From  Environment  variables   –  Execute  the  “test”  
  22. 22. Return  the  results  •  Output  parameters   –  As  console  output  between  special  markers  •  Console  logs  •  Excep?ons  (if  any)  •  All  return  values  are  in  “json”  format  
  23. 23. Run  TaaS  Server  •  Dedicated  machine  •  Provide  IP  address  /  host  name  +  port  to   poten?al  clients  
  24. 24. TaaS  Client  •  Collect  the  input  parameters  •  Invoke  the  contract  •  Consume  the  result  •  Con?nue  with  test  orchestra?on  
  25. 25. Some  code  snippets    from  Cucumber  /  Ruby  world  
  26. 26. Specify  the  Contract  
  27. 27. Specify  the  Contract  
  28. 28. Implement  the  Contract  
  29. 29. Parse  Input  parameters  
  30. 30. Return  Output  parameter  values  
  31. 31. Invoke  TaaS  Contract  
  32. 32. Other  features  •  Timeout  •  Console  logs  and  errors  •  Developed  in  Ruby  using  Sinatra  •  Available  on  github!  
  33. 33. Does  it  really  work?  
  34. 34. Another  analogy  
  35. 35. •  Error  codes  for  the  service  •  Service  Discovery  •  Security  •  Publish  as  a  gem  
  36. 36. Why  is  this  a  good  idea?  •  No  code  duplica?on  •  Implementa?on  of  contract  lies  with  the   framework  tes?ng  that  product   –  Evolves  with  product  changes  •  Each  product  is  tested  in  the  “best”  possible   way  •  Decoupling  of  technologies  •  Its  –  providing  Test  as  a  Service!!!  
  37. 37. Anand.Bagmar@thoughtworks.com                        Abagmar@gmail.com        @BagmarAnand   hcp://essenceo#es?ng.blogspot.com  hcps://github.com/anandbagmar/taas   hcp://in.linkedin.com/in/anandbagmar/  

×