Ashwini - Effective use of CI by QA


Published on

Ashwini Malthankar
Organization: ThoughtWorks
Topic: Effective use of CI by QA
Presented in vodQA - THE TESTING SPIRIT! on Oct 7, 2010 in ThoughtWorks, Pune

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Ashwini - Effective use of CI by QA

  1. 1. Thursday, October 7, 2010
  2. 2. Effective use of Continuous Integration by QA  Ashwini Malthankar  Lead Consultant QA  ThoughtWorks
  3. 3. Agenda Problem Statements About Continuous Integration How does Continuous Integration help resolve problem Case study on large scale distributed project Open source / commercial tools Q&A
  4. 4. Scenario for Thought Exercise Given it’s a Thursday today when we have Demo to customer then what is the state of a Tester? We have to showcase new facebook feature.
  5. 5. Problem Statements Identifying good build for Demo, Release or Regression testing Testing on various environments like the Grid shown below OS App Browsers Versions DB Servers Windows WebSphere Firefox ? Oracle Linux Weblogic IE ? MySql mac JBoss Safari ? Teradata
  6. 6. Why is it important to solve Constant availability of a "current" build for testing, demo, or release purpose and to avoiding last-minute chaos at release time.
  7. 7. About CI ?
  8. 8. High level Overview of Build Server http://twistcruise:8153/cruise/tab/pipeline
  9. 9. Pipeline in the box Pipeline Stages Jobs Jobs Jobs Jobs
  10. 10. Project/Product Development Process Flow R Code e Functional Test Plan + Unit Coverage p Automation Test case Testing o Test Suites r Product t s System Requirement Design Coding Testing Build Scope Test Document framework + Test cases BackEnd Continuous Build Testing Integration API level Defects Cycle testing Reports Reports
  11. 11. Development Tools usage Flow R Mingle e UI Automation / p Cucumber Test Plan + github RCov o Test case r t Ruby on Rails s System Requirement MSL Integration RSpec Testing Postgress Usecases Stub + Test cases available for Testing Go Defects Cycle Reports Reports
  12. 12. How does Continuous Integration help to resolve current problem • MATs /Smoke tests, SRGs, LRGs running it continuously to quickly Identify Demo, Release candidate build on any environment. • frequent code check-in pushes developers to create modular, less complex code • metrics generated from automated testing and Continuous Integration
  13. 13. Give visual representation of CI QA Builds Platform Specific Builds Schedules Performance Builds
  14. 14. Q&A
  15. 15. Case study on large scale distributed Project Project details  “largest distributed agile project“  for every check in, a build is triggered, installers deployed continuously  New release every 6 weeks.  260 Build & Deployment Servers  3 active development branches, 'branch per release pattern', SVN model.  Application has Service Oriented Architecture, having multiple tiers and is Web-based  15 Independent Installers to deploy on Dev, QA, UAT, Performance, Staging, Production environments
  16. 16. Deployment Architecture & Environments Channels channels channels Third Party Services(15) Apps Database
  17. 17. Iteration -0 Build and Installer Compile SVN Unit test Smoke test Publish Integration Test Build test
  18. 18. Iteration - 1 Build and Installer Compile Unit test Last good Last good Build and Installer SVN Smoke Test Publish Compile Unit test Build and Installer Last good Compile Unit test Iteration – 1 Build test
  19. 19. Build Cycle and Flow Build&Installer Compile SVN CodeQaulity Unit test Integration Smoke QA/validation Showcase Verification UAT Performance Staging
  20. 20. Build Layers Dev- Trunk Build Generate Install the Publish the feedback(28 Test the installer installer installer installer build in parallel) Install the Verify the Publish the Get the installer application installer installer Test Box Deployment Get the installer Install the Verify the Publish the Showcase application installer installer env Deploy
  21. 21. Continuous Integration things to watch • Initial setup time required • Well-developed test-suite required to achieve automated testing advantages • Large-scale re-factoring can be troublesome due to continuously changing code base • Hardware costs for build machines can be significant since CI like production makes more sense • Final feedback is only with system in production
  22. 22. What more can be Done • Every build with Performance test execution to know the performance of the application per check-in. • Code health check apart from Code Coverage which we can be integrated in the build with tools like Code City. It can give the code complexity metrics to know how complex the code is getting per check-in. • Continuous Integration  Continuous Deployments  Continuous Regression  Continuous Performance  Continuous Software Health
  23. 23. What are other commercial tools available in the market AnthillPro — continuous integration server by Urbancode Apache Continuum — continuous integration server Bamboo — commercial continuous integration server by Atlassian Software Go — Commercial agile build and release management software- ThoughtWorks Hudson — MIT-licensed, written in Java, runs in servlet container TeamCity — commercial continuous-integration server by JetBrains with free professional edition
  24. 24. Q&A