Continuous Interop Testing


Published on

This presentation represents the dimensions I have worked with in the development, testing, and cloud space.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Continuous Interop Testing

  1. 1. Continuous Interoperability Testing<br />TestForgeStatement of Direction<br />John Cocke<br />Strategy Architect<br />
  2. 2. Agenda<br />What is TestForge?<br />What are the issues with the current DOD/DISA T&E Process and Environment?<br />Why are we developing TestForge?<br />What are our goals and approach to TestForge?<br /><ul><li> Hosting
  3. 3. IA, C&A
  4. 4. Pilots and Selection Criteria
  5. 5. What is the end state?
  6. 6. What is the roadmap for getting us to the end state?</li></ul>Development Release Schedule and % Complete?<br />
  7. 7. What is TestForge?<br />Testforge is an event-driven net-centric test bed capable of providing<br />24 x 7 real-time information regarding any code, functional, or performance<br />changes to application components in a user’s particular Testforge<br />environment.<br />Testforge is offered as a tightly integrated service with<br />or it can be delivered as a shrink-wrapped stand-alone capability<br />for installation and use in a customer environment.<br />TEMC offers Testforge as a service including developer/administrative<br />to assist customers or as a platform service offering with no organizational <br />support.<br />
  8. 8. Current T&E Process and Environment<br />5 Test Disciplines<br /><ul><li> Developmental Testing (DT)
  9. 9. Operational Testing (OT)
  10. 10. Interoperability Testing (IOP)
  11. 11. Information Assurance (IA)
  12. 12. Certification & Accreditation (C&A)</li></ul>3 Decision Makers<br /><ul><li> Milestone Decision Authority (MDA)
  13. 13. Joint Staff (JS)</li></ul> Director of Command, Control, Comm., Computers (J6)<br /><ul><li> Designated Accrediting Authority (DAA)</li></li></ul><li>Support Implemented<br />User Training<br />OT&E<br />Tester Training<br />DT&E<br />Pilot<br />Record<br />OTRR<br />Eval Report<br />OTRR<br />Test Concept<br />Brief<br />Test Plan<br />Approved<br />Deployment<br />Decision Review<br />DIACAP<br />IAC&A<br />Operational Test Plan<br />Interop Testing<br />Interop Cert<br />60 days<br />60 days<br />60 days<br />14 days<br />Current T&E Process and Environment<br /><ul><li>5Test Disciplines – DT, OT, IOP, IA, C&A
  14. 14. 3 Decision Makers – MDA, JS J6,DAA</li></ul>T&E Plan – Test – Report cycle can exceed six months!<br />
  15. 15. 2hr. access window<br />Under Construction<br />Wrong test data <br /> Source <br /> Application<br />Dependency Constraints<br />System availability constraints restrict accessibility for development and delivery.<br />
  16. 16. Why TestForge<br />* Leverage DoDAssets* Provide Continuous Interoperability Testing<br />Continuous TestingFramework<br /><br />RACE<br />Functional Testing<br />Performance Testing<br />IA Testing<br />IOP Testing<br /><ul><li> Development Repository
  17. 17. Development Application </li></ul> Packaging<br /><ul><li> Centralized Test Management
  18. 18. Centralized Test Repository
  19. 19. Development Environment
  20. 20. Pre-Certified Environments</li></ul> to enhance time to market<br />
  21. 21. Provide continuous testing and build management <br />Continuous TestingFramework<br /><br />RACE<br /><ul><li> Developers commit code changes into the source control repository (Subversion).
  22. 22. The automated build server constantly monitors the repository for changes (Hudson)
  23. 23. As changes are made, the new code is checked out from the build server
  24. 24. The new code is built and run through a series of smoke tests (unit and functional) and static code analysis (security, complexity, standards adherence, well-formed code, etc)
  25. 25. Issues are immediately reported to team members via email or other CommunityForge channels
  26. 26. Builds are packaged and can be automatically deployed to Testforge for execution of heavier tests such as regression, performance , scale testing, IA testing, and integration testing.
  27. 27. Builds are also available for manual (on-demand) deployment to sandbox or demonstration servers for manual inspection and acceptance. </li></li></ul><li>TestForge Notional Scope<br />Virtual Service Testing<br />Software Testing Environments<br />To provide:<br />Fully Automated Provisioning<br />Multicloud API<br />Direct Virtualization<br />Bare Metal Virtualization<br />Cloud Management<br />.gov<br />Public Cloud<br />RACE<br />
  28. 28. DISA/RACE Integration - notional<br />
  29. 29. Initial Project Set-Up<br />
  30. 30. Test Bed Preparation<br />(all a developer should need to know about what’s under the hood)<br />
  31. 31. Build Management<br />
  32. 32. Scenario Creation<br />
  33. 33. Continuous InteroperabilityThe Testforge End State<br />
  34. 34. John R. Cocke<br />The Wave Group of VA, LLC<br />804-761-3863<br /><br />