Main Point: Dependency constraints are a critical problem for Netcentric applications. It’s a root cause of delays, increasing infrastructure costs, and quality problems.---------------------Development and testing should be an integrated activity, but we don’t have the tools to really make that work in parallel because of constraints and dependencies in the process. Constraints can take many forms:Still under developmentWrong data-can’t testOnly 2 hours of access
Projects are initially stood up in Forge.mil using the SoftwareForge repository, in a future release cloud resources for all Forge.mil projects will be provisioned by the TestForge-provided cloud management capability(IaaS provisioning). This assures continuity and ease from development all the way through push to production. The PKI and CAC credentials (roles/permissions) inherent in the developers, testers, administrators, and project managers for “this project” are established at “SoftwareForge time”. The cloud resources used for Dev and TEST and PROD will be different, however the vision is for a single TestForge cloud management capability to handle all provisioning.Initial code and unit test is performed in SoftwareForge, however developers will still be able to access SoftwareForge codebases after projects have moved to TestForge. The build management capabilities, specifically notifications within SoftwareForge will be integrated with TestForge and will be extended to trigger TestForge’s “continuous interoperability” cycle (assumes clean compile and code analysis scores (if required).When the project is ready to move into TestForge, the TestForge “account management” operational area assures that the developers, testers, administrators, and project managers are correctly permissioned (based on what was established at SoftwareForge time. These people will have the CRUD rights over those objects needed to complete a test and IA cycle: assets, scenarios, physical services (including IA controls and available application services), virtual services, etc.) PKI and CAC credentials in the initial Forge.mil edition of TestForge will be acquired from Forge.mil. The “Workgroup Edition” of TestForgebeing developed for the 350th ELSW at Hanscom is not addressed in this document.The Forge.mil Build Management capability is transparently provided by SoftwareForge.mil to TestForge. Notifications of successful and failed builds will be messaged in the TestForge Console and reporting stores. Asset Management is the operational area within TestForge where “system modules” (mostly IaaS in nature) and their related “assets” (middleware, databases, etc.) are created and/or selected by a tester. TestForge testing concepts and terminology differ slightly from the SoftwareForge approach to testing. The goal is complete transparency. However, SoftwareForge testing assets (including access-related) will be migrated to TestForge repositories at the time the project is moved to TestForge. In addition, all codebase originally developed in SoftwareForge will remain available in SoftwareForge for changes and build management at any time during the TestForge cycle. Physical entry points to applications (where available) and virtual services are created. IA compliance requirements are identified and appropriate controls are created(reused) as physical or virtual services. TestForge seeks to minimize the need for physical SOA infrastructure. The key strategy for doing this is to use our virtual service testing capability (currently iTKO LISA and VSE) to abstract SOA components. Developers, testers, administrators work together to vision test cases which are built at this time. We now begin scenario construction. The “scenario” is what is actually tested once the codebase, physical entry points, virtual services, and IA controls are identified and included in the scenario. The scenario may include multiple “assets” (each with their own operating system, middleware, codebase, apps) plus test cases to be executed plus virtual services to be executed. Once a scenario with all of its components is assembled and a triggering event (could be a clean compile in SoftwareForge, a state change in a database or client app, a successful change and retest of a virtual service, etc.), this project can be added into the Continuous Interoperability (CI) stack.Real time C&A reporting requirements coming out of TestForge IA controls during CI cycles need to be defined and incorporated.
Agenda<br />What is TestForge?<br />What are the issues with the current DOD/DISA T&E Process and Environment?<br />Why are we developing TestForge?<br />What are our goals and approach to TestForge?<br /><ul><li> Hosting
What is the roadmap for getting us to the end state?</li></ul>Development Release Schedule and % Complete?<br />
What is TestForge?<br />Testforge is an event-driven net-centric test bed capable of providing<br />24 x 7 real-time information regarding any code, functional, or performance<br />changes to application components in a user’s particular Testforge<br />environment.<br />Testforge is offered as a tightly integrated service with Forge.mil<br />or it can be delivered as a shrink-wrapped stand-alone capability<br />for installation and use in a customer environment.<br />TEMC offers Testforge as a service including developer/administrative<br />to assist customers or as a platform service offering with no organizational <br />support.<br />
Current T&E Process and Environment<br />5 Test Disciplines<br /><ul><li> Developmental Testing (DT)
Pre-Certified Environments</li></ul> to enhance time to market<br />
Provide continuous testing and build management <br />Continuous TestingFramework<br />Forge.mil<br />RACE<br /><ul><li> Developers commit code changes into the source control repository (Subversion).
The automated build server constantly monitors the repository for changes (Hudson)
As changes are made, the new code is checked out from the build server
The new code is built and run through a series of smoke tests (unit and functional) and static code analysis (security, complexity, standards adherence, well-formed code, etc)
Issues are immediately reported to team members via email or other CommunityForge channels
Builds are packaged and can be automatically deployed to Testforge for execution of heavier tests such as regression, performance , scale testing, IA testing, and integration testing.
Builds are also available for manual (on-demand) deployment to sandbox or demonstration servers for manual inspection and acceptance. </li></li></ul><li>TestForge Notional Scope<br />Virtual Service Testing<br />Software Testing Environments<br />To provide:<br />Fully Automated Provisioning<br />Multicloud API<br />Direct Virtualization<br />Bare Metal Virtualization<br />Cloud Management<br />.gov<br />Public Cloud<br />RACE<br />