Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Obsidian Agile DevOps


Published on

Agile DevOps is Obsidian’s approach to agile development, continuous integration, continuous testing, and continuous deliverythrough the use of automated tools, and streamlined processes. We deliver incremental development continuously to production, which reduces defects, eliminates excess cycle time, provides continuous feedback and eliminates outage windows when deploying to production.

In this white paper, we will take a deeper dive into Obsidian’s Agile DevOps approach and how leaders in the federal government can leverage this solution for faster adoption of DevOps into existing and new federal IT programs.

Published in: Technology
  • Login to see the comments

  • Be the first to like this

Obsidian Agile DevOps

  1. 1. CSC AND MERCK PROTECT WHAT MATTERS — ENABLING SUPERIOR SECURITY Cloud Integration IT Services SolutionsArchitecture Agile DevOps Providing a comprehensive solution for the government for agile software development, continuous integration, continuous testing and continuous delivery. 3 Metro Center, Suite 700 Bethesda, MD 20814
  2. 2. Agile DevOps Obsidian Proprietary Page i Cloud Integration • IT Services • Solutions Architecture Table of Contents Executive Summary....................................................................................................................1 Agile DevOps Approach .............................................................................................................2 Agile Development in Federal Contracting.....................................................................3 Continuous Integration (CI), and Continuous Delivery (CD) .........................................4 Continuous Monitoring and Continuous Feedback.......................................................6 Measuring Agile DevOps Success.............................................................................................7 Agile DevOps Metrics for Baselining and Measuring Success.....................................7 Process Metrics................................................................................................................7 Why Ask for an External Agile DevOps Maturity Assessment......................................8 Next Steps ...................................................................................................................................9 Conclusion ..................................................................................................................................9
  3. 3. Agile DevOps Obsidian Proprietary Page 1 Cloud Integration • IT Services • Solutions Architecture Executive Summary Today we live in a culture where we must have everything instantly from social media, news, instant messaging, and the web. Customers, whether they are commercial customers or Federal IT customers, all live in the same era of astonishing technology, instant information and rampant social networking. IT organizations have been traditionally setup to deal with a long tedious Software Development Lifecycle (SDLC) that has customers waiting for releases every quarter, bi- annually or annual updates to software that doesn’t meet the end-user needs, and still has defects requiring addional updates. In 2001, The Manifesto for Agile Software Development (Agile) was first introduced to address changing requirements, incremental delivery, self-organizing development teams, and quality software. Agile development, particularly SCRUM, started becoming prevalent in the Federal market around 2010, and yet many Federal IT organizations are still using traditional waterfall methodology to deliver capability to agency customers. Why? The reasons are simple. 1.) Many agile development efforts have been seen as failures in the federal government, 2.) adoption has been slow because of FAR regulations, and 3.) educating both the contractor and federal IT community didn’t start making traction until after 2010. Some Federal CIO’s might have lobbied earlier, but in general RFI’s and RFP’s didn’t start including agile as a preferred methodology for software development until after 2010 and still today we see many RFI’s and RFP’s that specify waterfall as the preferred methodology for delivering software. Is Agile the answer to delivering software faster, better, and cheaper to customers? Yes and No. Agile only addresses the requirements through software development and doesn’t address rapid delivery of software to production systems. To address the rapid delivery to production and disconnect between development and operation teams, a new trend has immerged - software developmentand operations (DevOps). DevOps addresses the collaboration, and automation between software development and operation teams. Agile DevOps is Obsidian’s approach to agile development, continuous integration, continuous testing, and continuous deliverythrough the use of automated tools, and streamlined processes. We deliver incremental development continuously to production, which reduces defects, eliminates excess cycle time, provides continuous feedback and eliminates outage windows when deploying to production. In this white paper, we will take a deeper dive into Obsidian’s Agile DevOps approach and how leaders in the federal government can leverage this solution for faster adoption of DevOps into existing and new federal IT programs.
  4. 4. Agile DevOps Obsidian Proprietary Page 2 Cloud Integration • IT Services • Solutions Architecture Agile DevOps Approach While most IT Executives, Program Managers, and technical staff are well versed in both Agile and DevOps methodologies, most implementations are partial forms of either Agile or DevOps because of a number of extenuating circumstances. For example, many programs implementing Agile suffer from a lack of understanding as to how to manage requirements, deal with change, and still deliver quality code while staying “agile”. Other programs implementing DevOps do not have a complete understanding of the processes/toolsets required, and how to manage diverse teams who may be working under different contracts, using different toolsets and may not be incentivized to work collaboratively to achieve a continuous stream of production-ready software.While there is not a single approach to address all scenarios, Obsidian has developed a methodology for Agile DevOps and an approach for leaders in the federal government to adopt these methodologies within their IT programs. Figure 1. Obsidian’s Agile DevOp Approach. Our approach enables rapid implementation with continuous feedback throughout software development, testing, and deployment to deliver quality services. The Obsidian Agile DevOps approach encompasses the entire SDLC. As seen in Figure 1, our approach includes agile development, continuous integration, continuous testing, and continuous delivery while providing continuous monitoring and feedback throughout every phase of the lifecycle. Automation is a critical component of a successful DevOps approach, and the tools in this space continue to rapidly advance.Our approach is agnostic to a particular toolset, and is 001 Obsidian, 02/24/2015 Agile DevOps Continuous Feedback Continuous Feedback Continuous Feedback Continuous Feedback Continuous Testing Continuous Integration Continuous Delivery Agile Development Product UAT QA Repository Manager Provisioning Tools CI Server Collaboration Test Scripts Test Suite CIServer Testing Metrics INT QA UAT Issue Tracking Dev Code Repository CIServer Build +Unit Test +Code Quality 2 Weeks Daily Standup Sprint Backlog Product Backlog Burndown Chart Potentially Shippable Product User Stories Commit Code Quality Metrics Artifact Test Environment Auto Ticket Creation Infrastructure as code Repository Manager
  5. 5. Agile DevOps Obsidian Proprietary Page 3 Cloud Integration • IT Services • Solutions Architecture customizable based on customer preference. The key steps for automation that enable our Agile DevOps approach include: 1. Daily Code Commit. Developers on a daily basis check-in code into a central source code repository. 2. Automated Builds. A Continuous Integration (CI) server is continually polling the source repository for changes, and when a change occurs the code is checked out of the repository and built. The built software is stored in a repository manager by the CI server. 3. Automated Testing. The code is automatically unit tested, code quality tested, smoke and UI tested, and performance tested. 4. Automated Delivery. The built version is deployed using provisioning tools that treat infrastructure as code. The entire pipeline, described in the next sections, is tailorable to add in manual validation at any phase. We increase efficiencies by automating the promotion of software code from development to testing and into production. Our Agile DevOps approach forms collaborative teams, and reduces the number of issues occurring during development, deployment, and operations, resulting in faster delivery of applications/features, and reduces Operations & Maintenance (O&M) costs. Agile Development in Federal Contracting Our agile development approach is flexible enough to use Scrum, Kanban, Test Driven Development (TDD), Feature Driven Development (FDD), SAFe, and others. Our experience is that many federal contracts are using Scrum, shown in Figure 2, and are becoming quite successful at using it. The trend as of late has been for the federal government to act as the Product Owner and ScrumMaster. In our experience it is more beneficial to have the ScrumMaster be part of the contractor development team to identify impediments and remove them without buderning our government customer. In either case, there are going to be pro’s and con’s to both models, and both can be successful. It’s up to the federal government which model works best for them. In cases where the government might not have the best certified ScrumMasters it might work best for the contractor to fill that role. The most important challenge to overcome is the management of the product backlog with the requirements. There are two common scenarios we’ve encountered: 1. Thousands of requirements are provided at the time of RFP. 2. No detailed requirements exists for the product backlog. Figure 2. Obsidian’s Agile Development. Our agile development teams produce quality software quickly, and manage changing requirements efficently providing continuous feedback to stakeholders. Agile Development 2 Weeks Daily Standup Sprint Backlog Product Backlog Burndown Chart Potentially Shippable Product User Stories
  6. 6. Agile DevOps Obsidian Proprietary Page 4 Cloud Integration • IT Services • Solutions Architecture With programs where thousands of the requirements are created and included in a RFP, we have found that the requirements are usually created without the domain knowledge or understanding of the system needed to develop systems that accurately represents the desired outcome. In this scenario, we overcome this challenge by providing a Sprint 0 where the entire team does a product backlog grooming exercise. We work as a partner with the government to identify requirements that are valid,requirements that should be discarded, and requirements that need further clarification so they are reflective of what the government wants. In the scenario where no requirements exist, we work with the government in a Sprint 0 to develop a product backlog at the Epic level and develop a product roadmap. Taking this approach provides the government with a high level product roadmap which gives them the ability to forecast for the option years of the contract and is usually done under a single award BPA. Using the product backlog that is developed during Sprint 0, the Product Owner can prioritize the Epics, and then break the epics into user stories and detailed tasks for each sprint. We find that taking the time up-front to develop a good product backlog that is reflective of the government’s desires, and setting up our Agile DevOps environments is key to program success. With a good product backlog, and the proper environments we can develop a good sprint backlog for Sprint 1+, and then conduct development using agile Scrum. We understand that requirements and priorities are going to change, but by establishing a good foundation we can alleviate the majority of the complexity that comes with implementing an agile development contract. Continuous Integration (CI), and Continuous Delivery (CD) Our Continuous Integration (CI) and Continuous Delivery (CD) approach is designed to create an automation environment for the entire end-to-end release process so that every change to the application results in a releasable version that is built automatically. With our approach applications are built from a very early stage in the development process at specific frequent intervals or on every change checked in by the developers. This effectively eliminates the need for integration testing because the code is incrementally being integrated on a daily basis. This removes the cost associated with developers spending time on this phase. The enablement of frequent incremental builds and mandating a comprehensive automated testing process also allows developers to detect problems early and as a result, ensure higher application quality. A typical CI/CD workflow is shown in Figure 3.
  7. 7. Agile DevOps Obsidian Proprietary Page 5 Cloud Integration • IT Services • Solutions Architecture Figure 3. CI/CD Workflow. Our approach eliminates the extra workload from merges and multiple baselines. To support rapid deployment and release we use tools that allow us to automate provisioning of infrastructure resources and platforms. The server configuration, packages installed, relationships with other servers are modeled with code, and is automated and has predictable outcomes, removing error-prone manual steps. Our approach uses software development best practices for the infrastructure code and stores the code in a Code Repository with tags and branches, and releases the code just as if it were applications software. Our infrastructure code is continuously integrated, tested and deployed right along the application software and is treated no differently. We configure our CI server with build steps to check for coding style, coding standards, static analysis, and other features using tools such as SonarQube. We run unit tests from the CI server and deploy the code to the development integration environment and execute additional functional test scripts. We use tools like Selenium for smoke and UI testing. Performance testing is part of our automated testing using tools like JMeter for load testing. Using CI server, we produce artifacts that documents the results of the build, unit testing, and deployment and functional testing along with the source version used for the build. Our automated deployment provides a continuous delivery pipeline that automates deployments to test and production environments. Our approach significantly reduces the manual intensive tasks, resource lag time and errors prone from manual repetition. We provide automated deployment 002 Obsidian, 03/09/2015 • The Continuous Integration server monitorsthe version controlsystem for changes and launchesthe build process. • The Continuous Integration server runs unit tests to validate the quality. • The code isthen run through the code quality tools like SonarQube. • The Continuous Integration server deploysthe same package on the testing environments (created and configured through infrastructure as code toolslike Puppet or Chef) for thorough user acceptance testing. • Once the acceptance tests are successfulthe same package is deployed on the production servers (created and configured through infrastructure as code toolslike Puppet or Chef). Implementing infrastructure and release automation enables end to end automation and allows provisioning and deploying the application packageson all servers with a click of a button. Commit Build Unit Test and Quality Check Deploy Testing Environments Deploy Production • During the development of the application, the developersare working on their local machines and once they are ready to submit their new code changesthey commit them to the central source code repository.
  8. 8. Agile DevOps Obsidian Proprietary Page 6 Cloud Integration • IT Services • Solutions Architecture tools and processes reducing deployment risk, and giving the option of deploying code multiple times per day without any degradation in service. The releases are small which reduces the risk for system instabilities and customer user experience issues. Every change is easier to roll back and easier to test because the number of changes per release is very small. Continuous Monitoring and Continuous Feedback Continuous monitoring across all phases of the application development, testing and deployment is crucial for a successful Agile DevOps implemenation. Our approach lowers the costs of errors and changes by providing continuous feedback throughout each phase of the lifecycle. Obsidian offers a unique approach to monitoring using multiple tools to monitor the applications, environments, and systems. Using tools like Evolven and LMC for environment management provides real-time user tracking and work flows to manage the technical users that develop and operate the application on a daily basis. We use tools like Splunk for log analysis for developers and tools like New Relic to monitor the performance of the applications from the user’s perspective such as page-load times, database-transactions, and systems monitoring to focus on CPU load, memory utilization, and disk space. These tools allow our teams to better understand issues and metrics, and ensures that we are optimizing resources to reduce operational expenditures.
  9. 9. Agile DevOps Obsidian Proprietary Page 7 Cloud Integration • IT Services • Solutions Architecture Measuring Agile DevOps Success A survey from the Vanson Bourne market research agency (with CA) late in 2013 indicated that 39% of those surveyed had adopted some form of DevOps and 27% were planning to do so in the near future. Despite this being such a hot topic in the IT sector with a high level of adoption, the question we are still most commonly asked is: “Where do we start?”. Organization’s must first baseline their current position. Having a baseline means you can build a business case, apply targets and goals to your projects and measure your success as you progress through your project. Agile DevOps Metrics for Baselining and Measuring Success There are hard, quantifiable technical and financial metrics we can measure, such as:  Number and frequency of software releases  Volume of defects  Time/cost per release  Mean Time To Repair (MTTR*)  Number and frequency of outages / performance issues  Revenue/profit impact of outages / performance issues  Number and cost of resources Process Metrics Agile DevOps encompases the entire SDLC that affects both development and operations staff to greater or lesser degrees that need to be taken into consideration. All of these process components can be improved and then optimized as the appropriate automation tools are integrated into the environment. An ultimate goal of an Agile DevOps project is often to attain true continuous delivery (CD) by linking these processes and tools together to allow fully tested, production ready, committed code to proceed to productionwithout impediment. When baselining the current state, it’s useful to measure these component processes and their relative maturity (taking into account use of existing tools and success of implementation). We look at:  Requirements management  Agile development  Build  Release and deployment  Unit testing  User Acceptance Testing  Quality Assurance  Application Performance Monitoring * The MTTR is the Mean Time to Repair, Resolve or Resolution - each of the definitions means the same and can be used interchangeably. This term is more commonly used when talking about Application Performance Management and the speed at which an outage or performance issue can be fixed, but equally can be used when talking about testing and eliminating defects
  10. 10. Agile DevOps Obsidian Proprietary Page 8 Cloud Integration • IT Services • Solutions Architecture Why Ask for an External Agile DevOps Maturity Assessment While no one is going to understand your business better than you do, we often meet organizations who are struggling to find the time to focus on becoming more effecient. They know there are improvements to be made but they are so busy with firefighting they can’t conceive of stopping to baseline and evaluate their current position. It’s easy to stick with “the devil you know”, but that won’t help you with your competitive posture or allow you to use newer technologies to cut costs and improve quality. That’s why it’s important to have an external assessment of your approach to Development and Operations.
  11. 11. Agile DevOps Obsidian Proprietary Page 9 Cloud Integration • IT Services • Solutions Architecture Next Steps Understanding Agile DevOps and its benefits are just the beginning. Obsidian can help facilitate the adoption of Agile DevOps in existing and new programs. Obsidian offers expertise to help cli- ents align business process and technology to their business needs. We help clients develop a strategic roadmap for how Agile DevOps technology may be used to meet mission and business requirements. Our approach helps clients understand the goals and IT strategies within the roadmap, and we execute projects to implement these Agile DevOps strategies. Our Agile DevOps experts assist clients in identifying the drivers, key considerations, and other important factors for migrating and modernizing IT Services. Our experts develop comprehensive IT Transformation plans that provide clear steps to increase the value of technology. Conclusion Our Agile DevOps approach addressess the entire SDLC. With the implemenation of automation tools and repeatable processess, we continuously deliver to production systems without outage windows, and unnecessary manual processes. Our approach reduces costs by providing environments that are automated thus removing the need for staff to spend time with manual processes. We make processes repeatable which allows for more frequent and less error-prone releases. Our vendor-agnostic approach drives increased efficiencies through improved development and operational processes, transparency via continuous monitoring and feedback, improved quality from continuous integration and testing, and less risk due to an automated enviornment. Obsidian has the vision, experience, and technical know how to assist Federal agencies to take full advantage of Agile DevOps. For more information on Obsidian’s Agile DevOps solution, contact David Callner,, 571.425.3031.
  12. 12. Agile DevOps Obsidian Proprietary Page 10 Cloud Integration • IT Services • Solutions Architecture About the Author Mr. Callner is currently the Chief Technology Officer for Obsidian Global, LLC and is responsible for the establishment and oversight of the technical direction of the company. He is charged with ensuring that Obsidian takes a proactive approach to identifying and deploying new technologies and ensuring alignment with our client’s technology and strategies. Mr. Callner received a M.S. in Computer Science from Johns Hopkins University, and his B.S. in Computer Science from University of Texas. About Obsidian Global, LLC Obsidian Global, LLC is a certified Small Business provider of Agile Development, DevOps, Cloud Integration Services, Solution Architecture Services, and IT Services. For more information, visit