Your SlideShare is downloading. ×
  • Like
Building a DevOps Pipeline
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.


Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Building a DevOps Pipeline


Every IT organization struggles to meet business expectations. Frequently, these expectations are in conflict.The business requires production systems to be highly reliable and available with a strong …

Every IT organization struggles to meet business expectations. Frequently, these expectations are in conflict.The business requires production systems to be highly reliable and available with a strong set of controls in place to ensure that whatever is deployed into the production system is of high quality and has been thoroughly vetted. But the business also expects to deploy new applications and capabilities quickly to adjust to ever-changing market conditions and new ventures. A key component in meeting these goals is establishing the right environments and practices to rapidly create applications of maximum quality. Find out how businesses can overcome these challenges by adopting more automated testing techniques to help create a streamlined continuous delivery pipeline.

Source: IBM Systems Magazine, Mainframe edition, May 2013, Tim Hahn, an IBM Distinguished Engineer.

Published in Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads


Total Views
On SlideShare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. M A Y / J U N E 2 0 1 3 i b m s y s t e m s m a g . c o m / m a i n f r a m e32very IT organizationstruggles to meet businessexpectations. Frequently,these expectations are in conflict.The business requiresproduction systems to be highlyreliable and available with a strongset of controls in place to ensurethat whatever is deployed intothe production system is of highquality and has been thoroughlyvetted. But the business alsoexpects to deploy new applicationsand capabilities quickly toadjust to ever-changing marketconditions and new ventures. Akey component in meeting thesegoals is establishing the rightenvironments and practices torapidly create applications ofmaximum quality.It’s a challenge to build testand development environments,but leveraging these automatedtesting techniques can helpcreate a streamlined continuousdelivery pipeline.Development EnvironmentsDevelopment teams needenvironments where they cancreate, investigate, prototype, tryout, vet and verify the capabilitiesand runtime characteristics oftheir work. These developmentenvironments typically areseparate from the systemswhere production applicationsoperate—and for good reason.As the size and complexityof applications and developmentteams increase, so do the require-ments on their developmentsystems. Multiple copies of thedevelopment and test environ-ments are often needed so teamscan work in relative isolation.While they must be a closeapproximation of the productionplatform, they usually differ tokeep costs down and to allowmore diagnostic capabilities andan opportunity to debug features.From time to time, develop-ment teams must be allowed toinvestigate the usage of new oradditional features offered by theruntime environment.With regard to functionalcapabilities offered and theconfiguration of various middle-ware employed, environmentsmust generally mirror each otherAutomated testing to support applicationdevelopment reaps many rewards By Tim Hahn • Illustration by Larry JostR e p o s t e d w i t h p e r m i s s i o n f r o m I B M S y s t e m s M a g a z i n e, M a i n f r a m e e d i t i o n
  • 2. J U L Y / A U G U S T 2 0 1 0 i b m s y s t e m s m a g . c o m / m a i n f r a m e22 i b m s y s t e m s m a g . c o m / m a i n f r a m e M A Y / J U N E 2 0 1 3 33➜ Automated testing can helpprogrammers and applicationdevelopers get work done morequickly and safely.➜ Many IT shops are too busy to getup to speed with the latestimprovements for tools and practices andsimply don’t know where to begin.➜ Organizations that make the effort tolearn about and implement automatedtesting, continuous integration andvirtualized services reap many benefits.
  • 3. M A Y / J U N E 2 0 1 3 i b m s y s t e m s m a g . c o m / m a i n f r a m e34so that test results will indicate a high probabilityof success in running the applications in theproduction environment. Therefore, devel-opment environments can be a challenge tobuild and maintain. Larger organizationsusually use several throughoutthe application developmentand testing process. These oftenfall under four categories:1Developer-focused environments—often virtual system instances—aretailored to match target environments,encouraged to be transient in nature, andrepeatedly stood-up, reset or reclaimed.2Build-focused and function/integration test environments are usedto run integrated builds or perform spot testing.These can be virtual, are usually longer-livedand contain all the tools necessary to compile,link and package the application. These are alsoeasily reset, re-created or reclaimed.3Performance/stress-focusedenvironments are often scaled-downversions of a production environment, allowingfor accurate extrapolation or estimation ofproduction-level performance. They frequentlycontain performance testing and measurementsoftware to drive workloads, and measurethroughput, latency and resource consumption(CPU, memory, network, I/O, disk, etc.).4Pre-production environments areclose approximations of the productionruntime environments and often contain thesame redundancy, failover and load-balancingcharacteristics. Testing in these environmentsusually includes determining that nounintended side effects or failures exist fromdeploying the application.Types of TestingDifferent forms of testing can validate differentparts of an application. Unit testing, for example,can verify that individual objects, functions andmethods are operating as expected. Testing eachof these items as units often requires scaffoldingcode or data to be wrapped around the object,function or method’s test. Unit test frameworksfor Java* (jUnit) and other languages (zUnitfor COBOL source code) are available to assistdevelopers in this area.A separate team often performs functional testing to ensurea degree of objectivity in both the test cases and invocations.This verifies the functional operation of the application.Automated functional testing usually takes the form of extensiveregression test cases.Integration testing verifies a system’svarious components integrate with oneanother. It’s best done prior to unit andfunctional testing to ensure that componentsintegrate early in the development lifecycle.Performance and stress testing involvessubjecting a deployed application to abattery of tests to determine its throughput,latency and resource consumption. Thisfrequently involves automated techniquesto create workloads for the application toprocess. Performance and stress tests canrun from several minutes to many days ofcontinual operation.Automation in ActionAutomating tests offers many benefits. Forinstance, unit testing can be rerun on aregular basis to verify new problems haven’tbeen introduced. These tests can be usedeven when the developer/owner of thecomponent changes to help the new ownerlearn how the program operates.Because systems change over time,function-test automation is very useful.When enhancements or bug fixes are made toan existing application, automated functionalregression testing checks that the applicationis still working correctly. Automation forperformance and stress testing is importantto generate the necessary load on the systemto gather accurate resource consumption,throughput and latency measurements.Automated testing should be used atall stages of the development lifecycle.By adding it as an extension to the buildprocess, tests become a form of buildverification. Accordingly, this approachis becoming a common practice used topromote software to different levels. As abuilt package is promotedto a new level, it’s deployedand assessed within a testenvironment. If tests aresuccessful, the package canbe promoted to the nextDevelopment: The work tocreate or modify applicationsto add capability, fix a problem,improve performance, etc.Build: The effort to transformsource code into deployablepackages to install and runin a target environment. Thiscan involve compilers, filemodification utilities, textprocessors, etc.Test: The act of validating theoperation of an applicationwithin an environment.Different forms of testinginclude unit, functional,integration, stress andperformance.Integration: The assemblyof various elements or piecesof an application in order totest or run them together.It typically collects builtartifacts from various placesor teams and prepares themfor deployment onto a targetenvironment.Promotion: Marking anapplication under developmentas ready for the next levelof testing, such as a useracceptance test or deploymentinto production environments.Deployment: The actof installing applicationcomponents and configuringthem into an environment.
  • 4. J U L Y / A U G U S T 2 0 1 0 i b m s y s t e m s m a g . c o m / m a i n f r a m e22level. If problems arise, it can be rejectedwith appropriate work items opened forfurther investigation.Continuous testing uses all of thesetechniques, particularly test automation,to move toward a constant awareness ofthe quality of the application under development.Conducted on built packages,automated tests provide reportsto development and test teams,alerting them quickly to defectsand regressions.Delivery PipelineBeyond testing, automation can beleveraged to create a continuousdelivery pipeline, similar inconstruct to an assembly line.A continuous delivery pipelinecan produce production-qualityapplications. On the input sideof the pipeline are businessrequirements implemented byapplication development teams.Automated build processing isemployed to take new capabilitiesand build them into packagesthat can be deployed into testenvironments. Automated buildverification performs unit andfunctional validation.As packages make their wayalong the delivery pipeline, they’redeployed into the appropriatefunctional and performance testenvironments where additionalautomated processes are executed.Again, if the result of testing issuccessful, the package is promotedto the next level, like moving downthe factory assembly line wheremultiple quality-assurance tests areperformed on the manufactureditems. At the conclusion of eachstage in the process, virtual testenvironments can be reclaimed,allowing physical resources tobe used for another build or testenvironment.The resulting pipeline, shownin Figure 1 (page 36), replacesthe manual and error-prone steps normally taken to build,obtain, deploy and run tests on applications at various stagesof the development lifecycle. The pipeline is constructedusing team expertise gained from years of experience.This speeds up development and improves the reliability,repeatability and accuracy of applications deployed into thevarious environments.
  • 5. Shift infrastructure management costs to spend more timemeeting business requirements Capture human expertise into repeatable, automated actionsbased on best practicesStreamlined application delivery is critical to meetthe conflicting business goals of having highly availableand reliable production systems as well as delivering quickturnaround on new business requirements.Through a collaboration of system programming staffand application development teams, a delivery pipelineenables automated build and testing and continuousintegration for software development. This can speed updelivery of features while increasing reliability and stability ofapplications deployed into production environments.M A Y / J U N E 2 0 1 3 i b m s y s t e m s m a g . c o m / m a i n f r a m e36Interface SpecificationsAs applications and solutions growin size, it becomes increasinglyimportant to vet and deploy separatepieces of an application or solutionwithout affecting the rest of thepieces. This prevents so-called “bigbang” migrations or cut-overs thatare historically fraught with risks.Interface specifications providean agreement between differentparts of a solution on how they’regoing to communicate. They alsoallow for parallel development ofeach of the pieces of the overallapplication or solution. Havingthese specifications allows forautomated testing to be employed atthese interface points.A consumer of an interfacecan be assessed using a test tool that emulates the providerof that interface. Alternatively, a provider can be testedby a tool that generates test messages or invocations of theinterface, acting as a consumer.Defining and using interface specifications is a bestpractice when developing complex applications. Doingso has many benefits, including parallel applicationdevelopment, test automation and runtime observation ofbehavior at the interface points.Working TogetherCreated through collaboration between application develop-ment teams and IT operations teams, a delivery pipeline thatincludes automated testing allows an organization to: Improve the speed and repeatability of application buildsand deployments Provide quicker feedback of test results to test anddevelopment teams Lower the cost of implementing new application capabilities Continuous Integration for System Integrated Solution for System z DevOps blog on Rational Test Hahn is an IBM Distinguished Engineer and the chiefarchitect for enterprise modernization tools within the IBMRational Software organization.Automating the Continuous Delivery PipelineEnhancement DefectsCRsWork ItemsBuildLoad TestFunctional DevelopmentBuild & Test AutomationEnvironmentImagesCode Base& ReleasesQATestedPre-PRODApplication Developers, Testers, System ProgrammersPre Prod✔RD&T x86 LPARDevelopmentChangeChangeTesting✔Validate