The Forrester Wave: Application Lifecycle Management Report


Published on

The Forrester Wave: Application Lifecycle Management Report

Published in: Technology, Business
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

The Forrester Wave: Application Lifecycle Management Report

  1. 1. Forrester Research, Inc., 60 Acorn park Drive, cambridge, mA 02140 UsATel: +1 617.613.6000 | Fax: +1 617.613.5000 | www.forrester.comThe Forrester Wave™: Application Life-Cycle Management, Q4 2012by Tom Grant, ph.D., October 23, 2012FOR: ApplicationDevelopment& DeliveryprofessionalsKey TaKeaWaysLeaders Build product strategy around aLM Use CasesForrester Wave evaluation Leaders IBM, Rally Software, PTC, CollabNet, Microsoft,and Serena Software offer strong solutions for ALM problems that no singlepoint solution could address. Their product strategies focus on such challenges asdelivery and the disruptions of Agile adoption, not just throwing new capabilities atthe customer.strong performers and Contenders Find a specific audienceAtlassian and Rocket Aldon don’t focus as much on specific use cases that spanmany ALM activities. Instead, they build tools to address smaller but widespreadproblems, such as release management and developer collaboration. Meanwhile,HP is expanding its ALM offering beyond Quality Center.aLM is Going Through a period of Redefinition and innovationHot ALM issues such as DevOps, embedded software, and requirements definitionshow how the concept of ALM has expanded and changed. During this transition,the seemingly crowded ALM market will benefit from the innovation that comesfrom a wealth of competing and complementary product offerings.
  2. 2. © 2012, Forrester Research, Inc. All rights reserved. Unauthorized reproduction is strictly prohibited. Information is based on best availableresources. Opinions reflect judgment at the time and are subject to change. Forrester®, Technographics®, Forrester Wave, RoleView, TechRadar,and Total Economic Impact are trademarks of Forrester Research, Inc. All other trademarks are the property of their respective companies. Topurchase reprints of this document, please email For additional information, go to Application Development & Delivery ProfessionalsWhy Read This ReportIn Forrester’s 116-criteria evaluation of application life-cycle management (ALM) vendors, we identifiedthe nine most significant software providers in the category — Atlassian, CollabNet, HP, IBM, Microsoft,PTC, Rally Software, Rocket Aldon, and Serena Software — and researched, analyzed, and scored them.This report details our findings about how well each vendor fulfills our criteria and where they stand inrelation to each other to help application development and delivery professionals select the right partnerfor their software-fueled business innovation efforts.Table Of ContentsInnovation Drives Interest In Application Life-Cycle ManagementApplication Life-Cycle ManagementEvaluation OverviewThe ALM Forrester Wave Finds ManyLeaders And Different DirectionsVendor ProfilesSupplemental MaterialNotes & ResourcesForrester conducted lab-based evaluationsin September and October 2011 andinterviewed nine vendor and user companies:Atlassian, CollabNet, HP, IBM, Microsoft,PTC, Rally Software, Rocket Aldon, andSerena Software.Related Research DocumentsThe Time Is Right For ALM 2.0+October 19, 2010The Forrester Wave™: Agile DevelopmentManagement Tools, Q2 2010May 5, 2010The Forrester Wave™: Application Life-CycleManagement, Q4 2012IBM, Rally Software, PTC, CollabNet, Microsoft, And Serena SoftwareLead The Pack, With HP, Atlassian, And Rocket Aldon Followingby Tom Grant, Ph.D.with Kyle McNabb and Alissa Anderson26101417October 23, 2012
  3. 3. For Application Development & Delivery ProfessionalsThe Forrester Wave™: Application Life-Cycle Management, Q4 2012 2© 2012, Forrester Research, Inc. Reproduction Prohibited October 23, 2012INNOVATION DRIVEs INTEREST IN APPLICATION LIFE-CYCLE MANAGEMENTBusiness innovation now drives the application life-cycle management (ALM) market. Thecontribution that application development and delivery makes to a company’s business goals —makingworkers more productive, creating engaging customer experiences, and bringing new softwareproducts to market more successfully — must be more direct and successful. Software increasinglyplays a central role in a firm’s ability to deliver new products and services or exploit new channels.Firms can no longer accept historical gulfs between business and application development and deliveryteams as, increasingly, firms now expect to manage application development and delivery as a businessand treat it as a competency.1Unfortunately, everyone has their long list of reasons why they struggle with better managingapplication development and delivery. The work is creative, and teams must figure out how to workwith one another. Tools or techniques that impose an assembly-line model will fail. Applicationdevelopment and delivery still need management, but the tools of management must fit the natureof the creative and collaborative work.Software and application development work itself is changing. This Forrester Wave™ evaluationexamines the most significant developments that affect both ALM strategy and ALM tools. Forexample, the definition of ALM has stretched to include not just development but also delivery (seeFigure 1). Some of the vendors we assess have expanded their tools portfolio to add or enhance theirsupport for release management and other delivery-related activities.The need to support the business management of application development and delivery underliesthis Forrester Wave’s evaluation criteria:■ Delivering business value, not just driving process efficiency. ALM’s original aim wasprocess improvement. Firms turned to ALM to do a better job of managing project timelines,dependencies, and deliverables; ensure that every requirement could be traced to a realsoftware capability; and test to ensure that the software worked. While process improvement isstill important, increasing efficiency in building and delivering software is as important a goalfor ALM as proving the value of the software, as it contributes to business outcomes.■ Supporting individual, team, and institutional management. ALM’s first wave of toolsfocused on — in truth — application development management (ADM), designed to increasedeveloper and tester productivity. Now, firms seek ALM tools to improve team success whilelinking these teams to the larger application development and delivery activity within theorganization or, in many cases (such as offshore partners), outside of it.■ Extending life cycles to include delivery. Organizations now recognize that the life cycleof ALM does not stop with check-in or build phases. The real life cycle and process goes onmuch longer, arguably all the way to the moment the application goes live and even beyond tosupport continuous feedback.
  4. 4. For Application Development & Delivery ProfessionalsThe Forrester Wave™: Application Life-Cycle Management, Q4 2012 3© 2012, Forrester Research, Inc. Reproduction Prohibited October 23, 2012■ Managing software that gets deployed everywhere. Software is truly everywhere, from datacenter servers to laptops, from mobile devices to cars and refrigerators. Regardless of wherethe software winds up, software development and delivery management is still necessary.■ Integrating across tools. Vendors have reluctantly admitted that customers are not interestedin their suite of tools, no matter how impressive or cost effective they might be. Switchingcosts can be high, and many capabilities, such as issue tracking and source control, are socommoditized that switching makes little sense. Some tools, such as HP Quality Center, are sowidely adopted that other vendors must integrate with them in some fashion. The speed of toolinnovation in some areas, such as continuous delivery and requirements, outpaces what anysingle vendor can keep up with, providing another reason for integrating across tools.■ Using reporting and dashboards as instruments of change. Many application developmentand delivery organizations aspire to be “learning organizations” and see their investmentin ALM tools in that light. The data in an ALM system provides a yardstick for a team’scontinuous improvement efforts. Some teams use this information to communicatewith outside parties about the shape and value of new Agile and Lean practices. Largerorganizations may use the same information to track software value streams across projectsand teams. In these and other scenarios, reporting and dashboards provide the ultimate payofffor ALM investments.Figure 1 ALM Boundaries Expand To Encompass The Software Value ChainSource: Forrester Research, Inc.60080Production planningclosed loopChange management Service managementPortfolio managementChange-awarecontinuous integrationJust-in-timedemandmanagementReleasemanagementTesting andquality assuranceBuild andsoftwareconfigurationmanagementDeploymentProjectmanagementProduction controlclosed loop
  5. 5. For Application Development & Delivery ProfessionalsThe Forrester Wave™: Application Life-Cycle Management, Q4 2012 4© 2012, Forrester Research, Inc. Reproduction Prohibited October 23, 2012ALM Vendors Have Taken Many Steps Forward, But A Long Road RemainsALM tools have made significant progress since our last evaluation:■ ALM vendors have discovered macro differentiators. In past years, ALM vendors did far lessin their product strategy and product marketing to clearly differentiate their products. Withnotable exceptions such as Rally Software, a company that built its ALM strategy around Agilesupport, ALM vendors had noticeable differences only if you carefully examined the details oftheir product features. Today, ALM vendors more clearly identify the types of customers theyserve and the problems they can help these customers address. Serena Software, for example,tells a very clear story about the challenges orchestrating development and delivery in large ITorganizations and how its products help organizations address those challenges.■ Vendors help improve the efficiency of the larger ALM process. As noted earlier, theboundaries of ALM have expanded to include more of the delivery phase. In response,ALM vendors have expanded their capabilities in this area to remove the inefficiencies thatarbitrarily lengthen and complicate the delivery phase.■ Requirements, a part of ALM, need innovation attention. Recent innovations inrequirements tools, such as visualization and social media integration, have increased thedepth and accuracy of insights and the speed of collecting them. These innovations haveoccurred in requirements-focused companies such as Balsamiq Studios, Blueprint SoftwareSystems, iRise, and Visure Solutions. Even though ALM vendors provide requirements tools,they have not kept pace with these innovations.■ Mobile and embedded development scenarios get insufficient attention. Embeddedsoftware developers still get scant attention, with the exception of MKS (PTC) and IBM.Surprisingly, given how hot the mobile development market is, ALM vendors by and large donothing special for mobile developers, including in areas such as testing where there is a largeneed for better tools.■ Collaboration is the lightning that ALM vendors can’t capture in a bottle. Many of thebiggest ALM challenges, such as coordinating the work across geographically distributed teams,boil down to collaboration problems. Collaboration presents some extremely vexing puzzles fortools vendors, particularly given how idiosyncratically a specific set of people may collaboratewith each other. Still, some basic collaboration features — desktop sharing for team members indifferent locations and wikis for capturing and updating information, for example — are far lesscommon than expected.
  6. 6. For Application Development & Delivery ProfessionalsThe Forrester Wave™: Application Life-Cycle Management, Q4 2012 5© 2012, Forrester Research, Inc. Reproduction Prohibited October 23, 2012The Market Is Still Open For Innovation And New ContendersThe vendors we assessed in this evaluation provide an impressive array of tools with a broad rangeof capabilities. Nevertheless, they do not represent every possible innovation in the expanding andevolving realm of ALM. The ALM market is still wide open for anyone to devise a better way toaddress a problem, identify new and unmet challenges, or give a particular class of customers moreattention than the bigger, general-purpose vendors can.Vendors listed in this section were not evaluated but represent various innovative approaches toALM, demonstrating how much room remains within this market for innovation and specialization:■ Blueprint focuses on requirements. Some aspects of ALM pose tough challenges that don’thave an easy solution. Smaller vendors focused exclusively on some of these challenges caninnovate in ways that larger suite vendors may not. Blueprint, a requirements tools vendor,provides a good example of the important role that these smaller companies with a muchnarrower focus play in today’s ALM market.■ Hansoft caters to video game developers. Since the video game market is nearly as large as themovie industry, it should be no surprise that at least one vendor, Hansoft, has identified thismarket as its core constituency.■ Polarion tackles difficult compliance challenges. Few of the vendors we evaluated have takensteps to address complex compliance requirements, leaving no small amount of opportunity fora smaller vendor such as Polarion Software.■ SmartBear Software makes quality the starting point. While many ALM vendors are shiftingtheir attention to later parts of the development and delivery process, SmartBear Softwaremakes those stages the starting point for its product strategy. This strategy includes both codequality improvement through review and testing and quality of service maintained through webmonitoring.■ Tasktop is the Switzerland of integration. Each ALM vendor has its own set of integrationswith other tools, both open source and commercial. Each ALM vendor also has its ownapproach to integration. Unsnarling this tangle sometimes requires going to a neutral thirdparty, such as Tasktop, for cross-ALM integration.■ ThoughtWorks provides thought leadership in continuous delivery. A real thought leaderin continuous delivery, ThoughtWorks provides a modest range of tools but a very establishedconsulting business (primarily aimed at helping with Agile adoption).■ VersionOne supports Agile. Rally has built Agile and Lean support into its tools, andother vendors such as Microsoft and IBM provide Agile-specific configurations. With Agile
  7. 7. For Application Development & Delivery ProfessionalsThe Forrester Wave™: Application Life-Cycle Management, Q4 2012 6© 2012, Forrester Research, Inc. Reproduction Prohibited October 23, 2012continuing to spread, there is still room for another Agile-focused ALM vendor such asVersionOne to meet the needs of Agile teams in a different way than its competitors.APPLICATION LIFE-CYCLE MANAGEMENT EVALUATION OVERVIEWTo assess the state of the application life-cycle management (ALM) market and see how the vendorsstack up against each other, Forrester evaluated the strengths and weaknesses of nine top vendors.These companies fell into two broad categories:■ Large software companies that, among other products, also offer ALM tools. In this category,HP, IBM, and Microsoft maintain a strong position in the ALM market.■ Smaller software companies that specialize in ALM. Other companies included in thisevaluation, such as Atlassian, CollabNet, Rally Software, Rocket Aldon, and Serena Software,have made ALM their exclusive business. Until its recent acquisition by PTC, MKS also fell intothis group.Evaluation Criteria Measured Breadth Of Portfolio And Support For Use CasesAfter examining past research, user need assessments, and vendor and expert interviews, wedeveloped a comprehensive set of evaluation criteria. We evaluated vendors against 116 criteria,which we grouped into three high-level buckets:■ Current offering. ALM covers a vast expanse of functionality for developers, business analysts,testers, project managers, and people in a wide array of other roles. Therefore, we developedmore than 86 distinct criteria to measure the strength of each vendor’s tools.■ Strategy. The value that a vendor brings to the table goes beyond just its current tools, so weincluded 18 criteria that assessed each vendor’s ability to support customer implementations, itscommitment to the ALM market, its ALM product strategy, and its corporate strategy.■ Market presence. We also took into account the size of the footprint each vendor has in thesoftware market. Forrester’s clients want to know how safe a bet it will be that a given ALMvendor will be in business several years from now in order to gauge how safely they can invest inthat vendor’s ALM tools.This year’s ALM evaluation represents a departure from the 2010 Forrester Wave evaluating Agiledevelopment management solutions in three ways:2■ Opening the aperture to encompass all ALM scenarios. The 2010 evaluation focusedexclusively on Agile support among ALM vendors. This year, we include both Agile and non-Agile use cases in the criteria.
  8. 8. For Application Development & Delivery ProfessionalsThe Forrester Wave™: Application Life-Cycle Management, Q4 2012 7© 2012, Forrester Research, Inc. Reproduction Prohibited October 23, 2012■ Treating Agile and Lean as a critical test of ALM offerings. At the same time, this year’sevaluation by no means leaves Agile behind. Instead, we treat Agile and Lean as critical tests ofan ALM vendor’s offering.■ Incorporating systems engineering and embedded software. Given the increasing quantityof hybrid products — those incorporating software into physical products ranging from carsto medical devices — we have also included criteria covering software engineering outside ofcorporate IT and independent software vendors (ISVs).Evaluated Vendors Have Strong Tools And Substantial Market PresenceForrester included nine vendors in the assessment: Atlassian, CollabNet, HP, IBM, Microsoft, PTC,Rally Software, Rocket Aldon, and Serena Software. Each of these vendors has (see Figure 2):■ Depth and breadth of ALM offering. While there are many commercial and open source pointsolutions for particular activities (test management, build management, issue tracking, etc.),the title “ALM vendor” goes to companies that do more than provide a single capability (sourcecontrol, defect tracking, etc.), however good it might be. We expect these tools to cover themajority of ALM use cases, as Forrester defines ALM.■ A broad ALM customer base. We also looked at the number of customers for a vendor’s ALMproducts in 2011 (at least 750) as well as the growth in the vendor’s customer base from 2010 to2011.■ Definitive revenue from ALM. Finally, the criteria included the revenue from ALM. Onceagain, we included both the numbers for 2011 (at least $19 million) and the growth since 2010.
  9. 9. For Application Development & Delivery ProfessionalsThe Forrester Wave™: Application Life-Cycle Management, Q4 2012 8© 2012, Forrester Research, Inc. Reproduction Prohibited October 23, 2012Figure 2 Evaluated Vendors: Product Information And Selection CriteriaSource: Forrester Research, Inc.VendorAtlassianCollabNetHPIBMMicrosoftPTCProduct evaluatedAtlassian OnDemand:JIRAGreenHopperBonfireConfluenceSharePoint Connector for ConfluenceTeam Calendars for ConfluenceBambooFishEyeCrucibleCloverCrowdBitbucketAtlassian IDE Connector for EclipseAtlassian IDE Connector for IntelliJ IDEAAtlassian IDE Connector for Visual Studio2008 and 2010HipChatStashSourcetreeJIRA Mobile ConnectJIRA OnDemandGreenHopper OnDemandBonfire OnDemandConfluence OnDemandTeam Calendars for ConfluenceOnDemandSource OnDemand Bundle (FishEye,Crucible, and Subversion)TeamForgeScrumWorks ProSubversion EdgeLab ManagementHP Application Lifecycle Management (HPALM) 11.5Rational Collaborative LifecycleManagement Solution (Rational CLM):Rational Requirements Composer 4.0Rational Team Concert 4.0Rational Quality Manager 4.0Microsoft Visual StudioIntegrityProduct versionevaluated5. builds3.5 or later7 and laterN/AWeekly builds1.1Weekly buildsN/AWeekly buildsWeekly buildsWeekly buildsWeekly buildsWeekly buildsWeekly builds6. dateJune 2012Weekly buildsWeekly buildsWeekly buildsWeekly buildsWeekly buildsWeekly buildsWeekly buildsWeekly buildsWeekly buildsJune 2012May 2012April 2012May 2012June 2012June 2012May 2010June 2012
  10. 10. For Application Development & Delivery ProfessionalsThe Forrester Wave™: Application Life-Cycle Management, Q4 2012 9© 2012, Forrester Research, Inc. Reproduction Prohibited October 23, 2012Figure 2 Evaluated Vendors: Product Information And Selection Criteria (Cont.)Source: Forrester Research, Inc.VendorRally SoftwareRocket AldonSerenaSoftwareProduct evaluatedRally Community EditionRally Enterprise EditionRally Unlimited EditionRally Quality ManagerRally Support ManagerRally Product ManagerRally Community ManagerRally Time TrackerRally Idea ManagerRally Portfolio ManagerAldon Lifecycle ManagerAldon Deployment ManagerAldon Community ManagerAldon Report ManagerAldon Agile Manager 1.0Development ManagerRelease ManagerRequirements ManagerService ManagerDemand ManagerRequest CenterDashboardChangeMan ZMFAgile PlannerProduct versionevaluatedWeekly builds(e) 6.1/(i) 7.6N/A9. dateWeekly buildsMay 2011N/ANovember 2010June 2011January 2011May 2012Vendor selection criteriaThe vendor provides designated ALM product(s) and a stated ALM road map.The vendor has more than 750 customers.The vendor provides tools that cover the majority of ALM use cases, as Forrester defines ALM.The vendor generates more than $19 million in revenue.
  11. 11. For Application Development & Delivery ProfessionalsThe Forrester Wave™: Application Life-Cycle Management, Q4 2012 10© 2012, Forrester Research, Inc. Reproduction Prohibited October 23, 2012THE ALM FORRESTER WAVE Finds Many Leaders And Different DirectionsThe vendors we evaluated for the ALM Forrester Wave provide a wide range of capabilities.Superficially, they look very much alike. Project management? Check. Issue tracking? Check. Withsuch feature breadth, you might conclude that ALM is a mature market.But where are the vendors heading? The answer is, in very different directions. Software may beeverywhere, but the form of software development and delivery varies widely across organizations.While this market now has many genuine leaders, they have reached this point only by identifyingtheir core market. Therefore, the market is less mature than it might seem on the surface. Whenevaluating ALM vendors, it’s important to consider not just the capabilities they provide but also theuse cases for which they are optimized.Nowhere is this differentiation more clear than in the overarching principles that guide theevaluated vendors’ product strategies (see Figure 3):■ Leaders IBM and Microsoft provide broad ALM capabilities. Both IBM and Microsoft havemaintained a strong portfolio of ALM tools designed to support common use cases for softwaredevelopment and delivery. The two companies both provide broad swaths of capabilities forcommon ALM activities, such as project definition and management, reporting, issue tracking,and task assignment.IBM and Microsoft come from a more general-purpose perspective on ALM, and bothtreat Agile practices as a special instance of ALM workflows, metadata, and reporting. Bothcompanies support Agile as a template, a special configuration of their ALM tools.■ Leaders PTC and Rally Software lead by focusing on specific use cases. In contrast to IBMand Microsoft, both Rally and PTC (originally MKS, before PTC’s acquisition of the company in2011) have succeeded in the ALM market by finding a niche in which to specialize — for Rally,Agile, and for PTC, compliance. While this approach may limit their ability to support a broaderswath of scenarios, their focus gives them the opportunity to provide thought leadership in theirrespective areas. Rally’s focus on Agile and Lean, for example, has led the company to buildtop-notch planning capabilities into its tools. PTC’s focus on compliance, including the highlyregulated world of embedded software, has made it the only ALM vendor to provide out of thebox many of the workflows and reports needed to satisfy regulatory authorities.■ Leaders CollabNet and Serena lead through delivering differentiated and broad capabilities.Both CollabNet and Serena have tailored their ALM offerings to more-general valuepropositions and needs. CollabNet sees its opportunity in the cloud, where ALM tools supporthas been lagging behind IT’s need to manage cloud development, testing, and deployment moreeffectively. Serena has recognized how important it is for organizations to effectively orchestratethe larger “meta-process” of software development and delivery, which spans the typical
  12. 12. For Application Development & Delivery ProfessionalsThe Forrester Wave™: Application Life-Cycle Management, Q4 2012 11© 2012, Forrester Research, Inc. Reproduction Prohibited October 23, 2012activities (coding, testing, building, etc.) that guide the development efforts of other commercialand open source tools.■ Strong Performers Atlassian and HP provide focused capabilities. Atlassian and HP showhow far an ALM vendor can go with strong support for a piece of the overall life cycle. Howevercapable individual tools such as JIRA and Quality Center may be, they are no substitute fora broader ALM offering. The two companies view this gap differently. Rather than defocusits product strategy, Atlassian prefers to provide a smaller range of strong capabilities thanpotentially do a mediocre job tackling a wider range of ALM challenges. This strategy makes iteasier for Atlassian to deal with one of its biggest challenges, perception as an enterprise-readyvendor, without the added complexity of too many products and features.In contrast, HP has pushed hard to fill in the gaps in its ALM offering, moving far past QualityCenter. The current version, ALM 11.5, provides a very robust set of tools that can compete withother ALM suites on a wide range of capabilities. Some of these new capabilities, such as theexecutive dashboard features, demonstrate that HP understands many tough ALM challengesand will invest in providing credible, differentiating features to address them. However, thesetools are relatively new, and Forrester has not encountered many implementations of them yet.The next year will be critical for HP’s ALM offering in gaining market traction.■ Contender Rocket Aldon is the operational dark horse of ALM. While most other ALMvendors strive to move down the value chain to the point of release and deployment, RocketAldon’s product strategy starts with these later stages of the application life cycle. Automationplays a prominent role in Rocket Aldon’s offerings, as do IT-governance-related features. TheRocket Aldon road map is, to some extent, extending upstream to cover scenarios (for example,product owner review of the backlog) that it currently does not address as thoroughly asproduction-related activities.This evaluation of the ALM market is intended to provide a starting point for your own evaluation.We encourage readers to view the detailed product evaluations and adapt the criteria weightings tofit their individual needs through the Forrester Wave Excel-based vendor comparison tool.
  13. 13. For Application Development & Delivery ProfessionalsThe Forrester Wave™: Application Life-Cycle Management, Q4 2012 12© 2012, Forrester Research, Inc. Reproduction Prohibited October 23, 2012Figure 3 Forrester Wave™: Application Life Cycle Management Tools, Q4 2012Source: Forrester Research, Inc.Go online to downloadthe Forrester Wave toolfor more detailed productevaluations, featurecomparisons, andcustomizable rankings.RiskyBets Contenders LeadersStrongPerformersStrategyWeak StrongCurrentofferingWeakStrongMarket presenceAtlassian CollabNetHPIBMPTCRallyRocket AldonSerenaMicrosoft
  14. 14. For Application Development & Delivery ProfessionalsThe Forrester Wave™: Application Life-Cycle Management, Q4 2012 13© 2012, Forrester Research, Inc. Reproduction Prohibited October 23, 2012Figure 3 Forrester Wave™: Application Life Cycle Management Tools, Q4 2012 (Cont.)Source: Forrester Research, Inc.CURRENT OFFERINGPlatform supportInstalling the product andgetting goingAdministrationSecurityIntegration and customizationManagementChange elementsLife-cycle traceabilityReportingAnalyticsRunning a projectGeneral collaboration capabilitiesProcess configurationSTRATEGYSupport for implementationsProduct strategyCorporate strategyPriceCommitmentMARKET PRESENCEInstalled baseFinancial strengthEmployeesSupport servicesChannel partnershipsGlobal presenceAtlassian3.473.404.804.002.204.803.952.473.632.751.384.754.702.242.922.502.904.500.005.003.943.304.’sWeighting50%5%5%2%3%10%5%10%15%5%10%10%10%10%50%15%80%5%0%0%0%35%5%0%35%20%5%CollabNet3.514.104.404.752.604.903.893.743.263.702.284.302.902.363.954.903.705. scores are based on a scale of 0 (weak) to 5 (strong).ALM Vendors Have Yet To Fully Address Many Common ChallengesWhile many aspects of the ALM tools market have improved substantially in the past two yearssince the Agile development management Forrester Wave evaluation, other areas remain ripe forinnovation and improvement:
  15. 15. For Application Development & Delivery ProfessionalsThe Forrester Wave™: Application Life-Cycle Management, Q4 2012 14© 2012, Forrester Research, Inc. Reproduction Prohibited October 23, 2012■ Testing. With one or two notable exceptions, the vendors we included in this study did not excelat any aspect of testing support. To some extent, the weakness of testing features is characteristicof the ALM tools market, not just the vendors we assessed. However, it’s still worth noting thatsome specialty vendors have done a better job of connecting testing to other ALM activity.Some requirements tools vendors, for example, provide rapid test generation from requirements,something that many purveyors of bigger ALM suites have yet to implement.■ Collaboration. While we have seen some progress in this area, that’s damning by faint praiseconsidering how often collaboration is at the core of an organization’s ALM challenges. Take,for example, the very common challenge of collaborating with an offshore partner. While somefeatures, such as feeds or wikis, make it easier for onshore and offshore teams to eavesdrop oneach other’s activities, these are not vehicles for communicating product vision, mentoringpeople on the other team, or suggesting ideas for process improvement.■ Mobile support. At the risk of stating the obvious, many aspects of mobile development — userexperience, testing, deployment, etc. — are significantly different from the elements involvedin building middleware. Yet the typical ALM vendor today has done very little to address theseneeds, despite the continued expansion of the mobile market.■ Integration. ALM vendors have acknowledged the inevitability of integration with othercommercial and open source tools. However, we’re a long way from arriving at a commonintegration strategy that makes it easier for customers to select ALM tools based on theirindividual merits outside of integration. Some vendors provide an application programminginterface (API) with few or no prebuilt connectors. Others provide plenty of connectors, withperhaps a weaker API for custom-built integrations. The fact that small companies, such asKovair and Tasktop, have established themselves as third-party providers of ALM connectors isa testament to the weaknesses of the overall ALM market’s integration efforts.VENDOR PROFILESLeaders■ IBM’s offering is becoming more than the sum of its parts. IBM’s position in this year’sassessment of the ALM market is a testament to an important principle: It’s not enough justto have good products — the products, in combination, must support common activitiesin software development and delivery. In previous years, IBM has certainly provided strongsolutions, such as Rational RequisitePro, Rational Team Concert, and Rational DOORS inspecific functional areas such as requirements, planning, and project management. However,IBM provided multiple solutions in some of these areas, leading to confusion about what exactlyeach of these tools was supporting and how they all worked together.
  16. 16. For Application Development & Delivery ProfessionalsThe Forrester Wave™: Application Life-Cycle Management, Q4 2012 15© 2012, Forrester Research, Inc. Reproduction Prohibited October 23, 2012That confusion has largely dissipated with IBM’s current ALM offering. Not only has IBMcontinued development of its strong suite of products, but it has also stitched them together ina more coherent way. The company has also made clearer the use cases it supports with its tools,such as Agile teams and embedded software development. Because of the company’s strongproduct and portfolio strategy, it is possible to write about IBM’s ALM offerings “taken as awhole” with far less irony.■ Rally Software continues its leadership in Agile/Lean. Rally targets a very healthy andgrowing opportunity: the expanding number of organizations that have adopted Agile andLean. Rally’s tools are optimized for Agile planning, project management, status reporting, andother actions that happen within and outside sprints. The company’s acquisition of AgileZen,a Lean project management tool, was a natural fit for both Rally and its customers. So too wasthe addition of Rally Portfolio Manager, a tool for planning, decision-making, and managementabove the level of an individual project or product.Rally also continues to provide thought leadership in the Agile and Lean community. However,this focus on Agile has a downside: non-Agile teams will find Rally’s products and services farless attractive than other general-purpose ALM tools. Rally’s leadership rests with its breadth anddepth of capabilities for Agile teams, combined with a strong and focused corporate strategy.■ PTC continues its leadership addressing regulated and “digital products” needs. PTCprovides strength where product-based development and compliance intersect. No othercompany, with the possible exception of IBM, has done as much to support teams of embeddedsoftware developers and systems engineers. PTC helps firms address “digital product”development needs — for products such as appliances, vehicles, and medical devices that havesoftware components — while still maintaining support for teams building and deliveringsoftware-only products and projects.PTC’s strength rests on its long-standing support for addressing governance and compliancechallenges. Burrowing into this niche has its concomitant cost: a more complex productthat is harder to justify adopting for teams that don’t have to satisfy the needs of auditors orgovernment regulators.■ CollabNet provides a flexible ALM framework optimized for the cloud. As already noted,one of the strengths of CollabNet’s ALM suite is the flexibility with which users can defineALM content. We also see this flexibility pay off when trying to effectively address the needs ofcomplex teams or complex projects, where there is less chance for a one-size-fits-all design.CollabNet has also invested heavily in supporting cloud development and deployment. Notonly are CollabNet’s tools available in the cloud (CloudForge), but they also include featuresoptimized for developing, testing, and deploying cloud systems.
  17. 17. For Application Development & Delivery ProfessionalsThe Forrester Wave™: Application Life-Cycle Management, Q4 2012 16© 2012, Forrester Research, Inc. Reproduction Prohibited October 23, 2012■ Microsoft shortens cycle times. Starting with Visual Studio at the front end and through toTeam Foundation Server (TFS) at the back end, Microsoft’s ALM portfolio provides broadlife-cycle support. Microsoft’s overarching goal is to shorten cycle times, which requires acombination of individual measures (task management, test automation, etc.).Microsoft deserves recognition for understanding how central collaboration is to ALM. Whilethe “M word,” management, is part of the ALM acronym, ALM initiatives frequently fail becausethey emphasize control over collaboration. Features such as visualization via PowerPoint andtranslating recorded manual testing into automated tests emphasize collaboration — in thesetwo examples, with business users.■ Serena Software orchestrates IT projects from idea to deployment. Since retooling its ALMstrategy, Serena has made impressive advances at both the individual tool and the suite level.Central to its product strategy is “orchestration,” a vision of ALM as an ongoing, rhythmicactivity. Not surprisingly, given this vision of a regular flow of activity (conceive, build, test,deploy), Serena has one of the best workflow designers available. Serena’s recent enhancementsto its ALM portfolio, such as the Serena Release Manager, provide more support in this end-to-end orchestration model.Strong Performers■ For HP, quality is truly central, but other ALM capabilities are growing. HP Quality Center isthe fulcrum on which HP’s ALM strategy rests. HP Quality Center is a ubiquitous part of manyorganizations’ ALM infrastructure. Consequently, other ALM vendors integrate with HP QualityCenter as a matter of course, almost as a price of entry into any potential customer’s shortlist.HP has improved upon and expanded its ALM capabilities beyond test management and testautomation in the two years since the previous Forrester Wave evaluation of this space. HPnow provides improved requirements management and traceability. Another example, HP’sApplication Lifecycle Intelligence (ALI), offers a broad set of ALM data aggregation andanalysis features, with strong reporting and dashboarding components. Given the importanceof this information for everything from team-level continuous improvement to executive-level portfolio management, it represents one of many areas where HP has moved quickly todistinguish itself as a serious ALM player.■ Atlassian focuses on issue tracking and collaboration within teams. Atlassian’s JIRA, a widelypopular issue tracking tool, often serves double duty as a requirements management system.The company’s Confluence Wiki is equally popular with development teams. The two products,in tandem, support the various kinds of day-to-day work done within software developmentteams, which are always looking for ways to better collaborate (for example, through a wiki).
  18. 18. For Application Development & Delivery ProfessionalsThe Forrester Wave™: Application Life-Cycle Management, Q4 2012 17© 2012, Forrester Research, Inc. Reproduction Prohibited October 23, 2012While Atlassian has decided to provide a narrower set of ALM tools than other vendors, itcontinues to hone the capabilities in the areas where the company believes it can excel, such asproviding a marketplace for its developer community to provide plug-ins. The company hasmade moves into more enterprise-level scenarios, with new capabilities such as Stash, a tool forthe management of Git repositories that enterprise customers demand. Atlassian has also madeenhancements to its support, sales force, and other aspects of its business needed to gain groundin the enterprise market.Contenders■ Rocket Aldon provides strong operational support. Unlike other ALM vendors, RocketAldon’s roots lie in software delivery, not software development. Rocket Aldon has focused fromthe beginning on the operational side, including both automation and reporting. Unlike othervendors, Rocket Aldon is expanding backward, not forward, in the software timeline, addingsupport for more upstream ALM activities, such as demand management and requirements.Supplemental MATERIALOnline ResourceThe online version of Figure 3 is an Excel-based vendor comparison tool that provides detailedproduct evaluations and customizable rankings.Data Sources Used In This Forrester WaveForrester used a combination of three data sources to assess the strengths and weaknesses of eachsolution:■ Hands-on lab evaluations. Vendors spent one day with a team of analysts who performed ahands-on evaluation of the product using a scenario-based testing methodology. We evaluatedeach product using the same scenario(s), creating a level playing field by evaluating everyproduct on the same criteria.■ Vendor surveys. Forrester surveyed vendors on their capabilities as they relate to the evaluationcriteria. Once we analyzed the completed vendor surveys, we conducted vendor calls to clarifyand confirm their products’ capabilities.■ Customer reference calls. To validate product and vendor qualifications, Forrester alsoconducted reference calls with two of each vendor’s current customers.
  19. 19. For Application Development & Delivery ProfessionalsThe Forrester Wave™: Application Life-Cycle Management, Q4 2012 18© 2012, Forrester Research, Inc. Reproduction Prohibited October 23, 2012The Forrester Wave MethodologyWe conduct primary research to develop a list of vendors that meet our criteria to be evaluatedin this market. From that initial pool of vendors, we then narrow our final list. We choose thesevendors based on: 1) product fit; 2) customer success; and 3) Forrester client demand. We eliminatevendors that have limited customer references and products that don’t fit the scope of our evaluation.After examining past research, user need assessments, and vendor and expert interviews, we developthe initial evaluation criteria. To evaluate the vendors and their products against our set of criteria, wegather details of product qualifications through a combination of lab evaluations, questionnaires,demos, and/or discussions with client references. We send evaluations to the vendors for their review,and we adjust the evaluations to provide the most accurate view of vendor offerings and strategies.We set default weightings to reflect our analysis of the needs of large user companies — and/orother scenarios as outlined in the Forrester Wave document — and then score the vendors basedon a clearly defined scale. These default weightings are intended only as a starting point, and weencourage readers to adapt the weightings to fit their individual needs through the Excel-basedtool. The final scores generate the graphical depiction of the market based on current offering,strategy, and market presence. Forrester intends to update vendor evaluations regularly as productcapabilities and vendor strategies evolve.Endnotes1 Forrester’s research indicates that the dependence on software to achieve business outcomes will onlycontinue through the next decade. Therefore, while packaged applications will continue to be important,software development and delivery — for building new applications, customizing packaged systems, orperforming custom integrations — will be critical for bridging the gap between software capabilitiesand business needs. See the January 30, 2012, “BT 2020: To Thrive In The Empowered Era, You’ll NeedSoftware, Software Everywhere” report.2 The previous Forrester Wave contained many similar questions but focused more narrowly on Agile teamsupport. See the May 5, 2010, “The Forrester Wave™: Agile Development Management Tools, Q2 2010”report.
  20. 20. Forrester Research, Inc. (Nasdaq: FORR) is an independent research company that provides pragmatic and forward-thinking advice toglobal leaders in business and technology. Forrester works with professionals in 17 key roles at major companies providing proprietaryresearch, customer insight, consulting, events, and peer-to-peer executive programs. For more than 29 years, Forrester has been makingIT, marketing, and technology industry leaders successful every day. For more information, visit 60080«Forrester Focuses OnApplication Development & Delivery ProfessionalsResponsible for leading development and delivery of applicationsthat support your company’s business strategies, you also choosetechnology and architecture while managing people, skills, practices,and organization to maximize value. Forrester’s subject-matter expertiseand deep understanding of your role will help you create forward-thinkingstrategies; weigh opportunity against risk; justify decisions; and optimizeyour individual, team, and corporate performance.Andrea Davies, client persona representing Market Insights ProfessionalsAbout ForresterA global research and advisory firm, Forrester inspires leaders,informs better decisions, and helps the world’s top companies turnthe complexity of change into business advantage. Our research-based insight and objective advice enable IT professionals tolead more successfully within IT and extend their impact beyondthe traditional IT organization. Tailored to your individual role, ourresources allow you to focus on important business issues —margin, speed, growth — first, technology second.for more informationTo find out how Forrester Research can help you be successful every day, pleasecontact the office nearest you, or visit us at For a complete listof worldwide locations, visit supportFor information on hard-copy or electronic reprints, please contact Client Supportat +1 866.367.7378, +1 617.613.5730, or We offerquantity discounts and special pricing for academic and nonprofit institutions.