Agile ALM Tool Comparison


Published on

Presentation compares all top ALM tools

  • Thanks for this information, really helpful. I believe they left the big ones out because of what is stated on slide 5 '... Info-Tech focused on those vendors that have strong market presence and/or reputational presence among small to mid sized companies.'
    Are you sure you want to  Yes  No
    Your message goes here
  • thank you for meaningfull presentation.
    But what about the 3 companies of the ALM market. HP, IBM and Microsoft. is this some kind of relatively mid-size companies presentation or something?
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Agile ALM Tool Comparison

  1. 1. Vendor Landscape: Agile ALM ALMs for the poor, and for the rich.Info-Tech Research Group 1
  2. 2. IntroductionAs development platforms, coding methodologies, and devices increase innumber, Agile Application Life Management (ALM) tools support integrationswith an ever-increasing range of systems.This Research Is Designed For: This Research Will Help You: Enterprises seeking to select a solution for Agile Understand what’s new in the Agile ALM market. Application Lifecycle Management (ALM) technology Evaluate Agile ALM vendors and products for your enterprise needs. Their Agile ALM use case may include: Determine which products are most appropriate for • Providing testing support for a quality-centric particular use cases and scenarios. development environment. • Supporting Agile development. • Integrating lifecycle management with Integrated Development Environments.Info-Tech Research Group 2
  3. 3. Executive SummaryInfo-Tech evaluated eight competitors in the Agile ALM Info-Tech Insightmarket, including the following notable performers: 1. ALM and BPM start to intermingle:Champions: Premium offerings give the ability to• CollabNet TeamForge, a comprehensive, full-featured ALM tool customize process. Evaluate your need at a reasonable price. Despite its impressive range of offerings, for, and the cost of, process customization CollabNet supports the SMB. before you make your purchase.• Micro Focus Borland ALM, a long-standing leader in the market with a broad feature offering and strong product support. 2. The range of platforms is growing:• Serena Suite, a robust offering backed by strong customer Premium products support development support and a broad reseller network. across the growing range of platforms and development methodologies, includingValue Award: mobile, web, and infrastructure.• TechExcel DevSuite, a comprehensive offering with a price tag far below its comparable competitors. 3. New entrants offer attractive options: Niche offerings can serve your needs at a fraction of the cost of a premium tool. For example, consider specialized tools for Agile and testing integration.Info-Tech Research Group 3
  4. 4. Market Overview How it got here Where it’s going• Long-standing vendors in the ALM space can trace their • Agile ALM has moved into the Cloud. Major vendors of roots to the 1980s, or even earlier. cloud collaboration software have moved into the ALM space and new SaaS-only ALM tools have appeared.• Since the 1990s, ALM tools have played a key part in the project manager’s arsenal, allowing him or her to • Process flexibility has become key. Instead of a strict track project status and progress towards objectives. adherence to Agile or waterfall development, most firms have pursued a middle path and customized• Managers have paid top dollar for ALM suites that could their methodology to meet their own needs. Some track project data and provide meaningful reporting. Agile ALM tools cater to the need for flexibility.• As development environments have evolved, the tools • Quality control has become a core part of ALM. and components being used to manage this process Several major ALM tools are built around testing tools have grown to become integrated and convenient, and process maturity. covering all phases of the development lifecycle including architecture, testing, and deployment with a • Going forward, expect stronger integration between single common interface. ALM tools and the ecosystem of products supporting development, such as testing, PLM tools, and IDEs. As the market evolves, capabilities that were once cutting edge become default, and new functionality becomes differentiating. Traceability has become a table stakes capability and should no longer be used to differentiate solutions. Instead, focus on workflow and reporting to get the best fit for your requirements.Info-Tech Research Group 4
  5. 5. Agile ALM Vendor Landscape selection / knock-out criteria:Market share, mind share, and market consolidation • The Agile ALM space continues to see new entrants, new products, and new features. Ongoing demand for development on mobile and web platforms creates new opportunities for vendors focused on a particular market niche. • For this Vendor Landscape, Info-Tech focused on those vendors that have a strong market presence and/or reputational presence among small to mid-sized enterprises. Included in the Vendor Landscape: • CollabNet TeamForge. A relatively recent entrant with a strong offering and a reasonable price tag. • Micro Focus Borland ALM. A large, ALM-focused vendor that provides strong support for its product. • PTC Integrity. A strong ALM offering tightly integrated into a PLM platform. • Parasoft Concerto. A product targeted towards development shops with a high level of process maturity. • Serena Suite. The complete ALM package controlled by a complete business process engine. • SmarteSoft SmartSuite. ALM tools built around a quality-control platform. • TechExcel DevSuite. A flexible tool that supports development on a variety of platforms. • ThoughtWorks Suite (Mingle, Twist, Go). A strong, Agile-focused product offering that provides some flexibility within the Agile framework.Info-Tech Research Group 5
  6. 6. Agile ALM Criteria & Weighting FactorsProduct Evaluation Features Usability 30% 30% The solution provides basic Features and advanced feature/functionality. The solution’s dashboard and reporting tools Architecture 20% 20% Affordability Usability are intuitive and easy to use. The five year TCO of the solution is Product Affordability economical. 50% The delivery method of the solution aligns with Architecture what is expected within the space.Vendor Evaluation 50% Vendor is profitable, knowledgeable, and will Vendor Viability be around for the long term. Vendor is committed to the space and has a Viability Strategy Strategy future product and portfolio roadmap. 25% 30% Vendor offers global coverage and is able to Reach sell and provide post-sales support. 15% Channel Reach 30% Vendor channel strategy is appropriate and the Channel channels themselves are strong.Info-Tech Research Group 6
  7. 7. The Info-Tech Agile ALM Vendor Landscape Champions receive high scores for most evaluation criteria and offer excellent value. They have a strong market presence and are usually the trend setters for the industry. Innovators have demonstrated innovative product strengths that act as their competitive advantage in appealing to niche segments of the market. Market Pillars are established players with very strong vendor credentials, but with more average product scores. Emerging players are newer vendors who are starting to gain a foothold in the marketplace. They balance product and vendor attributes, though score lower relative to market Champions.For an explanation of how the Info-Tech Vendor Landscape is created please see the slide entitled “Vendor Evaluation Methodology” in the appendix.Info-Tech Research Group 7
  8. 8. Every vendor has its strengths & weaknesses;Pick the one that works best for you Product Vendor Afford- Archi- Overall Features Usability Overall Viability Strategy Reach Channel ability tecture CollabNet PTC Micro Focus Parasoft Serena SmarteSoft TechExcel ThoughtWorks Legend =Exemplary =Good =Adequate =Inadequate =Poor For an explanation of how the Info-Tech Harvey Balls are calculated please see the slide entitled “Vendor Evaluation Methodology” in the appendix.Info-Tech Research Group 8
  9. 9. TechExcel captures the highest value score with acomprehensive, affordable offeringWhat is a Value Score? On a relative basis, TechExcel Champion maintained the highest Info-Tech Value ScoreTM of the vendor group. VendorsThe Value Score indexes each vendor’s product were indexed against TechExcel’soffering and business strength relative to their performance to provide a complete,price point. It does not indicate vendor ranking. relative view of their product offerings.Vendors that score high offer more bang for thebuck (e.g. features, usability, stability, etc.) thanthe average vendor, while the inverse is true forthose that score lower.Price-conscious enterprises may wish to give the 100Value Score more consideration than those whoare more focused on specific vendor/productattributes. 64 Average Score: 31.5 55 22 13 0 0 0 TechExcel CollabNet Serena SmarteSoft Micro *PTC *Parasoft *Thought- Focus Works*Vendor declined to provide pricing. For an explanation of how the Info-Tech Value Index is calculated please see the slide entitled “Value Index Ranking Methodology” in the appendix. For an explanation of how normalized pricing is determined please see the slide entitled “Product Pricing Scenario & Methodology” in the appendix.Info-Tech Research Group 9
  10. 10. Table Stakes represent the minimum standard; without thesea product doesn’t even get reviewedThe Table Stakes What Does This Mean?Feature Description The products assessed in this Vendor LandscapeTM meet, at the very least, theBasic task The product allows the user to create requirements outlined as Table tasks, assign resources to those tasks, and view project status reporting. Many of the vendors go above and beyond theGraphical displays The product displays tasks and other outlined Table Stakes, some even do so in objects as editable, graphical objects. multiple categories. This section aims to highlight the products capabilities in excess of the criteriaTraceability The product ties together objects created listed here. at various stages of the lifecycle, showing how tasks relate to requirements, etc.Links to external The product allows the user to link todocuments external documents for requirements and other content.Central data store The product stores data in a centralized repository, not on local disk. If Table Stakes are all you need from your Agile ALM solution, the only true differentiator for the organization is price. Otherwise, dig deeper to find the best price to value for your needs.Info-Tech Research Group 10
  11. 11. Advanced Features are the market differentiators that make orbreak a productScoring Methodology Advanced FeaturesInfo-Tech scored each vendor’s features offering Feature What We Looked Foras a summation of their individual scores across Requirements Users can enter requirements in athe listed advanced features. Vendors were given Management hierarchical point for each feature the product inherentlyprovided. Some categories were scored on a Change Management The product packages and traces changes.more granular scale with vendors receiving halfpoints. Workflow The product provides automated workflow. Source Code The product manages source code or Management integrates with SCM solutions. Task Management The product manages timelines and tasks. Testing The product provides test management. Defect/Bug Tracking The product manages and traces defects. Resource The product permissions users to edit Management objects and evaluates resource capacity. Reporting & Analytics The product analyzes project progress and provides meaningful reporting. Release Management The product helps with release planning. Content Management The product organizes documents.Info-Tech Research Group 11
  12. 12. Each vendor offers a different feature set; concentrate on whatyou need Evaluated Features Req. Change Work- Task Defect Res. Report- Rel. Content SCM Testing Mgmt Mgmt flow Mgmt Track Mgmt ing Mgmt Mgmt Collab- Net PTC Micro FocusParasoft SerenaSmarte- Soft Tech- ExcelThought- Works Legend =Feature fully present =Feature partially present/pending =Feature absentInfo-Tech Research Group 12
  13. 13. CollabNet offers a strong product with a broad range offeatures and a reasonable price tag Champion Overview • A recent entrant to the ALM space (but in SCM since 1999), Product: TeamForge CollabNet offers the range of functionality normally expected Employees: Unavailable by large organizations, but targets the SMB market with a Headquarters: Brisbane, CA relatively affordable price tag. Website: Founded: 1999 Strengths Presence: Private company • TeamForge provides all of the functionality associated with full- featured ALM, from requirements through to release. • Despite its robust offering and some well-known clients, CollabNet has a large number of SMB clients as well. • CollabNet has strong support offerings, both for its clients and its channel partners. Challenges • CollabNet’s limited revenue and emphasis on North America sales restricts its global reach. However, the company has experienced strong revenue growth. • TeamForge’s native resource management solution lacks $1 $1M+ functionality, although it offers basic capacity management. 3 Year TCO: Tier 8; between $250K and $500KInfo-Tech Recommends: CollabNet offers a very strong product at a reasonable price, supported by a recognized vendor in the source code management space.Info-Tech Research Group 13
  14. 14. Micro Focus provides full-featured ALM with strong customerand reseller support Champion Overview • With 30 years of experience in application management, Micro Product: Borland ALM Focus continues to lead the market with a strong offering. Employees: 1,200 Headquarters: Newbury, UK Website: Founded: 1976 Strengths Presence: LON: MCRO FY11 Revenue: $436M • The Borland ALM suite benefits from Micro Focus’s strong global reach and extensive experience in ALM. • Micro Focus combines large size with a focus on ALM. • Micro Focus offers strong support for its products, including multi-language, 24x7 support. • Micro Focus’s reseller networks stretch across four continents. Challenges • The Borland ALM suite excludes some key functionality such as resource utilization management, assuming that the user will integrate with a project management tool. • Borland ALM does not have burn-down reports or agile task $1 $1M+ management reporting. 3 Year TCO: Tier 8; between $250K and $500KInfo-Tech Recommends: Organizations seeking a premium product and willing to pay a substantial amount should consider Micro Focus.Info-Tech Research Group 14
  15. 15. Serena provides all the ALM tools, packaged within a powerfulprocess orchestration engine Champion Overview • Serena continues to achieve competitive differentiation Product: Serena Suite through its Serena Business Manager, a full-fledged process Employees: 700 management solution that orchestrates development. Headquarters: Redwood City, CA Website: Founded: 1980 Strengths Presence: Private company • Serena includes all of the functionality associated with full- featured ALM, from requirements to release management. • Serena has proven longevity in the ALM space, having entered the market over 30 years ago. • Serena’s support offices and reseller network stretch across four continents. Support is available in ten languages. Challenges • Serena support centers provide only callback service on weekends. $1 $1M+ 3 Year TCO: Tier 8; between $250K and $500KInfo-Tech Recommends: Serena will make sense for organizations needing a high level of process customizability and workflow, and willing to make a substantial investment.Info-Tech Research Group 15
  16. 16. TechExcel offers flexibility and a range of ALM features;support offering is weak Innovator Overview • Despite being a long-standing player in the ALM business, Product: DevSuite TechExcel has only a small share of the market. Employees: 155 Headquarters: Lafayette, CA Website: Founded: 1999 Strengths Presence: Private company • DevSuite includes most of the functionality associated with full- featured ALM, from requirements to task management. • DevSuite offers a high degree of flexibility, making it useful for development on a range of platforms. • TechExcel integrates with a variety of development tools and supports development using diverse methodologies. Challenges • TechExcel offers relatively weak support for a major player, providing telephone support only during business hours. • TechExcel lacks key release management functionality and some reporting and analytics capabilities as well. $1 $1M+ • Despite its longstanding presence in the market, TechExcel 3 Year TCO: Tier 7; between $100K and $250K lacks a strong global revenue footprint.Info-Tech Recommends: Organizations seeking a full-featured ALM solution should consider TechExcel, particularly if they have limited support needs.Info-Tech Research Group 16
  17. 17. Concerto provides a range of functionality and targets high-maturity development shops Market Pillar Overview • A long-standing player in the ALM market, Parasoft has a wide Product: Concerto array of patent holdings that gives it a competitive edge in the Employees: Unavailable marketplace. Headquarters: Monrovia, CA Website: Founded: 1987 Strengths Presence: Private company • Concerto includes most of the functionality associated with full- featured ALM, including requirements and task management. • Concerto can integrate with .NET and Eclipse IDEs. • Concerto allows the user to create custom workflows in UML. • Parasoft has a strong market focus on the ALM space. Challenges • Concerto does not include release management functionality and Parasoft has no intention of adding this. • While Parasoft has clients in the SMB space, Concerto targets a level of process maturity (CMMI 3 or above) that most Info- $1 $1M+ Tech SMBs do not have. Pricing was not made available • Parasoft has a limited footprint compared to larger players.Info-Tech Recommends: Concerto will provide strong support for organizations of any size that require a high level of development process maturity.Info-Tech Research Group 17
  18. 18. PTC Integrity provides a range of ALM offerings within aProject Lifecycle Management context Emerging Player Overview • PTC acquired MKS in 2011 to integrate MKS’s Integrity ALM Product: Integrity offerings into PTC’s suite of Product Lifecycle Management Employees: 5,000 (PLM) tools. Headquarters: Needham, MA Website: Founded: 1985 Strengths Presence: NASDAQ: PMTC FY10 Revenue: $1B • Integrity provides almost all of the functionality associated with full-featured ALM, from requirements to task management. • Integrity supports both Agile and waterfall methodologies, plus the ability to graphically edit processes. • Integrity benefits from strong integration with other lifecycle management tools sold by PTC. Challenges • The target client for the product has at least 25 software developers, making it a difficult proposition for many SMBs. • PTC limits phone support to weekdays, unusual for a vendor of this size. $1 $1M+ • While PTC has a long history in the PLM space, its ability to Pricing was not made available manage the ALM side of the business remains unproven.Info-Tech Recommends: Integrity has a strong offering for larger organizations, or any organization seeking to integrate ALM with a broader PLM suite.Info-Tech Research Group 18
  19. 19. SmarteSoft provides full-featured ALM at a substantial price;support offering is weak Emerging Player Overview • SmarteSoft entered the ALM field in 2005 with its SmarteSuite Product: SmarteSuite of tools. Employees: 50 • The suite centers around SmarteQM, a quality-oriented ALM Headquarters: Austin, TX and testing tool. Website: Founded: 1999 Strengths Presence: Private company • Despite its positioning as a test automation tool, SmarteQM has most of the functionality associated with ALM. • Half of SmarteSoft’s clients are still mid-sized businesses, a relatively strong focus on the SMB client. • Customizable workflow in SmarteQM allows users to adapt to various development platforms and methodologies. Challenges • SmarteSoft offers relatively weak support given the cost of SmarteQM: initial contact is usually via email with a 24-hour turnaround time. • Given its recent entry, SmarteSoft lacks proven longevity in the $1 $1M+ space, a concern given the high price tag. 3 Year TCO: Tier 8; between $250K and $500KInfo-Tech Recommends: SmarteSoft’s credentials in test management software will make this a suitable choice for users with heavy testing and quality management needs.Info-Tech Research Group 19
  20. 20. ThoughtWorks offers a strong offering for Agile developmentbut limited support for waterfall or other methods Emerging Player Overview • A recent entrant into the Agile ALM market, ThoughtWorks Product: Mingle, Twist, Go provides a robust offering but limited support for non-Agile Employees: Unavailable development activity. Headquarters: Chicago, IL Website: Founded: 1993 Strengths Presence: Private company • ThoughtWorks includes most of the functionality associated with full-featured ALM, from requirements to task management. • ThoughtWorks has an organizational focus on Agile ALM and long-standing experience in Agile development. • ThoughtWorks testing is designed for a variety of platforms. Challenges • ThoughtWorks offers email-only support and a one-day turnaround, a relatively weak support offering. • ThoughtWorks has a strong focus on the Agile methodology, limiting the usefulness of the tool for waterfall or other $1 $1M+ methods. Pricing was not made available • As well, integration with .NET is not offered.Info-Tech Recommends: ThoughtWorks can make sense for Agile-oriented organizations, or organizations seeking training and help in moving towards Agile.Info-Tech Research Group 20
  21. 21. Identify leading candidates with the Agile ALM VendorShortlist ToolThe Info-Tech Agile ALM Vendor Shortlist Tool is designed to generate acustomized shortlist of vendors based on your key priorities. This tool offers the ability to modify: • Overall Vendor vs. Product Weightings • Top-level weighting of product vs. vendor criteria • Individual product criteria weightings: Features Usability Affordability Architecture • Individual vendor criteria weightings: Viability Strategy Reach ChannelInfo-Tech Research Group 21
  22. 22. Quality-sensitive development shops require tight integrationwith a top-notch testing toolLook for a vendor that provides a full-featured testing solution as part of theALM offering. Exemplary Performers1 Quality-sensitive development Viable Performers2 Agile focused3 Development integrated Adequate Performers4Info-Tech Research Group 22
  23. 23. Agile-focused environments should seek a tool with strongsupport for AgileSeek an ALM product with native support for Agile processes and artifacts. Exemplary Performers1 Quality-sensitive development Viable Performers2 Agile focused3 Development integrated Adequate Performers4Info-Tech Research Group 23
  24. 24. Organizations requiring development integration should seekflexible integration capabilitiesLook for native integrations as well API-based integrations for niche IDEs. Exemplary Performers1 Quality-sensitive development Viable Performers2 Agile focused3 Development integrated Adequate Performers4Info-Tech Research Group 24
  25. 25. Appendix • Vendor Evaluation Methodology • Value Index Ranking Methodology • Product Pricing Scenario & MethodologyInfo-Tech Research Group 25
  26. 26. Vendor Evaluation MethodologyInfo-Tech Research Group’s Vendor Landscape market evaluations are a part of a larger program of vendor evaluations that includes SolutionSets that provide both Vendor Landscapes and broader Selection Advice.From the domain experience of our analysts, as well as through consultation with our clients, a vendor/product shortlist is established. Productbriefings are requested from each of these vendors, asking for information on the company, products, technology, customers, partners, salesmodels, and pricing.Our analysts then score each vendor and product across a variety of categories, on a scale of 0-10 points. The raw scores for each vendor arethen normalized to the other vendors’ scores to provide a sufficient degree of separation for a meaningful comparison. These scores are thenweighted according to weighting factors that our analysts believe represent the weight that an average client should apply to each criteria. Theweighted scores are then averaged for each of two high level categories: vendor score and product score. A plot of these two resulting scoresis generated to place vendors in one of four categories: Champion, Innovator, Market Pillar, and Emerging Player.For a more granular category by category comparison, analysts convert the individual scores (absolute, non-normalized) for eachvendor/product in each evaluated category to a scale of zero to four whereby exceptional performance receives a score of four and poorperformance receives a score of zero. These scores are represented with “Harvey Balls,” ranging from an open circle for a score of zero to afilled in circle for a score of four. Harvey Ball scores are indicative of absolute performance by category, but are not an exact correlation tooverall performance.Individual scorecards are then sent to the vendors for factual review, and to ensure no information is under embargo. We will make correctionswhere factual errors exist (e.g. pricing, features, technical specifications). We will consider suggestions concerning benefits, functional quality,value, etc; however, these suggestions must be validated by feedback from our customers. We do not accept changes that are notcorroborated by actual client experience or wording changes that are purely part of a vendor’s market messaging or positioning. Any resultingchanges to final scores are then made as needed, before publishing the results to Info-Tech clients.Vendor Landscapes are refreshed every 12 to 24 months, depending upon the dynamics of each individual market.Info-Tech Research Group 26
  27. 27. Value Index Ranking MethodologyInfo-Tech Research Group’s Value Index is part of a larger program of vendor evaluations that includes Solution Sets that provide both VendorLandscapes and broader Selection Advice.The Value Index is an indexed ranking of value per dollar as determined by the raw scores given to each vendor by analysts. To perform thecalculation, Affordability is removed from the Product score and the entire Product category is reweighted to represent the same proportions.The Product and Vendor scores are then summed, and multiplied by the Affordability raw score to come up with Value Score. Vendors arethen indexed to the highest performing vendor by dividing their score into that of the highest scorer, resulting in an indexed ranking with a topscore of 100 assigned to the leading vendor.The Value Index calculation is then repeated on the raw score of each category against Affordability, creating a series of indexes for Features,Usability, Viability, Strategy and Support, with each being indexed against the highest score in that category. The results for each vendor aredisplayed in tandem with the average score in each category to provide an idea of over and under performance.The Value Index, where applicable, is refreshed every 12 to 24 months, depending upon the dynamics of each individual market.Info-Tech Research Group 27
  28. 28. Product Pricing Scenario & MethodologyInfo-Tech Research Group provided each vendor with a common pricing scenario to enable normalized scoring of Affordability, calculation ofValue Index rankings, and identification of the appropriate solution pricing tier as displayed on each vendor scorecard.Vendors were asked to provide list costs for Agile ALM software licensing to address the needs of a reference organization described in thepricing scenario.Additional consulting, deployment, and training services were explicitly out of scope of the pricing request, as was the cost of enhancedsupport options, though vendors were encouraged to highlight any such items included with the base product acquisition. Vendors were askedto prepare a three-year total acquisition cost for their respective Agile ALM solutions. This three-year total acquisition cost is the basis of thesolution pricing tier indicated for each vendor.Finally, the vendors’ three-year total acquisition costs were normalized to produce the Affordability raw scores and calculate Value Indexratings for each solution.Key elements of the common pricing scenario provided to Agile ALM vendors included:• A six-site organization with 2,200 employees, located at two locations in each of the US, England, and France. The US locations create software to be used internally, as well as for external clients, while the development resources in England and France are focused almost entirely on externally facing applications, including mobile solutions.• The development organization has grown rapidly through acquisitions. While the teams work well together, projects are often seen as chaotic when crossing office boundaries.• The corporate development group has determined that implementing ALM consistently across all locations and projects would greatly improve the overall efficiency of the collective global development group.• The corporate development group would like to create visibility for the corporate stakeholders into all projects being executed with real-time (or near real-time) access to data. Dashboards and reports should be able to be filtered by project, by development group, and across the entire corporation.• Development projects are typically run with an Agile approach, following two-week sprints for most projects.Info-Tech Research Group 28