Software Risk Management for IT Execs CAST


Published on

Gartner Research Director Thomas Murphy notes that software quality is often a poor misnomer for the current practice of risk management applied by most companies. Many organizations use risk management to mitigate delivery risk, typically at the expense of application quality. Learn about the importance of focusing on application structural quality to reduce business disruption risk in this Gartner-CAST paper.

Published in: Business, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Software Risk Management for IT Execs CAST

  1. 1. S f r R s Ma a e n: ot e ik wa n g me tAP i r o I E e uie rme fr T x c t s v
  2. 2. Software Risk Management: A Primer for IT Executives Controlling the Structural Drivers of Business RiskFeaturing research from
  3. 3. Excerpt from: Predicts 2010: Agile and Cloud Impact Application Development Directions 2 The following section is excerpted from Predicts 2010: Agile and Cloud Predicts 2010: Agile and Cloud Impact Impact Application Development Directions. To view the full note, click Application Development Directions anywhere in the section. 4 Strategic Planning Assumption: Through 2015, the shift toward Software Risk Management cloud architecture will create demand for new skills, practices and objectives for software quality. 6 About CAST Analysis By: Thomas Murphy Key Findings: Software quality is often a poor misnomer for the current practice of risk management applied by most companies when it comes to practices and scheduling in software projects. The focus is not to drive quality, but to mitigate risk. While this is a viable approach, it also goes together with a concept that quality equals the absence of defects. Although this is theoretically true, the application is often too narrow to say that from this, quality software is delivered. The International Organization for Standardization (ISO) produced a standard (9126) that is generally ignored, because quality costs, and often is not seen as providing a return on investment. However, as organizations seek to drive down maintenance costs and adapt to the shorter project life cycles found in agile practices, there is a need to focus efforts on a broader quality definition. In addition, organizations will need to invest in additional tools and skills to deal with increasingly complex distributed applications. Development frameworks may hide some of the complexity of creating these applications, but it won’t help with the testing of applications. We are seeing strong growth now in tools that support a more automated test lab environment. This includes: • Virtual lab management • Virtualization of servicesApplication Structural Quality is the Key to Software Risk Management is published by CAST Inc. Editorial supplied by CAST Inc. is independent of Gartner analysis. All Gartner research is © 2010 by Gartner, Inc. and/orits Affiliates. All rights reserved. All Gartner materials are used with Gartner’s permission and in no way does the use or publication of Gartner research indicate Gartner’s endorsement of Cast’s Software products and/or strategies. Reproduction and distribution of this publication in any form without prior written permission is forbidden. The information contained herein has been obtained from sources believed to be reliable. Gartnerdisclaims all warranties as to the accuracy, completeness or adequacy of such information. Gartner shall have no liability for errors, omissions or inadequacies in the information contained herein or for interpretationsthereof. The reader assumes sole responsibility for the selection of these materials to achieve its intended results. The opinions expressed herein are subject to change without notice. 2
  4. 4. 3• Improved tools for test data management, including subsetting Market Implications: The shift first toward SOA, then to rich Internet and data masking applications has stressed the ability of testing tools to keep up with technology shifts, and for testing teams to keep up with the pace• Integration into life cycle tools to improve traceability and of technology and application changes. The complexity of testing automation of workflow, and to close gaps in the common bugs scenarios requires vendors to also deliver a broader spectrum of that cannot be reproduced in tester/developer interaction tools. This is resulting in a number of new companies and products coming to market, and will also result in increased acquisition activityHowever, these are all just improvements to business as usual. as existing market leaders look to fill out their solutions.While ALM tools provide better accountability to requirements, qualitysoftware has a variety of attributes not directly connected to normal While many organizations will be attracted to the promise of reuserequirements, including: from SOA, success will be limited because of the lack of skills and structure to support reusable assets. Reuse requires a view toward• Understandability governance, ownership and quality.• Completeness Because software quality can’t be tested at the end, organizations will need to look at facilities and practices that drive quality through• Conciseness the development life cycle. This will include using practices from agile, such as TDD, and using tools that drive repeatable processes, such as• Portability continuous integration (CI). This will also create a continued drive for the use of ALM solutions that provide integration across the life cycle.• Consistency A great challenge will be dealing with development that happens• Maintainability outside the traditional IT process. Simplified business process management (BPM) and mashup tools make it easy for business• Testability analysts and end users to quickly assemble new solutions. However, this requires that the underlying components are stable, secure and• Usability scalable. It also requires that organizations are consistent. These requirements will continue to drive the market for static analysis tools• Reliability and service registries and repositories.• Structure Recommendations: • Develop testing practices and expertise in security, scalability and• Efficiency automation.• Security • Drive practices that drive quality from start to finish on a project. This includes shoring up weak requirements practices.The ongoing promise of evolving Web application architectures is todeliver applications and services that are customizable by business • Establish quality career path and standard definitions to setanalysts and end users. Just as many organizations have moved expectations and drive consistency.more than 50% of their “development” budgets into packagedimplementations, we believe that this trend will continue withincreased capabilities for non-developer-targeted development.However, companies that seek to utilize technology to drive business Source: Gartnerinnovation will evolve a more holistic view of software quality,because without it, they will not be able to support the ever-increasingmaintenance burden. 3
  5. 5. Software Risk ManagementThe process by which IT business software is However, as Thomas Murphy points out in managing delivery risk alone only addressesbuilt and the resulting software product itself the excerpt above, software quality is often a part of the problem. It’s like addressing theare to some extent intertwined. It’s tempting erroneously equated with mitigating risk symptoms of a disease rather than takingthen to think that when we have reliable, in “practices and scheduling in software aim at curing its cause. To get to the rootrepeatable processes for building the projects.” There is much more to software causes, we have to define, analyze, andsoftware product, the quality of the resulting risk than that. The main, if not only, reason measure software product quality.product will be equally good. for building and maintaining applications is for the business value they generate. With 2. Three Kinds of Software ProductDespite that temptation, we have all known this in mind, let’s distinguish three kinds of Quality and the Importance of Structuralfirst hand that an application delivered on business risk from software applications. Qualitytime, on budget, and even on scope cannot Let’s begin by distinguishing three basicachieve its business goals if it is slow, 1. Delivery Derailment Risk – risks that add types of software product quality.behaves unpredictably, or compromises IT cost or stop business revenue due toprivacy. Moreover, a poorly built application delayed launch or cancellation. 1. Functional Quality – a measure ofis expensive and slow to respond to what the software does versus what it’sbusiness, further eroding present and 2. Business Case Risk – risks that affect the supposed to do.future business value. Nonetheless, most quality of a delivered application; evendiscussions of managing software risk though the application works, it doesn’t 2. Non-Functional Quality – a measure ofcontinue to equate the quality of the process work as well as it should. The number how well it does it versus how well it’swith the quality of the resulting product. of successful transactions per unit time supposed to do it. cannot be completed to fulfill the benefitsTo truly manage the business risk of articulated in the business case. 3. Structural Quality – a measure of howapplications, we must move beyond the well it will continue to perform as it isquality of the process to the quality of the 3. Business Opportunity Risk – risks that meant to in the future.product itself. The main aim of this article make the application hard to maintainis to distinguish three kinds of software and change in the face of pressing When it comes to the quality of the softwareproduct quality: functional, non-functional, business demand. The resulting loss of product, functional quality alone is notand structural quality and explain why agility damages future business revenue. enough. If all that matters is having the rightstructural quality is essential for managing functionality, then every car that lines up onthe root drivers of IT costs and business Managing delivery derailment risk alone is the NASCAR starting grid would win the race!risks in your mission-critical applications. insufficient for generating business value. But of course, winning the race takes moreStructural quality metrics enables us to Reliable project management processes than satisfying the functional specification – itunderstand, predict, and control the key and the right functionality are nothing if the takes superior performance in the real world.drivers of software costs and business risks. application works unpredictably, is slow, or breaks down often. In addition to on-time, Similarly, non-functional quality is not1. Why Software Quality is Critical on-budget and on-scope delivery, business enough. Non-functional quality focusesSoftware is the backbone of the modern value is generated by the functionality on the visible behavior of the softwareenterprise. Software animates critical working like it should. When the application – the availability and latency of criticalelements of an enterprise’s value chain. is not performing like it should you cannot transactions. While this is important (inThese statements are obvious enough to achieve the benefits articulated in the addition to the software’s usability), thesebe clichés. But current conditions – both business case. performance indicators are skin deep. Totechnological and business – make equate them with product quality would bemodern value chains increasingly difficult to Unlike the quality of the process by which to equate, for example, the destruction leftanimate without incurring large amounts of software is built, enhanced, and maintained, in the wake of an uncontrolled skid with thebusiness risk. functional, non-functional, and structural quality of the suspension system that was the quality have to do with the software product root cause of this destruction. itself – the asset that generates business value. Managing software quality by 4
  6. 6. 5Availability and latency are classic examples Analyzing the quality of modern applications in To truly manage the cost and business risksof “visible” or “above-the-waterline” metrics. the context of the numerous interconnections of your mission critical applications you mustThey are rear-view mirror metrics with little with other code, databases, middleware, and move beyond process metrics to productor no predictive power. They tell you how APIs is monstrously complex. It can only be quality metrics and in particular, measurethe system is doing (symptoms) but not why accomplished with an automated system that the structural quality of applications.things are going well (or badly). analyzes the inner structure of all components and evaluates their interactions in the context of Structural quality metrics are forward lookingOn the other hand, structural quality measures the entire business application. and actionable – they go beyond functionalhow well the application is designed and how and non-functional quality to the root causeswell it is implemented (the quality of the coding Moreover, the component and/or its of application costs and business risks. Theypractices and the degree of compliance with environment changes as a result of technology give you the visibility and control you need tothe best practices of software engineering that upgrades, user needs, and business needs. This manage your mission-critical applications.promote security, reliability, and maintainability. means that any system for measuring productStructural quality metrics track the root causes quality must have both breadth and depth. The multi-tier, multi-language, and multi-of application latency and availability. They platform nature of modern applicationsare forward-looking metrics that enable us • Breadth: Comprehensive coverage of the make automation essential for measuringto control how an application performs, how entire system from end to end. In modern structural quality. No human or team hasreadily it can be enhanced in response to systems, this means it has to cover a sufficient end-to-end visibility of the entireurgent business requests, and how much it will multiplicity of languages, technologies, and system. Moreover, because structuralcost to maintain. frameworks all the way from the GUI front quality is contextual, it requires sophisticated end to the middleware to the database. algorithms to analyze and measure it.Let’s consider what structural quality would befor a house. In this analogy, structural quality • Depth: A detailed architectural/logical In addition to the breadth of technologywould not be about the number of rooms or view of the entire system from end to end. coverage and the sophistication ofthe way in which the rooms are furnished. The quality measurement system must contextual quality analysis, automatedRather, structural quality would be about the be able to create detailed architectural systems for analyzing and measuringengineering design (e.g. where the load- maps of the entire system -- views of all software quality must also be able tobearing walls are placed, the strength and the components and how they are inter- provide detailed information on the rootpliability of materials used) and how well the connected. It must be able to capture the cause of quality problems, and providematerials come together (e.g. the soundness logical aspects of the system, not simply practical guidance on how these root causesof joints, the organization of the electrical and the physical representation of it. It must can be fixed once and for lines). A house with high structural be able to use this detailed logical view toquality is typically easy to maintain and extend. evaluate the product quality of the system When these quality metrics are measured in the context of the entire system. over time, they provide valuable informationBut what do structural quality metrics look on quality trends – actionable informationlike and what does it take to measure them? Conclusion for prioritizing focus areas and allocatingLet’s consider those questions next. Application quality is often equated with the resources for improvement. And these very results of testing or with being able to manage same quality metrics (the change in their3. How to Measure Structural Quality delivery derailment risks. This is dangerous values) serve to measure the effectiveness ofStructural Quality is Contextual because it completely misses the key reason for these improvement efforts. building and operating business applications: Source: CAST Inc.The fundamental challenge of software product the creation of business value.quality is that it is contextual. The quality of asingle component depends on its local andglobal environment. The quality of a singlecomponent cannot be evaluated independentlyfrom its environment.11 Olivier Bonsignour and Bill Curtis, “Why Application Quality Is Different From and More Important Than Code Quality”( 5
  7. 7. About CASTThe CAST Application Intelligence Platform is the only enterprise-gradesoftware quality assessment and performance measurement solutionavailable in the market today. The CAST solution inspects the sourcecode, identifies and tracks quality issues, and provides the data to www.castsoftware.commonitor development performance. CAST Headquarters North America: +1 212-871-8330CAST can read, analyze and semantically understand most kinds Europe: +33 1 46 90 21 00of source code, including scripting and interface languages, 3GLs,4GLs, web and mainframe technologies, across all layers of anapplication (UI, logic and data). By analyzing all tiers of a complexapplication, CAST measures quality and adherence to architecturaland coding standards, while providing visual specificationmodels. Managers get real time access to this information via aweb interface by which they can proactively monitor, measure andimprove application health and development team performance.CAST’s unique technology is the result of more than $80 million inR&D investment. Top engineering talent, dedicated to building thebest technology for assessing the structural quality of mission-critical applications, has made CAST the leader in AutomatedApplication Intelligence. CAST’s mission is to use softwaremeasurement to transform application development into amanagement discipline.Founded in 1990, CAST has helped more than 650 organizationsworldwide speed IT delivery to the business, mitigate risks inproduction, improve customer experience, and reduce the totalcost of application ownership. CAST is listed on NYSE-Euronext(Euronext: CAS) and serves Global 2000 organizations worldwidewith a global network of locations in the US and Europe. 6
  8. 8. L a nmo ea o t A T er r bu C S w w c ss f aec m w .a tot r.o w bo .a tot aec m lgc ss f r.o ww w fc b o .o c so q a t w . e o kc m/a tn u ly a iw w sd s aen t a tot ae w . ie h r.e/ ss f r l c w w w t ie.o O Q a t w . t r m/ n u ly wt c i