Your SlideShare is downloading. ×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

120908 Cast Forrester Webinar Final

1,177
views

Published on


0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,177
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
32
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Managing Risk and Quality In Today’s Economy Margo Visitacion Vice President Forrester Research
  • 2. Software Quality Assurance is as much about business risk as it is about software performance 3 Entire contents © 2008 Forrester Research, Inc. All rights reserved.
  • 3. Agenda • Why quality, why now • Why should business drive QA? • Dynamic QA – processes for the 21st century • Today’s quality assurance organization • How to prepare for adaptation and adoption 4 Entire contents © 2008 Forrester Research, Inc. All rights reserved.
  • 4. Why Quality, Why Now? • Poor software quality costs over $60B per year • Finding, repairing defects = approximately 35% of project budget • Developers generally find only about 50% of their own bugs • Typical testing only finds 75% of potential defects 5 Entire contents © 2008 Forrester Research, Inc. All rights reserved.
  • 5. Poor internal quality is a major contributor to high maintenance spending “Approximately what percent of your software budget will go to new initiatives and projects versus ongoing operations and maintenance?” 2008 33% 67% 2007* 33% 67% Base: 680 North American and European enterprises *Base: 451 North American and European enterprises Source: Enterprise And SMB Software Survey, North America And Europe, Q3 2007 *Source: Forrester Business Technographics September 2006 North American and European Enterprise Software Survey 6 Entire contents © 2008 Forrester Research, Inc. All rights reserved.
  • 6. Why Quality, Why Now? •Customer tolerance for defects lower each year – quality can affect spending in the long term •IT spending forecasts are being recast – and it’s not a pretty picture – absent quality can trend toward budget cuts in the wrong places 7 Entire contents © 2008 Forrester Research, Inc. All rights reserved.
  • 7. Why Should Business Drive QA? Drivers: Business domain expertise trumps all Reduce frustration from IT “not getting it right” Meeting shifting market objectives – lower tolerance for waste Reduced bandwidth compromises success 8 Entire contents © 2008 Forrester Research, Inc. All rights reserved.
  • 8. Business Drivers Require Greater Examination of Risk in QA Practices ►Internal quality: The ►External quality: The way an application is way an application constructed behaves ►Will the application ►Are we getting expected perform as required? outputs? ►Have we done everything ►Is the application usable? to prevent security leaks? ►Are we as defect-free as ►Have we considered possible? liability? 9 Entire contents © 2008 Forrester Research, Inc. All rights reserved.
  • 9. Traditional QA Practices From Quality Assurance Versus Quality Control, December 2004 10 contents © 2008 Entire Forrester Research, Inc. All rights reserved.
  • 10. Agile Practices Take Development to the 21st Century 1 month 11 contents © 2008 Entire Forrester Research, Inc. All rights reserved.
  • 11. Making Quality More Adaptable - Dynamic QA Post Mortem UAT Business determines objectives, sets Integrated requirements Quality Processes Continuous Builds Collaborative Application Design and Determination Test Development Quality Criteria QA advises BAs & DEV on testing criteria QA determines and quality “testability” thresholds 12 contents © 2008 Entire Forrester Research, Inc. All rights reserved.
  • 12. Planning, Strategies and Resource Allocations • Quality planning is more than determining what you’re going to test – Business determines highest risks and values – Include How, Why and Value – Metrics, acceptance and performance criteria determined at kick off – Analysis criteria for application lifecycle • Strategies extend to resources – Leverage internal IP – Make best use of outsourcing – Earlier analysis and inspections to bake quality in and optimize resource usage 13 contents © 2008 Entire Forrester Research, Inc. All rights reserved.
  • 13. Key Actions in Dynamic QA • Test management – Test planning/strategies – Analysis and quality design – Resource planning and allocation – Prioritization – Risk adjusted testing • Visibility – Collaboration – Defect and change management – Expectations management 14 contents © 2008 Entire Forrester Research, Inc. All rights reserved.
  • 14. Prioritization and Risk Adjusted Quality Processes Taking a portfolio view of quality • Align emphasis with business objectives, technical complexity • Prioritize based upon objectives, resources and risks • Ensure that processes include internal and quality • Metrics must show coverage, business acceptance, value to stakeholders Include risk assessments in making decisions about quality Requirement Tech Imp Bus Imp • Risk measures must be prioritized business exposure, liability, complexity XXXX M H ability to deliver XXXX H L • Business must sign off on risk levels at XXXX L L the test planning stage and validate at the M M XXXX acceptance stage • Test planning must be based upon risks to business and ability to support 15 contents © 2008 Entire Forrester Research, Inc. All rights reserved.
  • 15. Visibility and Collaboration • Role appropriate dashboards • Code complexity • Adherence to standards • Universally understood metrics • Defect removal efficiency • Standard measures • Mean time to detect • Relative contextual information • Mean time to repair • Easy to access 16 Entire contents © 2008 Forrester Research, Inc. All rights reserved. • Stakeholder quality
  • 16. Investing in Quality Can Reduce Risk, Cost and Raise Value Average Best in Class • Defect Potential – 1.00 • Defect Potential – 0.40 (Requirements) (Requirements) • Average Defect Removal • Average Defect Removal Efficiency – 85% Efficiency – 96% • Delivered Defects - .75 • Delivered Defects - .13 (per FP) (per FP) • ROI - > $15 for every $1 spent Capers Jones 2008 17 contents © 2008 Entire Forrester Research, Inc. All rights reserved.
  • 17. Investing in Quality Can Reduce Risk, Cost and Raise Value 120 100 80 60 Cost to Repair 40 20 0 Requirements Code Operations 18 contents © 2008 Entire Forrester Research, Inc. All rights reserved.
  • 18. Today’s QA Organization Characteristics Description More Mature – Development Organization or Operations/ Location Service Delivery To CIO or Head of Development Org. Peer to Reporting Development Mgmt Increased Emphasis on Strategic Orientation – varied Focus technical requirements Management Shifting Skill Sets – Vendor, Relationship Mgmt Leadership and IP – internal Make – up Testing resources –combination internal/outsourced Increased technical expertise – testing requirements Testers expanded 19 contents © 2008 Entire Forrester Research, Inc. All rights reserved.
  • 19. Keys to Adoption • Universal language • Automate the process to – Glossaries open doors – Process – Eliminate barriers wherever possible – Hand-offs – Test management tools are – Internal/External quality key • Business context – Leverage integration with – Risk other tools to support collaboration – Testability • Educate the executives – Usability – What’s in it for them – Performance – Include metrics 20 contents © 2008 Entire Forrester Research, Inc. All rights reserved.
  • 20. Recommendations • Make risk assessments part of test planning AND execution • Encourage business to determine more than just requirements • Empower QA to be active part of PROJECT lifecycle • Raise the bar on QA career path to support risk and business driven testing • Use tools to enable, not hide critical information 21 contents © 2008 Entire Forrester Research, Inc. All rights reserved.
  • 21. Thank you Margo Visitacion +1 856-334-8522 mvisitacion@forrester.com http://www.forrester.com/rb/analyst/margo_visitacion www.forrester.com 22 contents © 2008 Entire Forrester Research, Inc. All rights reserved.
  • 22. Achieve Insight. Deliver Excellence. CAST Application Intelligence Platform December 2008
  • 23. CAST Application Intelligence Platform Transparency! Automated.  Automated analysis of entire applications  Immediate, unbiased quality assessment  Executive level of synthesis & trending  Drill down to root cause in the source code Manage Risk at Less Cost
  • 24. Analyzing the entire business application Enterprise Applications Middleware Web / Client Server Applications  ASP/JSP/VB/.NET Presentation Tier Web Services Application Logic  Java, C++, …  Frameworks Struts MVC, Spring Business Logic CICS Tier Legacy Applications Connector CICS Monitor (Cobol) Tuxedo Monitor (C) Data Management Layer  EJB – Hibernate - Ibatis Batch COBOL Data Shell Scripts Tier Database Storage Group Files Databases Database Database Only CAST can analyze this
  • 25. Deep structural analysis of software quality Quality Metrics Subset Quality Indicators Health Factors Application Quality SQL Complexity Distribution Class complexity (Inh. depth) Performance Complexity Over 800+ architectural and language-specific code checks Class complexity (Inh. width) Artifacts having recursive calls Immediate Impact Method complexity (control flow) Robustness Multiple artifacts inserting data on the same SQL table Architecture Coupling Distribution Security File conformity Dead code Application Structuredness Quality Programming Controled data access Practices Empty code Modularity Encapsulation conformity Naming Inheritance Conventions Package naming Class naming Transferability Interface naming Package comment Documentation Class comment On-Going Impact Changeability Method comment Package size Class size (methods) Size Maintainability Interface size
  • 26. Profile, assess, and benchmark applications and teams
  • 27. Major global telecommunications company  230,000 employees, $100 billion revenue, 40 million accounts  Billing & OSS Solutions  120 billion call records and 1 billion invoices per year  Also, SAP, Siebel, all front end apps that power e-commerce sites  Running CAST one or two times per quarterly release  Penalties in contracts based on CAST  Aggregation of CAST metrics into C-level management dashboards CAST Dashboard Project #1 100% 90% • 150+ applications 80% 70% Project #2 60% 50% 40% • 4 apps silos, with 50+ CAST consumers 30% Project #3 20% 10% 0% 1.1 2.0 2.5 Project #n plus management in each Team #1 Vendor A Neutral & independent vendor unit is running the CAST AI Center for the customer Team #2 Vendor B Parser Agent Team #3 Vendor C Vendor D facility Team #4 Vendor D
  • 28. Insights for both buyers and vendors Solution Information What IT constituents need Management Division CIO and VP, Apps Delivery visibility • Ensure teams are working efficiently Overall team and • Manage stability, security & project risks application KPIs • Better relationships with outsourcers Measure of conformance Project Managers, Architects and Quality Assurance to standards & • Ensure architectural compliance architecture • Ensure projects are not at risk • Metrics – quality, quantity, technical Identify specific application quality issues Guidance for Internal and Outsourced Teams developer • Immediate feedback regarding code quality Identify code-level style and quality issues Java developers .NET developers DBAs CAST AIP
  • 29. Q&A