Ensure Application Quality with Vendor Management Vigilance


Published on

Copy and paste the link to access the full recording: http://www.castsoftware.com/news-events/event/vendor-management-forrester?gad=ss
Hear Stephanie Moore, Vice President and Principal Analyst at Forrester, discuss a major challenge that 48% of IT executives say they face with existing outsourcing relationships – poor quality. Expanding on findings in the July 2011 report “Maintaining Vendor Management Vigilance in the Overheated Global Sourcing Market”, Stephanie will talk about how quality is at the heart of vendor management---whether in the processes and standards used by your vendors, the staff assigned to your projects, or the application code itself.

Published in: Business, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Ensure Application Quality with Vendor Management Vigilance

  1. 1. Stephanie Moore Vice President and Principal Analyst Forrester Research, Inc Ensure Application Quality with Vendor Management Vigilance
  2. 2. CAST Confidential 2 Speakers Stephanie Moore Vice President and Principal Analyst Forrester Research, Inc Lev Lesokhin Vice President, Worldwide Marketing CAST
  3. 3. © 2011 Forrester Research, Inc. Reproduction Prohibited3 © 2009 Forrester Research, Inc. Reproduction Prohibited Transparency as an Antidote to Global Sourcing Risk
  4. 4. © 2011 Forrester Research, Inc. Reproduction Prohibited4 Technology is more critical to the business than ever before; it’s a market maker, not just an enabler. Bottom Line: IT has never been so important to the business, and yet due to years of cost cutting and “optimization”, so ill- equipped to support business needs 64% believe: “Technology is CENTRAL to how we differentiate ourselves” 69% believe: “Technology is a CORE COMPONENT of our products or services” 87% believe: “Technology is CRITICAL to serve and support our customers”
  5. 5. © 2011 Forrester Research, Inc. Reproduction Prohibited5 Software is everywhere and growing more complex Product Lines of code Order entry system 1.7 million F-22 Raptor 1.7 million Space Shuttle 2 million Microsoft Word 2 million (27,000 in first release) F-35 Joint Strike Fighter 5.7 million Airline reservation system 6.1 million Boeing 787 Dreamliner 6.5 million S Class Mercedes-Benz radio with navigation system 20 million
  6. 6. © 2011 Forrester Research, Inc. Reproduction Prohibited6 65% The business says says: “I buy technology and technology services without involving IT.”
  7. 7. © 2011 Forrester Research, Inc. Reproduction Prohibited7 “I download applications to do my job because my company doesn’t provide them.” 34%
  8. 8. © 2011 Forrester Research, Inc. Reproduction Prohibited8 The gap between business requirements and IT’s ability to deliver is widening How are IT and the business filling this the gap? In many cases, through third party IT service providers
  9. 9. © 2011 Forrester Research, Inc. Reproduction Prohibited9 © 2009 Forrester Research, Inc. Reproduction Prohibited Understanding the trends in global outsourcing
  10. 10. © 2011 Forrester Research, Inc. Reproduction Prohibited10 IT Industry Perspective: Key Trends Today  Outsourcing is viewed as an essential tool for companies to reduce costs, improve service and help the business innovate  The large scale, top tier IT services and outsourcing providers begin to look the same – Whether you work with Accenture, IBM, Infosys or TCS, much of your applications work will be performed offshore  While offshore outsourcing continues to grow, satisfaction levels decrease  Managed outcome services are recognized as essential to optimizing outsourcing relationships –from a cost and quality perspective. But, lack of transparency into code quality and structure is a challenge for many customers  Domestic outsourcing emerges due to a productivity-price trade off. Sometimes it is faster and cheaper to outsource locally due to the need for contextual, not just technology, understanding.
  11. 11. © 2011 Forrester Research, Inc. Reproduction Prohibited11 Global outsourcing: India is the destination of choice  Market is mature and large, cost is low and quality is high  However, quality is on the decline (C-Player and “fresher” syndrome)  Escalating costs are narrowing gap relative to US costs  Wage inflation, COLAs  Cost of gaining requisite contextual knowledge  Rework often required once code is repatriated to US  High employee turnover due to constrained labor market  Higher onsite component required for communications and context  Increased difficulty and expense in landing offshore resources onsite using H1B and other visas  Communications challenges  Time zone challenges  Difficulty in performing early lifecycle work where business, process and vertical knowledge is required
  12. 12. © 2011 Forrester Research, Inc. Reproduction Prohibited12 © 2009 Forrester Research, Inc. Reproduction Prohibited Managing your vendors for success
  13. 13. © 2011 Forrester Research, Inc. Reproduction Prohibited13 © 2011 Forrester Research, Inc. Reproduction Prohibited 17% 40%41% 48% 0% 10% 20% 30% 40% 50% 60% Quality of work done is poor 2007 2008 2009 2010 Customers’ outsourcing frustration is rising Which are your firm's biggest challenges with your existing outsourcing relationships?
  14. 14. POLL QUESTION 1 What types of business risks do you most closely associate with product quality? Increased cost 28% Slower time to market 22% Loss of revenue 12% Loss of customers 18% Damage to company brand 20%
  15. 15. © 2011 Forrester Research, Inc. Reproduction Prohibited15 So, vigilance required in managing global outsourcing vendors  Quality standards and certifications: are they real and are they real for your activity?  Resource qualifications and certifications – How proficient are the teams you engage? – Do they have necessary certifications? – Do they really know how to code software?  Continuous evaluation of vendors’ HR and recruiting and retention capability  Continuous evaluation of vendors’ training capability  Innovation capability: show me your vertical and domain expertise
  16. 16. © 2011 Forrester Research, Inc. Reproduction Prohibited16 The importance of vendor training investments
  17. 17. © 2011 Forrester Research, Inc. Reproduction Prohibited17 Vigilance required in managing global outsourcing vendors (cont.)  Productivity vs. price trade offs – How to measure  Evaluating deliverables from managed service engagements – Is the code maintainable gong forward? – Is it efficiently built?  Vendor viability assessments a must – This is much more difficult for small and privately held vendors – Clients have to be clever in researching their vendors and their vendors’ performance and track record. – Employment websites – Social media
  18. 18. POLL QUESTION 2 Do your outsourcing/customer SLAs include metrics on the technical/structural quality of code being delivered? Yes 19% No 37% Sometimes 25% I don't know 19%
  19. 19. © 2011 Forrester Research, Inc. Reproduction Prohibited19 © 2009 Forrester Research, Inc. Reproduction Prohibited Managed Outcomes
  20. 20. © 2011 Forrester Research, Inc. Reproduction Prohibited20 Manged services model: benefits and risks Client buys people Client buys an outcome directly And manages them to deliver an outcome Staff Augmentation Model Managed Services Model
  21. 21. © 2011 Forrester Research, Inc. Reproduction Prohibited21 Why is a managed services model better?  Better savings including a productivity boost beyond labor cost savings – Potentially a one-time benefit, however according to case history examples, savings can range from 10% to 25%  Movement of risk from client to vendor  Greater potential for process improvement – Allows your IT services vendor to operate at its own level of process maturity  Improved foundation for knowledge retention – Knowledge does not walk out the door with the “person”  Improved knowledge transfer in terms of time and money expenditure – Vendor is responsible for its own staffing  But, how do you ensure your black box is filled with good things?
  22. 22. © 2011 Forrester Research, Inc. Reproduction Prohibited22 Keys to Success for Managed Services  A mechanism for auditing code quality and its adherence to standards. – This transparency is critical to both the client and the vendor  Change in the way suppliers are managed – Managing vendors instead of programmers – Having an effective governance model which includes vendor participation – Ability to review vendor performance with links to penalties or rewards grounded in the Master Services Agreements (MSA) and Service Level Agreements (SLAs)  Change in the way work is organized for outsourcing. – A portfolio-driven view of potential work to be outsourced and the ability to batch work where possible  Ability to envision service level requirements and outcomes and document them in statements of work
  23. 23. © 2011 Forrester Research, Inc. Reproduction Prohibited23 Transitioning to managed outcome relationships requires an SLA “starter set”  Start with SLAs that are simple to track: – Number of incidents, enhancements, patches – Availability – Quality of support/responsiveness and resolution – First time right  Allow for a “true-up” with the supplier in a mutually agreeable time frame. – Allow time to build up historical data patterns.  Create a path forward to more demanding SLAs for the future – it seems like magic. – Quality improvements – Productivity improvements – “Innovation”
  24. 24. Because you can’t manage what you don’t measure May 2012 Actionable Application Intelligence for IT Executives
  25. 25. CAST Confidential Market leader in Software Analysis & Measurement 25 Ambitious Mission Rock Solid Foundation Market Leader Introduce fact-based transparency into application development and sourcing to transform it into a management discipline  Broad market presence in Europe, North America and India  Strongly endorsed by software industry gurus and long term investors  Over $100 million of investment in R&D, driven by top talent in computer science and software engineering  Pioneer and recognized market leader since 1999  CAST Research Labs, the world’s largest R&D facility dedicated to the science of software analysis & measurement (SAM) “CAST metrics have become the de facto standard for measuring the quality and productivity of application services.” – Helen Huntley, Research VP, Gartner
  26. 26. CAST Confidential Product Quality Metrics Most enterprises measure everything but the product delivered to the business CAST measures the product itself Robustness Performance Security Changeability Transferability Size Planning Estimation Scheduling Time Tracking Cost Tracking ProductProcess Time & Duration Effort & Budget Function & Scope Quality &Size Requirements Earned Value User Acceptance Usability CAST AIP
  27. 27. CAST Confidential Driving software measurement in the ADM industry 27 Key Influencers Recognize CAST 250 Global Leaders Rely on CAST Institutions Engage CASTSIs Resell CASTSIs Use/Resell CAST Top technology First in business IT Biggest benchmark DB
  28. 28. CAST Confidential CAST dashboards, reports & benchmarks 28 CAST Highlight Portfolio Analysis  Size  Complexity  Risk  Technical debt estimation Zero Deployment  No centralized source code collection  Portal results  Full analysis report CAST Application Intelligence Platform Risk Drivers  Robustness  Performance  Security Cost Drivers  Transferability  Changeability Alerts, trending, root cause analysis Discovery Portal Automated App Blueprint Discover, modernize and change applications Function Point Manager • Automated FP counts • Technical Sizing • Effort Estimation Function Point Changes Due to a Sequence of Change Requests 0 5 10 15 20 25 30 35 40 0 50 100 150 200 Cumulative Effort (Staff Hours) #FunctionPoints 1 52 3 4 Benchmarking Services Compare to industry business process and technology
  29. 29. CAST Confidential ArchitectureCompliance Enterprise-grade analysis requires a 3-tier approach 29  Intra-technology architecture  Intra-layer dependencies  Module complexity & cohesion  Design & structure  Inter-program invocation  Security Vulnerabilities Module Level  Integration quality  Architectural compliance  Risk propagation simulation  Application security  Resiliency checks  Transaction integrity  Function point & EFP measurement  Effort estimation  Data access control  SDK versioning  Calibration across technologies System Level Data FlowTransaction Risk  Code style & layout  Expression complexity  Code documentation  Class or program design  Basic coding standards Program Level Propagation Risk Java JSP EJB PL/SQL ASP.NET Oracle SQL Server DB2 T/SQL Hibernate Spring Struts .NET C# VB COBOL C++ COBOL Sybase IMS Messaging Java Web Services APIs 1 2 3
  30. 30. CAST Confidential Project #n Project #3 Project #2 Example: Managing multiple ADM vendors at global telecom company  230,000 employees, almost $100 billion revenue, 40 million customer accounts  Billing & OSS Solutions – 120 billion call records and 1 billion invoices per year – Also, SAP, Siebel, all front end apps that power e-commerce sites  Running one or two times per quarterly release – SLAs in contracts based on CAST – Aggregation of CAST metrics into C-level management dashboards Software analysis Vendor facility Team #1 Team #2 Team #3 Team #4 Neutral & independent vendor unit is running the CAST AI Center for the customer Project #1 • 150+ applications • 4 apps silos, with 50+ CAST consumers plus management in each CAST Dashboard 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 1.1 2.0 2.5 Vendor A Vendor B Vendor C Vendor A
  31. 31. CAST Confidential 31 Structural quality impact at a major services brand Measured impact in a complex enhancement-heavy environment 304 222 196 385 401 231 198 242 279 167 112 258 274 149 140 245 188 61 56 78 97 62 57 81 75 87 40 260 181 167 225 265 220 151 195 232 154 150 295 385 228 163 223 186 148 84 121 136 96 38 11 10 8 11 0 100 200 300 400 500 600 700 R1 R1.1 R1.2 R2 R2.1 R3 R3.1 R4 R5 R6 R7 R7.1 R8 R9 R9.1 R9.2 R10 R10.1 R10.2 R10.3 R11 R11.1 R11.2 R11.3 R12 R13 R14E Code No RC Non Code Projected Count StructuralqualitySystemtestdefects Trend line Before CAST implementation 0 500 1000 1500 2000 2500 3000 3500 R1 R1.1 R1.2 R2 R2.1 R3 R3.1 R4 R5 R6 R7 R7.1 R8 R9 R9.1 R9.2 R10 R10.1 R10.2 R10.3 R11 R11.1 R11.2 R11.3 R12 R13 R14E Order Management Inventory Management Billing Customer Service New critical violations CAST Analysis starting point CLIENT STUDY OVER 24 MONTHS
  32. 32. CAST Confidential CLIENT ADM VENDOR 32 Structural quality metrics can help both clients and vendors  Takes the guesswork out of quality  Improved code acceptance criteria  Objective performance measurements  Take control of accountability to the business  Improve collaboration between multisourcing parties  Justify the quality of work  Benchmark applications against industry and clients applications  Better release management  Increase productivity and improve bottom line  Remove risk from subcontracting decisions  Improve customer satisfaction  Improve collaboration between multisourcing parties Transparency between customer and vendor encourages a healthier and more risk-free relationship
  33. 33. CAST Confidential Contact Information Lev Lesokhin Stephanie Moore l.lesokhin@castsoftware.com smoore@forrester.com www.castsoftware.com www.forrester.com blog.castsoftware.com http://blogs.forrester.com/ @OnQuality @forrester slideshare.net/castsoftware