Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Managed Services Using SLAs and KPIs

11,773 views

Published on

Sarat Addanki's Presentation at the UK TestExpo 2014.

Published in: Technology
  • Hi there! I just wanted to share a list of sites that helped me a lot during my studies: .................................................................................................................................... www.EssayWrite.best - Write an essay .................................................................................................................................... www.LitReview.xyz - Summary of books .................................................................................................................................... www.Coursework.best - Online coursework .................................................................................................................................... www.Dissertations.me - proquest dissertations .................................................................................................................................... www.ReMovie.club - Movies reviews .................................................................................................................................... www.WebSlides.vip - Best powerpoint presentations .................................................................................................................................... www.WritePaper.info - Write a research paper .................................................................................................................................... www.EddyHelp.com - Homework help online .................................................................................................................................... www.MyResumeHelp.net - Professional resume writing service .................................................................................................................................. www.HelpWriting.net - Help with writing any papers ......................................................................................................................................... Save so as not to lose
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Hello! I can recommend a site that has helped me. It's called HelpWriting.net So make sure to check it out!
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Bill Fullilove, disabled with cerebral palsy, and bankrupt from high medical bills, says he was able to turn $150 into $1,000 with the "Demolisher" Betting System:  http://t.cn/A6zP2GDT
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Managed Services Using SLAs and KPIs

  1. 1. Testing as a Managed Service Using SLA’s and KPI’s Sarat Addanki Head of Practice CONNECT WITH US:
  2. 2. Corporate overview 03 CONNECT WITH US: Agenda 01 04 2 Public | Copyright © 2014 Prolifics 06 05 02 SLAs KPIs Sample SLA reporting Q&A Demo
  3. 3. Best-in-class architects and specialty experts: BPM, IIB, DataPower, WSRR, and other IBM products and tools. CONNECT WITH US: 3 Prolifics at a Glance 5-Years Compound Annual Growth Rate 19% Employees Worldwide 1,500 Global Presence Years in Business 35+ United States, United Kingdom, Germany, India 20+ Technology Accelerators 550+ Technical Certifications Over 10 Technology and Solutions Awards since 2009 including Business Agility, Customer Integration and Digital Experience Over 150 global customers are currently Fortune 1000 companies 96% industry-high 96% success rate of deploying robust solutions. Offices 14 Awards IBM Technology Expertise Fortune 1000 91% Rate of Repeat Engagements Public | Copyright © 2014 Prolifics
  4. 4. CONNECT WITH US: Our Services Public | Copyright © 2014 Prolifics 4 • Application Development, Maintenance and Support • Outsourced Product Development • Application Modernization • Portal Content & Collaboration • Test Advisory • Functional Testing • Packaged Apps • Custom Apps • Cloud • Test Automation • Middleware and SOA • Performance Engg • Mobile Testing BPM & Connectivity User Experience Security Application Performance Management Enterprise Content Management Business/Data Analytic
  5. 5. CONNECT WITH US: How Prolifics Can Help You 5  Adapt faster than the competition with continuous process improvement  Transform your applications to increase business agility  Create best-in-class customer and employee experiences. Become more social and collaborative  Gain actionable insight with an intelligent approach to security BUSINESS PROCESS MANAGEMENT SOA INTEGRATION USER EXPERIENCE SECURITY  Drive business growth with proactive IT management  Maximize IT investments with end-to-end testing  Optimize business performance with smarter, faster decisions  Increase efficiency within your organization and achieve “straight through processing” APPLICATION INFRASTRUCTURE TEST ADVISORY, APPLICATION TESTING & TEST AUTOMATION BUSINESS ANALYTICS OPERATIONAL DECISION MANAGEMENT (ODM)  Simplify employee and customer access to the documents they need when they need them  Get the most out of your IT to create competitive advantage.  Leverage global expertise to support your IT environment  Optimize your software spend to achieve business value ENTERPRISE CONTENT MANAGEMENT ADVISORY SERVICES MANAGED SERVICES SOFTWARE Public | Copyright © 2014 Prolifics
  6. 6. Technology Government CONNECT WITH US: 6 Prolifics Customers Telecommunications Insurance HealthCare Healthcare 6 Financial Services Utilities Retail & Distribution Media & Entertainment Public | Copyright © 2014 Prolifics
  7. 7. Evaluate value Measure quality Why Managed Service and SLAs? Measure productivity Incident DRE Mgmt. Companies are moving from Outsourcing Model to Managed Services Model as it allows them to govern the vendor services with the help of mutually decided upon Service Level Agreements (SLA’s). CONNECT WITH US: 7 Continues improvement Determine cost of testing Create accountability Process Compliance Within Budget % of Automation Business Process Knowledge Test Design Efficiency Manage vendor SLA’s are vital signs. You can tell where a testing organization is headed and also measure it’s maturity level
  8. 8. CONNECT WITH US: Key Performance Indicators (KPI’s) 8 Test case effectiveness Test progress plan vs actual Cost of testing as a % development Skill set matrix Quality of development No. of regression test cases updated per release Quantifiable regression execution
  9. 9. SLAs and Measurement Intervals Monthly Quarterly Quarterly Quarterly Monthly Monthly CONNECT WITH US: Defect Removal Effectiveness Process Compliance Business Process Knowledge General Incident Management Automated Regression library - Execution Regression Library - Updates 1234 56 78 9 10 11 12 13 14 15 16 17 Offshore Technical Skillset Effort per Defect Savings thru Defects Through by Project types Resource Utilization Submission of SLA Report Review of RCA Test Design Efficiency Delivery within budget Sizing and Estimation Turnaround Metrics Accuracy Monthly Monthly Monthly Monthly Monthly Monthly Monthly Monthly Quarterly Quarterly Quarterly S. No SL A Non SLA Metrics Measurement Interval 9
  10. 10. DRE Process Compliance Business Process Knowledge CONNECT WITH US: SLA Target, 95% General Incident Mgmt. Automated Regression Library - Execution SLA Target, 100% Regression Library - Updates 100% 98% 100% 91% 100% 100% Snapshot of SLA Metrics June, 2014 10 SLA Target, 100% SLA Target, 98% SLA Target, 90% SLA Target, 98%
  11. 11. Description Formula Target Frequency CONNECT WITH US: Measure of the quality delivered in a given engagement by measuring the number of defects that seeped into production Performance = Number of non rejected defects found prior to production / (total number of non rejected defects registered during development life cycle from requirements phase up to and including warranty period) * 100 Critical and High - 95% Medium – 90 % Low – 85 % Monthly Defect Removal Effectiveness 11
  12. 12. Process Compliance Description A quarterly audit to measure vendor’s compliance with SV&T key process areas Formula Significant deficiency score = Sum of % non-compliance of critical process activities/number of audit assessments Compliance score = Sum of % compliance of key process activities/number of audit assessments Measurement Interval Monthly Target <2% significant deficiencies on critical process activities ≥90% compliance on key process activities CONNECT WITH US: 12
  13. 13. 13 Quarterly assessment to measure service provider’s staff proficiency in McKesson business process knowledge and knowledge management. Performance = Count of number of people rated at different expertise levels (Advanced, Proficient) segmented by McKesson business process Monthly CONNECT WITH US: 15 Business Process (“BP”) Knowledge Description Formula Measurement Interval Target For each BP rated as Critical or High: 1. A minimum of 2 resources rated at advanced 2. A minimum of 2 rated as proficient For each BP rated as Medium or Low: 1. A minimum of 1 resources rated at advanced level 2. A minimum of 1 rated as proficient level
  14. 14. CONNECT WITH US: General Incident Management Description A quarterly assessment of incident closure rates within the prescribed response time Formula 14 Measurement Interval Target Performance = Number of occurrences where incident is closed on or before the expected closure date/Number of total occurrences – Segmented by severity Quarterly High Severity Issue - Respond within 2 hours (from the time of notification). Resolve within 8 hours. Medium Severity Issue - Respond within 4 hours (from the time of notification). Resolve within 24 hours. Low Severity Issue - Respond within 8 hours (from the time of notification). Resolve within 5 business days unless both parties agree to incorporate the resolution in the next scheduled release (not to exceed 90 days)
  15. 15. Description Formula Measurement Interval Target CONNECT WITH US: Automated Regression Test Library 2 week rolling cycle for periodic runs and on demand cycle based on specific request Performance = Actual test completion date ≤ expected completion date Quarterly 15 >=90% Successful execution of Test Automated Regression test Cases <=10% Successful execution of Test automated regression test cases through manual approach in case of Data setup issues.
  16. 16. 16 Offshore BI/BW Skill set Advanced Proficient CONNECT WITH US: Offshore Technical Skillset 1 Offshore SAP Technical Skill set Advanced Proficient Basic 12 0 1 3 10 2 3 0 1 0 0 1 2 6 2 0 0 23 5 3 1 6 0 0 0 0 11 0 0 0 12 4 2 6 11 2 1 1 0 25 20 15 10 5 0 1 4 2 1 2 1 2 2 2 1 2 3 2 1 3 5 4 3 2 1 0 SAP BO SAP BW Datastage SAP BODS SAP HANA
  17. 17. Offshore HRIS team Skillset Advanced Proficient Offshore Bench Strength CONNECT WITH US: Offshore Technical Skillset 2 1 Offshore Vistex Skill set Advanced Proficient Basic 2 2 3 1 1 3 0 0 2 Offshore Legacy & Automation Skill 1 0 Advanced Proficient Basic 1 1 3 3 0 2 3 1 0 2 1 0 set 1 2 0 0 3 4 3 2 1 4 3 2 1 0 0 Acumax FSCM SCORE TANDEM QTP WMI 0 VISTEX CHARGEBACKS VISTEX CONTRACTS VISTEX MDM VISTEX REBATES 1 2 2 2 0 1 2 SAP ABAP 5 SAP BI/BW, BO SAP SD SAP FI No. of Resources 17 2 1 0 2 2 1 2 2 0 0 Peoplesoft Manual Testing ETL Testing
  18. 18. CONNECT WITH US: Effort per Defect & DRE FY13YTD, FY14YTD & April ’14, May ‘14 18 DRE Effort per Defect 99% 99.30% 98% 98% 50 39 43 63 70 60 50 40 30 20 10 0 100% 95% 90% 85% 80% 75% 70% FY13 YTD FY14 YTD April '14 May '14
  19. 19. CONNECT WITH US: 19 Submission of SLA Report Reporting Period Target Date Publishing Date Published on time Jan 2014 2/5/2014 2/5/2014 Yes Feb 2014 3/5/2014 3/3/2014 Yes Mar 2014 4/5/2014 4/2/2014 Yes Apr 2014 5/5/2014 5/1/2014 Yes May 2014 6/5/2014 6/4/2014 Yes
  20. 20. CONNECT WITH US: 20 Review of RCA’s This SLA is included to track RCA’s created vs. percentage of RCA’s reviewed and signed off. Total of 20 RCA’s are created till May 2014. 5 RCA’s are added in May 2014, two for SO 3163 (One Medium, One Low), one medium defect each for SO 2344, SO 4921, SO 3264. 20 20 20 18 16 14 12 10 8 6 4 2 0 Total RCA's Created Total RCA's Reviewed No. of RCA's
  21. 21. Description Formula Target Frequency CONNECT WITH US: A quarterly assessment to measure turnaround time for estimation and sizing of testing initiatives Performance = number of sizing requests closed on or before scheduled closure date/number of sizing requests submitted – segmented by project type (large, medium and small) Large Projects – 10 Business Days Small & Medium projects – 5 Business Days Quarterly Estimation Turnaround Time 21
  22. 22. Metrics Accuracy Description Formula Target Frequency CONNECT WITH US: A quarterly assessment of accuracy of metrics provided by service provider. McK SV&T will identify ≥30% of metrics to audit. Prolifics will provide data used to calculate metric. Prolifics will provide/ identify source of data for analysis. Number of measurement items reported accurate/Total number of measurement items reported. ≥95% accuracy of reported metrics Quarterly 22
  23. 23. Performance will be measured on 2 dimensions: Significant Deviation Measure =( Number of projects where actual QA cost exceeds budgeted/forecasted QA cost by 25%/Total number of projects)*100 Accuracy Measure =( Number of projects where actual QA cost is within +/- 10% of budgeted/forecasted QA cost/Total number of projects)*100 CONNECT WITH US: Description Formula Budget = baseline budget + delta cost. Baseline budget includes effort and cost estimated for test lifecycle activities, Delta cost includes Service Level Credit to cost and effort caused due to authorized changes to scope or other project constraints . Delivery within Budget Target Frequency Significant deviation measure = 0% Accuracy measure = >= 95% Quarterly 23
  24. 24. CONNECT WITH US: Delivery within Budget No specific data point for this, based on the comments from RTM’s. This metric is difficult to accurately track at this point as the total budget for Prolifics and SV&T employees is combined SVTCs and RTMs do the estimates in most cases In BI space, (which is completely outsourced to Prolifics) all SO’s have been delivered with in the budget TARGET ACTUAL (YTD) 95% 100% 24

×