Your SlideShare is downloading. ×
Process Performance Models:Not
Necessarily Complex
                  -Himanshu Pandey
                    Nishu Lohia
    ...
Process Performance Models - Not Necessary Complex
HMBP 2010
21-May-2010
Who We Are

Aricent is the world’s leading independent communications
   software company.
   – Dedicated focus on communi...
What We Offer

 Wireless                        Data                      Signaling                Packet networks/
      ...
Agenda

      Problem Statement
      Aricent’s way to Resolution
      Process Performance Modelling – Overview
     ...
Problem Statement

• CMMI Level 5 can not be achieved without Statistical process
  performance models in place.
• Models ...
Aricent’s Way Of Resolution

• Identify generic life cycle steps where PPM could be applied across
  maximum projects
• Id...
Process Performance Modeling

• The CMMI definition of PPM -> description of the relationships among
  process attributes ...
In-house Statistical Tools
                         Rayleigh’s Defect Prediction Model
                         Test End...
In-house developed Models Overview

• The 3 models developed in-house are:


   Rayleigh’s Defect Prediction Model

   Tes...
Rayleigh’s Defect Prediction Model

• Input to the Tool:
    • Total number of detected/injected defects in past phases
  ...
In-house developed Models Overview

• The 3 models developed in-house are:


    Rayleigh’s Defect Prediction Model

    T...
Test End Date Prediction Model – Overview

• Performs simulation to forecast:
    • Testing End date
    • Using EWMA cont...
Test End Date Prediction Model cont..

• Input to the Tool:
    • Total number of cases planned for execution and actual t...
In-house developed Models Overview

• The 3 models developed in-house are:


    Rayleigh’s Defect Prediction Model

    T...
NHPP + Gompertz Model – Overview
                                                                    CURRENT STATUS OF
• P...
NHPP + Gompertz Model Cont..

• Input to the Tool:
    • Actual test case execution data; each case run failed or passed.
...
Process Performance Models
                           NHPP- Gompertz + Test End Date Prediction
                        ...
NHPP- Gompertz + Test End Date Prediction




• Test End Date Prediction Tool takes test execution data as input and provi...
Rayleigh’s Model + Test End Date Prediction




• Test End Date Prediction Tool takes test execution data as input and pro...
NHPP-Gompertz + Test End Date Prediction + Rayleigh’s Model




• Test End Date Prediction Tool usage along with NHPP and ...
NHPP-Gompertz + Test End Date Prediction + Rayleigh’s Model


• Special scenario:
    • When Rayleigh’s and NHPP’s forecas...
Conclusion

• Yes! We applied these in-house developed PPMs and achieved CMMI level 5 (v1.2) successfully.
• Applied on wi...
Thank You!




7-Jun-10                24
Click here for:

High Maturity best practices
HMBP 2010 Presentations
organized by QAI

         Click here
Upcoming SlideShare
Loading in...5
×

CMMI High Maturity Best Practices HMBP 2010: Process Performance Models:Not Necessarily Complex by Himanshu Pandey and Nishu Lohia

914

Published on

Process Performance Models:Not Necessarily Complex -Himanshu Pandey and Nishu Lohia(Aricent Technologies) presented at
1st International Colloquium on CMMI High Maturity Best Practices held on May 21, 2010, organized by QAI

Published in: Business, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
914
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "CMMI High Maturity Best Practices HMBP 2010: Process Performance Models:Not Necessarily Complex by Himanshu Pandey and Nishu Lohia"

  1. 1. Process Performance Models:Not Necessarily Complex -Himanshu Pandey Nishu Lohia (Aricent Technologies)
  2. 2. Process Performance Models - Not Necessary Complex HMBP 2010 21-May-2010
  3. 3. Who We Are Aricent is the world’s leading independent communications software company. – Dedicated focus on communications software – Unmatched depth and breadth of services and products – Culture of innovation, excellence and results More than 400 customers across the world Equipment Manufacturers Device Manufacturers Service Providers 7-Jun-10 3
  4. 4. What We Offer Wireless Data Signaling Packet networks/ Communications VoIP GSM, GPRS/EDGE, 3G, ISDN, SS7 and SIGTRAN IMS, SIP, H.248, MGCP, WCDMA, CDMA2000, Routers, VPN and QoS, VoWiFi, Interworking WiMAX, UMA, Femtocell ATM, IP, MPLS, GigE, Platform 8,000+ employees with expertise in all Communication Engineering major communications categories Applications Messaging, Location Based ATCA, Network Processors Services, Workforce Automation, Voice Applications Mobile Handsets Billing and OSS User Interface, Multimedia Multi-vendor Billing, Applications, Physical OSS Integration, Service Layer, Middleware, Multiple Activation, OSS/BSS OS and Platforms, DSP Business Process Re-eng DSP Broadband and Network Transmission Video and Voice Processing, Wireless access Management Audio/Video Codecs, xDSL, Satellite, Cable, TL1, SNMP, CORBA and SONET/SDH, RPR, DWDM Network Processor 802.11a/b/g/i and WiMAX CLI Application, Microcode Design 7-Jun-10 4
  5. 5. Agenda  Problem Statement  Aricent’s way to Resolution  Process Performance Modelling – Overview  Models- Overview – Rayleigh’s Defect Prediction Model – Test End date prediction Model – NHPP + Gompertz Model  Combinations of Models  Conclusion  Q&A 7-Jun-10 5
  6. 6. Problem Statement • CMMI Level 5 can not be achieved without Statistical process performance models in place. • Models available in the market are: • Too Costly • Complex to understand and implement • Limited availability • Non-customizable • Law of inertia is applicable to software industry as well. • High resistance for change due to strict timelines. • Implementation of models is an over head. 7-Jun-10 6
  7. 7. Aricent’s Way Of Resolution • Identify generic life cycle steps where PPM could be applied across maximum projects • Identify in-house developed statistical tools which are being used most frequently across the organization. • Identify the tools those can be used in collaborative mode effectively to: • Perform What if Analysis • Implement PDCA cycle 7-Jun-10 7
  8. 8. Process Performance Modeling • The CMMI definition of PPM -> description of the relationships among process attributes and its work products that are developed from historical process-performance data and calibrated using collected process and product measures from the project which are used to predict results to be achieved by following a process • Makes quantitative predictions about a particular process • May estimate resource consumption, effectiveness, efficiency depending upon organization goals 7-Jun-10 8
  9. 9. In-house Statistical Tools  Rayleigh’s Defect Prediction Model  Test End Date Prediction Model  NHPP + Gompertz Model 7-Jun-10 9
  10. 10. In-house developed Models Overview • The 3 models developed in-house are: Rayleigh’s Defect Prediction Model Test End Date Prediction Model NHPP + Gompertz Model 7-Jun-10 10
  11. 11. Rayleigh’s Defect Prediction Model • Input to the Tool: • Total number of detected/injected defects in past phases • Duration of the past and upcoming phases • Output from the tool: • Phase wise Estimated vs Actual Defect Distribution • Estimation of Defects passed to customer Phase wise & Cumulative Defect Distribution 90 160 80 Phase wise Defects 140 70 • Benefits: 120 Cum_Defects 60 100 50 80 40 60 • Forecast on Defect Passed to Customer 30 20 40 10 20 • Comparison of estimated vs actual defects in phases 0 SRS Design CUT RT PQT 0 Actual Defects (Phase wise) Estimated Defects(Phase wise) Estimated Defects (Cumulative) Actual Defects (Cumulative) 7-Jun-10 11
  12. 12. In-house developed Models Overview • The 3 models developed in-house are: Rayleigh’s Defect Prediction Model Test End Date Prediction Model NHPP + Gompertz Model 7-Jun-10 12
  13. 13. Test End Date Prediction Model – Overview • Performs simulation to forecast: • Testing End date • Using EWMA control charts which helps in • Observing/monitoring the variations in current test execution rate • Generating results to predict number of days to finish testing Test Case Executed Per Day 40.00 35.00 30.00 25.00 20.00 15.00 10.00 5.00 0.00 1 2 3 4 5 6 Exp. Mov. Average UCL LCL 7-Jun-10 13
  14. 14. Test End Date Prediction Model cont.. • Input to the Tool: • Total number of cases planned for execution and actual test case execution data • Revised test execution data included failed test cases • Output from the Tool: • Predicts the number of days pending to finish the remaining planned number of test cases • Benefits • Strategizing remaining number of days in order to finish testing on time Distribution of Possible Days to End Testing Date at 90% Replicate1 350 100% Replicate2 X Axis - Days to finish testing Y Axis - Frequency Replicate3 90% Replicate4 300 Replicate5 80% Replicate6 250 70% Replicate7 Replicate8 200 60% Replicate9 Number of days Replicate10 50% Replicate11 150 Replicate12 40% Replicate13 Replicate14 100 30% Replicate15 Replicate16 20% Replicate17 50 10% Replicate18 Replicate19 0 0% Replicate20 14.7 to 14.75 14.8 to 14.85 14.9 to 14.95 15 to 15.04 15.09 to 15.14 15.19 to 15.24 15.29 to 15.34 15.39 to 15.44 15.49 to 15.54 15.59 to 15.64 15.69 to 15.74 15.79 to 15.83 15.88 to 15.93 15.98 to 16.03 16.08 to 16.13 16.18 to 16.23 16.28 to 16.33 16.38 to 16.43 16.48 to 16.53 16.58 to 16.62 16.67 to 16.72 16.77 to 16.82 16.87 to 16.92 16.97 to 17.02 17.07 to 17.12 17.17 to 17.22 17.27 to 17.32 17.37 to 17.41 17.46 to 17.51 17.56 to 17.61 % Distribution of Replicate1 90%_Line 7-Jun-10 14
  15. 15. In-house developed Models Overview • The 3 models developed in-house are: Rayleigh’s Defect Prediction Model Test End Date Prediction Model NHPP + Gompertz Model 7-Jun-10 15
  16. 16. NHPP + Gompertz Model – Overview CURRENT STATUS OF • Performs forecast on SOFTWARE RELIABILITY DATE FOR WHICH S/W RELIABILITY • Reliability of the software under testing 23-Mar-10 STATUS IS GIVEN SOFTWARE RELIABILITY ON ABOVE 79.72% MENTIONED DATE • based on MTBF UPPER LIMIT OF 95% CONFIDENCE INTERVAL Almost 100% LOWER LIMIT OF 95% CONFIDENCE 50.94% • based on Gompertz reliability growth model INTERVAL • Expected number of failures in remaining number of testing days • Based on NHPP equation 7-Jun-10 16
  17. 17. NHPP + Gompertz Model Cont.. • Input to the Tool: • Actual test case execution data; each case run failed or passed. • Number of days of testing and average testers per day • Output from the Tool: • Reliability Growth Pattern • Current reliability of software • Failure Forecast • Benefits: • Strategizing remaining number of testing days in order to improve reliability and minimize failures passing to customer 7-Jun-10 17
  18. 18. Process Performance Models  NHPP- Gompertz + Test End Date Prediction  Rayleigh’s Model + Test End Date Prediction  NHPP-Gompertz Model + Test End Date Prediction Model+ Rayleigh’s Model 7-Jun-10 18
  19. 19. NHPP- Gompertz + Test End Date Prediction • Test End Date Prediction Tool takes test execution data as input and provides the number of days pending to finish the testing • NHPP Model, along with execution data, require to know how many more days testing will continue in order to predict the failures • Number of failed test cases predicted by NHPP then in turn act as revised input for Test End Date Prediction to re-calibrate the testing end date • Perform multiple calibrations in order to arrive at best suitable situation to strategize where highest software reliability can be achieved within lesser time and lesser failures 7-Jun-10 19
  20. 20. Rayleigh’s Model + Test End Date Prediction • Test End Date Prediction Tool takes test execution data as input and provides the number of days pending to finish the testing • Rayleigh’s Model, require to know for how many days testing will continue in order to predict the defects • Derive number of test case failures from number of defects predicted by Rayleigh’s model using baseline defects/failure rate and use this as input in Test End Date prediction for revised number of days. • Perform multiple calibrations in order to know estimated defect leakage to customer and strategize for the prevention. 7-Jun-10 20
  21. 21. NHPP-Gompertz + Test End Date Prediction + Rayleigh’s Model • Test End Date Prediction Tool usage along with NHPP and Rayleigh models separately has been discussed • Now, comparing the results of two, brings down to an interesting analysis in order to verify the defects/TC failure rate with baseline figures. • Hence, knowing your current project’s performance in comparison to past project’s performance justifies our subjective confidence on how far/close we are w.r.t. to achieving our targets. • Re-calibration and verification analysis based PCDA format, helps in strengthening our prevention strategies and hence achieving great results. 7-Jun-10 21
  22. 22. NHPP-Gompertz + Test End Date Prediction + Rayleigh’s Model • Special scenario: • When Rayleigh’s and NHPP’s forecasts are not in synch: • Study the phase wise defect distribution curve • Study Actual value curve vs Estimated value curve • Identify and Analyze the phases outside our control • Study the co-efficient of determination • Correctness of available data • Availability of support data used for forecasts • Decision • Choose the model for which most of the above parameters are satisfied 7-Jun-10 22
  23. 23. Conclusion • Yes! We applied these in-house developed PPMs and achieved CMMI level 5 (v1.2) successfully. • Applied on wide range of Projects. • The models applied ,undoubtedly: • Has almost no Cost • Are easy to understand and implement • Are Easily Customizable • Are not Overhead to Implement • Implies no resistance to use • Has no constraints of availability • Satisfies CMMI practices completely 7-Jun-10 23
  24. 24. Thank You! 7-Jun-10 24
  25. 25. Click here for: High Maturity best practices HMBP 2010 Presentations organized by QAI Click here

×