PROMISE 2011: What Prediction Model Should Be?


Published on

"What Prediction Model Should Be?"
Qing Wang

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

PROMISE 2011: What Prediction Model Should Be?

  1. 1. Whatprediction model should be?<br />Institute of Software Chinese Academy of Sciences<br />Qing Wang<br />
  2. 2. Why we use prediction model?<br />Understand the past behavious of process<br />Control the present behavious of process <br />Predict the future behavious of process<br />2<br />
  3. 3. How to understand the past behavious?<br />Based on the historical data which has been collected from relative processes<br />Apply appropriate measure and analysis techniques to analyze the data quantitatively.<br />Try to find some nature characteristics<br />Is stable?Nature bounds?<br />Does some quantitative relationship among process attributes?<br />Can be stratified /segmented or composed further?<br />3<br />
  4. 4. How to control the present behavious?<br />Based on the nature bounds (baselines) to establish the desired objectives of project<br />Compare the present data to say if it is compliant with the nature bounds.<br />Take proper action to correct the deviation<br />Monitor and evaluate the effect of correct action <br />4<br />
  5. 5. How to predict the future behavious?<br />Based on the quantitative relationship which established depend on the understanding of past behavious.<br />Predict the interim outcome and final outcome of project to know if the project is going well toward the project objectives.<br />5<br />
  6. 6. What is the essential ?<br />Differ from estimation model<br />Deterministic, uncertainty, ……<br />Use past and present data to predict future<br />Some dependent X factors, predict independent Y outcomes<br />The measurement for Xs and Y make sense<br />Data collection timely and frequently<br />Dynamical <br />Statistical, probability, simulation-based<br />6<br />
  7. 7. Must be careful when establish prediction model?<br />Use real world and relevant data<br />Align the necessary project objectives<br />SMART principle<br />Specific<br />Measurable<br />Attainable<br />Relevant<br />Time-bound<br />Cost-benefit balance<br />Feedback and refinement<br />7<br />
  8. 8. Practical Case – A CMMI ML4 organization<br />Issue: the organization found there are more than 90% projects delay is due to test and fix bugs need much more time than estimated<br />Question: How to predict the effort and schedule of test process?<br />Method: <br />Identify the factors which impact the effort and schedule of test process<br />Establish a prediction model use these factors<br />Measure and control these factors<br />Predict the effort and schedule of test based on the factors when they were got <br />
  9. 9. The process<br />Each user case has been implemented by requirement analyze, design and coding<br />reviews for each user case for each development activity were conducted respectively and defect data was recorded<br />Each iteration has a unique testing and data was recorded<br />The subsystem of each iteration was delivered to customer as a intermediate result, they must be on schedule and have good quality<br />System testing was conducted when all iterations were completed (each release)<br />
  10. 10. Process improvement objectives<br />Goal<br />G1: manage the schedule of each iteration and reduce the defect as soon as possible<br />G2: manage the system test to detect and fix defects as many as possible<br />
  11. 11. G1: in each iteration <br />All defect data related requirement analysis, design and coding were collected before testing<br />Can we find some quantitative relationship between these data and the effort of testing process? <br />There are two primary activities in testing process<br />Detect defects<br />Fix defects<br />
  12. 12. Get data from 16 history projects<br />
  13. 13. Defect injection analysis - 1<br />13<br />Defect injected in requirement analysis<br />Defect injected in design<br />
  14. 14. Defect injection analysis - 2<br />Defect injected in coding<br />The factor can be control , it can be used to establish prediction model<br />Y is the effort of fix defects. <br /> If the predicted effort beyond estimate, they should assign more engineers to assure on schedule deliver of each iteration.<br />
  15. 15. G2: for system testing<br />
  16. 16. Use controllable factor to predict<br />16<br />Similar prediction model for system testing can be established.<br />The possible impact factors could be:<br />The defects injected in requirement, design and coding<br />The defects removed respectively<br />The defects removed in integrated testing of each iteration<br />……<br />
  17. 17. Summary <br />17<br />All data can be get<br />The factors which used to establish prediction model are controllable and independent <br />Statistical techniques are used to do analysis<br />Prediction object is relevant to the organization process improvement objectives<br />The prediction model can be refined along with the using and more data was got<br />
  18. 18. Thanks!<br />18<br />