This document describes an experiment to evaluate methods for simplifying the use case points (UCP) software effort estimation technique while maintaining accuracy. The experiment analyzed 14 projects to test hypotheses around simplifying factors like actors, transactions vs steps, and technical/environmental factors. Results found that transactions outperformed steps, many factors had minor impact or overlapped and could be removed, and total steps or transactions could estimate effort nearly as well as UCP with simpler calculations. Threats to the study's validity were also addressed.

1. Prepared by: AbdulRahman Shaheen
Summerized By Shaheen

2. Agenda
 Introduction
 Experiment Setup
 Experiment Framework
 Experiment Theses and Results
 Threats to Validity
 Conclusion
Summerized By Shaheen

3. Introduction
 Software Estimation is key aspect of
software project management.
 Software Project Outsourcing estimation
problems:
 Underestimation: can lead to delays or poor
quality software.
 Overestimation: might result in bid rejection.
 Possible Solution:
 Simplify the process of Effort Estimation while
maintaining or improving the quality of
estimation.
Summerized By Shaheen

4. History
 Function Point Analysis (FPA)
introduced 1979 by Albrecht.
 Mark II FPA in 1991 by Symon
.
 UCP in 1993 by Karner.
Summerized By Shaheen

5. Use Cases
Summerized By Shaheen

6. Use Case Point Methods
 Assess Complexity of Actors
 aWeight(c): {1, 2, 3}
1. -Simple: API Communication
2. -Averge: Protocol communication
3. -Complex: GUI communication
 aCardinality: number of actors belonging to
the class
 Unadjusted Weight Factor
Summerized By Shaheen

7. Use Case Point Methods - cont
 Assess Complexity of Use Cases
 complexity depends on number of transactions
 Given use case u, number of transactions
#trans(u)
 uWeight(c) = 5 for simple, 10 for average, and
15 for complex;
 uCardinality(c) is the number of use-cases
assigned to class c (depends on a described
system).
Summerized By Shaheen

8. Technical and Environmental
Factors
Summerized By Shaheen

9. Use Case Point Methods - cont
 Add Adjustment Factor(Technical +
Environmental)
 TF_weighti is the weight of the ith technical
complexity factor (see Table 1);
 valuei is the predicted degree of influence
of the ith technical complexity factor on the
project (value between 0 and 5).
Summerized By Shaheen

10. Use Case Point Methods - cont
• EF_weighti is the weight of the ith environmental
factor (see Table 1);
• valuei is the predicted degree of influence of the
ith environ mental factor on the project (value
between 0 and 5).
Summerized By Shaheen

11. Use Case Point Methods -
cont
 UUCP = UAW + UUCW
 UCP = UUCP * TCF * EF
 Effort Estimations
 Effort = UCP * PF
○ [Karner]
 PF = 20 hours / UCP
○ [Schneider and Winters]
 PF=20h if EF<3
 PF=28h if 3<EF<4
 PF=36h if EF>4
Summerized By Shaheen

12. Use Case Point Methods -
cont
 Calibrating UCP with historical data:
Summerized By Shaheen

13. Projects Setup
 14 projects
 277 to 3593 man hours
 Projects labeled according to their
development environment
 A – G: industrial projects
 H – N: Pazon Univ. (PUT)
 Origion
 U:budgeted
 S2B: student to business
 I: software development company
Summerized By Shaheen

14. Projects Information
Summerized By Shaheen

15. Projects UCP
Summerized By Shaheen

16. Subsets of Datasets used in the
study
Summerized By Shaheen

17. Framework
 UCP calculations:
1. Reviewing use cases and reject:
○ Business use case.
○ Include + extend use cases, if they describe
the same logic with lower abstraction
○ Unimplemented + duplicated.
2. Counting transactions and steps
○ External expert was used for validation of
correctness of step counting.
3. Obtaining TCF and EF
4. Calculating UCP
Summerized By Shaheen

18. Framework
 Evaluation of prediction accuracy
1. Ordinary least squares regression (OLS)
○ pi is the project for which effort is estimated;
○ᵝSize is the slope for Size;
○ ᵝ is the constant or intercept (set to 0);
0
○ Size (pi) is the value of the size metric
calculated for the project pi.
Summerized By Shaheen

19. Framework
 Evaluation of prediction accuracy
2. Multiple regression
○ pi is the project for which effort is estimated;
○ ᵝ Size is the slope for Size;
○ ᵝ Factor is the slope for Factor;
○ ᵝ is the constant or intercept (set to 0);
0
○ Size (pi) is the value of the size metric calculated
for the project pi.
○ Factor (pi) is the value of the additional factor
included in the regression model (chosen among
the EFs, TCFs, and team size) for the project pi
Summerized By Shaheen

20. Framework
Summerized By Shaheen

21. Framework
 The evaluation criteria
○ magnitude of relative error (MRE) [inversely]
 Agregated using Mean of MRE (MMRE)
○ mean relative error (Mean RE)
 bias of the estimates
○ Pred (e)=k/n [proportional]
 prediction quality is calculated on a set of n projects,
where k is the number of projects for which
estimation error (MRE) is less than or equal to e
 To test Statistical Significance: two-tailed
Wilcoxon signed-rank test with the
significance level a set to 0.05 was used.
Summerized By Shaheen

22. Experiment
 Actors complexity in Use Case Points
 UAW had only minor impact on the accuracy of the
effort estimation based on UCP
 Adjustment factors in Use Case Points
 Problems:
○ Lack of standardized (agreed) scale
○ Not verified weights of factors
 Investigation
○ Overlapping
○ Minor influence on estimation
 Result:
○ 13 to 4 technical complexity factors
○ 8 to 2 environmental factors
Summerized By Shaheen

23. Experiment
 Use Case Points – steps vs. transactions
 Transactions: either executed fully or not at all.
 Counting steps will simplify calculation of UCP
method.
 Thesis: The value of UCP calculated based on
steps is the same as if calculated based on
transactions.
 Results are not the same.
 UCP-T = 0.584 * UCP-S
○ Can not be generalized.
○ Depends on SRS writing styles.
Summerized By Shaheen

24. Experiment
 Thesis
 The accuracy of UCP calculated based on steps
is not worse than if calculated based on
transactions.
 Results:
 Almost the same, however, on average UCP-S
more accurate.
 Minor tendency to overestimate.
 Mean RE close to Zero. (almost no bias)
 Conclusion: if a company use standard
style of writing they can use UCP-S instead
of UCP-T.
Summerized By Shaheen

25. Experiment
 Can the estimation be further simplified to
count just the total number of steps in all
use cases?
 Thesis: The Steps metric, which is the total
number of steps in all use cases describing
the system, can be used to estimate effort
with similar accuracy to UCP.
 Result:
 Can be used to simplify the effort.
 Suffers from sensitivity to writing styles.
Summerized By Shaheen

26. Use-case transactions methods
 The Transactions metric is defined as the
total number of trans- actions that can be
identified in all use cases describing the
system under development.
 The TTPoints metric is also based on the
number of transactions, but it includes
additional information about the semantics
of transactions (twelve semantic
transaction types have been identified so
far ).
Summerized By Shaheen

27. Experiment
 Thesis: Use-case transactions can be used
to provide effort estimation with better
accuracy than UCP calculated based on
steps.
 Investigation: compare two projects with:
 different writing styles.
 Same actual effort.
 Result: Based on post-productivity factor
UCP-T is better if there historical data
suffer either from writing style or
abstraction level.
 Lower Variability on prediction
Summerized By Shaheen

28. Experiment
 TTPoints outperform all step based
metrics.
 However, calculation is complex.
Summerized By Shaheen

29. Threats to validity – internal
validity
 Projects developed by different
organizations.
 Maturity of development process is different.
 Some records does not show activities.
 Objectiveness of Transaction [style of
writing]
 Solved by allowing only one person to
extract the transactions and cross validate it
by external expert.
Summerized By Shaheen

30. Threats to validity – Conclusion
validity
 Large number of direct comparison
 Can be solved using Bonferroni correction.
 Power of 10
 sensitivity analysis was performed: and
conclusion was “whenever we were not able
to reject the null hypothesis, it could mean
that either the null hypothesis was really true
or the effect size was too small to be
detected in this study”.
Summerized By Shaheen

31. Threats to validity – External
validity
 Size of Data Sets:
 Largest data sets used to study UCP
 heterogeneity of the analyzed set of
projects
 Same cross-validation procedure for all
combinations.
○ MMRE shows the difference in data set does not
influence the visibility of findings.
 Data-intensive systems
 This study can be generalized to this kind of
systems.
Summerized By Shaheen

32. Conclusion
 Potential ways to simplify effort
estimation:
 Reject UAW [minor impact on accuracy]
 Reduce Technical and Environmental
Factors.
 Use steps to calculate UCP. Further more,
you can use steps metric.
 Transaction Metrics can replace UCP-T.
 Measures based on Steps are sensitive
to writing styles.
Summerized By Shaheen

33. Future Work
 Developing standards of writing Use
Cases. This might include enforcement
to use specific phrases to describe use
cases.
 Study if a combination or all suggested
simplification can be used together to
reduce effort while maintaining accuracy.
Summerized By Shaheen