This document discusses Orange Polska's use of function point analysis (FPA) for estimating the size, effort, and pricing of IT projects, as well as their use of automated code review. It describes how Orange Polska counts over 1 million function points annually across 800 projects in many technologies. It also explains their four-step process for converting function points to price, including custom adjustment rules. Additionally, it outlines the quality metrics and tools used for automated code review of vendors' source code.

1. 1
Orange Polska for IWSM Mensura, Cracow 2015
Orange Polska
IWSM Mensura 2015
Practical usage of FPA
and automated code
review
Piotr Popowski & Bogusz Jeliński
IT Applications

2. 2
Orange Polska for IWSM Mensura, Cracow 2015
What we measure, count or look for in Orange
Function Points
 Manual, IFPUG-based counting for pricing
Quality of code (static review)
 Violations of patterns
 Weighted average of low-level grades
 Defects per FP
Software process
maturity, repository, continuous integration
Automated FP
Complete source code

3. 3
Orange Polska for IWSM Mensura, Cracow 2015
Measurement in figures
MANY TECHNOLOGIES
JAVA, C/C++, PHP, PL/SQL, WEB METHODS, PEOPLE SOFT, POWER
BUILDER, ABAP, ORACLE FORMS, etc.
1 000 000
FP COUNTED
800
PROJECTS
(GO-LIVES)
1 500
CODE AUDITS
7
DOMAINS
5
VENDORS
4 800
ENHANCEMENT COUNTS (incl. CRs)
200
IT SYSTEMS
5 YEARS
EXPERIENCE

4. 4
Orange Polska for IWSM Mensura, Cracow 2015
Part 1
Estimating size and price for IT projects
using FPA method
Piotr Popowski – IT Applications, Orange Polska
Contact: piotr.popowski@orange.com

5. 5
Orange Polska for IWSM Mensura, Cracow 2015
Context
In 2010 Orange Polska transformed its sourcing strategy from multi-vendor to
consolidated vendor agreements. In return for a long-term contract, the
company offered its suppliers exclusive rights to deliver software changes in
particular IT areas.
 Vendors
consolidation
 Solution
 Challenge Lack of free market pricing mechanisms, as implied by the contracts, imposed
the necessity to implement a dedicated sourcing and pricing method.
To meet the new challenges Orange Polska adopted a new method of
estimating IT development size, workload and finally, pricing.

6. 6
Orange Polska for IWSM Mensura, Cracow 2015
From FP to price
In four steps
FP and SNAP count based
on IFPUG and custom
rules
FP adjustment
Adjusting number of FP.
Impact ratios based on
COCOMO II
effort calculation
Contract-based
productivity ratios related
to vendor, system and
technology
price
Price calculation using
rates defined in contracts
FP
SP
Calculated by Orange and
vendor analysts
Verified and authorized by OPL
IFPUG certified team
Orange and vendor
analysts
Sourcing division Sourcing division

7. 7
Orange Polska for IWSM Mensura, Cracow 2015
Customization
Custom rules
The custom rules cover the areas where IFPUG FPA proved to be in some way deficient
Custom rules
 5% of FPs is „Rule 27” algorithmic complexity - internal data processing where data does not cross application boundary
 7% of FPs is „Rule 30” reusability – a reusable component, where development is made once but analysis and tests are conducted
for many elementary processes
 20% of FPs – dedicated rules for 26 systems in the following areas
 Integration
 Business Intelligence (ex. Data Warehousing)
 Wholesale

8. 8
Orange Polska for IWSM Mensura, Cracow 2015
Customization
FP adjusting
The main goal of adjusting is to assess effort
 Adjustation is made separately for each BFC (Base Functional Component)
 Impact ratios are defined based on COCOMO II Equation 4
 Impacts are expert-estimated and mutually agreed
 Also using NESMA impacts as reference
Creation or huge change Moderate change Small change Suppression or COTS
Effort

9. 9
Orange Polska for IWSM Mensura, Cracow 2015
Customization
Productivity
 Productivity ratios were established during vendor selection process
 COCOMO II factors were used as reference for determining productivity
Productivity
Technology System Vendor
Year of
contract

10. 10
Orange Polska for IWSM Mensura, Cracow 2015
Knowledge base and tools
FP calculation
 CARTO – Excel spreadsheet for doing detailed calculations based
on HLD
 Full decomposition: project > system > functionality > BFC
(or SCU for SNAP)
 Additional information for effort calculation (e.g. technology,
impact)
 FP adjustment
Document and report
 FP Repository – all data from CARTOs recorded in a dedicated system
 Over 30 on-line generated reports available from the repository
 Progressive build of Application Count
HLD
CARTO
REPOSITORY
REPORTS

11. 11
Orange Polska for IWSM Mensura, Cracow 2015
Common pricing model
 Instead of tenders, quotes and negotiations – objective pricing
Budget and capacity early estimation
 Based on historical data
Project Roadmap management
 Project assignment to particular path depending on path capacity
 Path management – schedule and capacity
KPIs
 Cost efficiency – spending/FP
 Quality – errors/kFP
Process optimization
 Better and more precise analytical documents (HLD)
 Tracing volume of changes in IT projects (e.g. number of projects, number of change
requests, changing size of average project in particular domain)
Benefits

12. 12
Orange Polska for IWSM Mensura, Cracow 2015
Challenges
Introduce another measuring method in order to verify or improve
current model (e.g. Use Case Points)
Measuring Agile projects
Improving the way to measure configuration changes
Introduce NESMA impacts

13. 13
Orange Polska for IWSM Mensura, Cracow 2015
Orange Polska experience
Since 2010
 Has a dedicated, internal, IFPUG certified Metric Team
 Uses IFPUG FPA (and SNAP since 2012 ) for pricing IT projects
 Uses metrics to manage Project Roadmap and IT budget
Dedicated Metric Team provides the following services
 Design and implementation of effort estimation and pricing models for IT contracts
 FPA, SNAP and custom trainings
 Methodology, contract and project audits
Services are offered internally and externally, both on-site
(currently Warsaw and Cracow) as well as remotely

14. 14
Orange Polska for IWSM Mensura, Cracow 2015
Part 2
Automated code review
dr Bogusz Jeliński – IT Architecture Strategy, Orange Polska
Contact: bogusz.jelinski2@orange.com

15. 15
Orange Polska for IWSM Mensura, Cracow 2015
What we measure, count or look for in Orange
Function Points
 Manual, IFPUG-based counting for pricing
Quality of code (static review)
 Violations of patterns
 Weighted average of low-level grades
 Defects per FP
Software process
maturity, repository, continuous integration
Automated FP
Complete source code

16. 16
Orange Polska for IWSM Mensura, Cracow 2015
Code review – history & inspirations
 1976 - Fagan’s classic article in IBM Systems Journal
 1979 - Lint for Unix V7
 1991 - ISO/IEC 9126
 1992 - Technical debt metaphor by Ward
Cunningham
 90’s - Sun Java Code Conventions
 Our own manual review (WebMethods, Drools)

17. 17
Orange Polska for IWSM Mensura, Cracow 2015
Tools used for the static review of source code
pros & cons
 Many supported technologies
 Tracks cross-layer dependencies
 Semi-automated count of function points
 Cost of license
 Needs extensive training to operate
 Not automated (as its manufacturer might
claim)
 Cannot be run by any vendor before code
hand-over
CAST
Sonar
 No license fee
 Integrated with dev tools
 Supports few technologies
 Some plugins are not open-source
HP Fortify
 Security oriented
 Only security oriented
 Not free of charge

18. 18
Orange Polska for IWSM Mensura, Cracow 2015
Quality metrics
Content of contracts with vendors
Low level rules
Aggregate
Total
Grade
Expected
improvement
< 3.10 0,25
3.1 - 3.5 0,15
> 3.50 0,10
Expected
yearly
improvement
Tree-based aggregation with weights
Values from 1 (bad) to 4 (very good)

19. 19
Orange Polska for IWSM Mensura, Cracow 2015
We prevented the increase of technical debt
Technical
Quality
Index
 Discovered acts of sabotage – removing comments to hamper the change of vendor
 Created rules for the analyzer to prevent technology-specific anti-patterns
(based on test reports)
 Fewer production incidents by 4%
 Increase of code performance prevented infrastructure spending
Benefits

20. 20
Orange Polska for IWSM Mensura, Cracow 2015
Challenges
We found that vendors differ vastly
in terms of delivered quality
Reviews may be used to create
a ranking of software vendors
Functionalities always prevail over
internal quality
We need quality metrics in contracts
Vendors tend to claim too many
false-positives
Anti-patterns remain in the code,
aggregate metrics (CAST TQI) may be
used to exert pressure on the vendor
There have to be penalties in contracts
for poor quality. Or at least KPIs
inspected by top management
Good-will assumption does not work

21. 21
Orange Polska for IWSM Mensura, Cracow 2015
Service for third parties – scope and tools
Quality audits by Orange Polska*
Static code review
 CAST & Sonar (cyclical & one shot), including adding new rules
Productivity review
 CAST (automated function points)
Open-source licenses review
 Palamida
Security audit
 HP Fortify
Software process improvement
 SVN, Git, Jenkins (continuous integration)
* provisioned in English, French and Polish

22. 22
Orange Polska for IWSM Mensura, Cracow 2015
Thank you
Bogusz Jeliński
bogusz.jelinski2@orange.com
Piotr Popowski
piotr.popowski@orange.com
IWSM Mensura 2015
Estimating size and price
for IT projects using FPA
method
Automated code review