IWSM Presentation

1. IWSM Mensura 2014
Nesma working group Benchmarking
Harold van Heeringen, Sogeti Nederland B.V. (chair)
Ton Dekkers, Interdependent
Marcel Pereboom, Mediaan
René Nijland, Capgemini
wgBenchmarking@nesma.org

2. Nesma wg Benchmarking
 Full ISBSG member since many years;
 Actively promotes ISBSG and sells products;
 Organizes workshop ‘Performance measurement
and Benchmarking’;
 Attended ISBSG workshops >10 years;
 Currently holds position of ISBSG president;
 Offers ISBSG reports to members for free.
| IT CONFIDENCE - HISTORICAL DATA

3. Size does matter, but…
 We measured 350 FP, so... What is the estimate?
  Effort, duration, quality, risk?
  Price per FP?
 Why are we losing most of the bids we do?
 Competitive advantage? Or not?
 What is our productivity compared to the industry?
 We are going to send out a request for bid based on price/FP
 How can we compare the proposals?
 How can we exclude unrealistic (overoptimistic) ones?
 What should the market average bid approximately look like?
 I did a project and the client thinks it was too expensive
 How did I perform against similar projects?
| IT CONFIDENCE - HISTORICAL DATA

4. The answer:
Historical data of
completed projects!!
| IT CONFIDENCE - HISTORICAL DATA

5. ISBSG
 International Software Benchmarking Standards Group
 Independent and not-for-profit
 Full Members are non-profit organizations, like Nesma, DASMA,
IFPUG, FiSMA, Swiss-ICT, JFPUG, CESI.
 Grows and exploits two repositories of software data:
 New development projects and enhancements (> 7.000 projects)
 Maintenance and support (> 1.100 applications)
 Everybody can submit project data
 DCQs on the site
 Anonymous
 Free benchmark report in return
| IT CONFIDENCE - HISTORICAL DATA

6. ISBSG challenges
 Rate of new project submissions is decreasing;
 Projects in new technologies are needed;
 DCQ’s are considered hard to use and time-consuming:
 Many questions;
 Conversion company data to ISBSG data;
 DCQ’s may not be suitable for certain projects:
 Agile projects / sprints/ releases;
 DevOps;
 Mobile apps;
 Cloud transition projects.
| IT CONFIDENCE - HISTORICAL DATA

7. Data collection questionnaires
 IFPUG / NESMA (Full DCQ)
 COSMIC (Full DCQ and Concise DCQ)
 SNAP (Full DCQ)
 Maintenance & Support (online DCQ / Full DCQ)
 Full IFPUG / NESMA DCQ:
 In MS-Word
 143 questions, 43 pages
| IT CONFIDENCE - HISTORICAL DATA

8. Part of DCQ
| IT CONFIDENCE - HISTORICAL DATA

9. Data collection initiatives
 Nesma (2014/2015)
 COSMIC (2011, and now starting 2014)
 NASSCOM (2014/2015)
 ….
| IT CONFIDENCE - HISTORICAL DATA

10. Workshop objectives
 Brainstorm and discuss:
 How can we increase data submissions for the current
repositories;
 What is the best tradeoff between number of questions
asked and data attributes really needed;
 What other types of repositories are needed;
 How would we get data for these?
 What could be alternative ways of collecting data;
 What other ideas can we suggest to the ISBSG
workshop?
 Other topics?
| IT CONFIDENCE - HISTORICAL DATA

11. Increase submissions
| IT CONFIDENCE - HISTORICAL DATA

12. Tradeoff questions / data
| IT CONFIDENCE - HISTORICAL DATA

13. Other types of repositories
| IT CONFIDENCE - HISTORICAL DATA

14. How to get data for these
| IT CONFIDENCE - HISTORICAL DATA

15. Alternative ways to collect
| IT CONFIDENCE - HISTORICAL DATA

16. Other ideas
| IT CONFIDENCE - HISTORICAL DATA

17. Other topics
| IT CONFIDENCE - HISTORICAL DATA

18. wgBenchmarking@nesma.org