Application Development Research: Development vs. Maintenance budget Level of effort in development (requirements, design, code, testing, etc.) Level of effort in maintenance (bugs, new enhancements, infra. upgrades) Development internally, outsourced and package modification Regulation vs. planned vs. unplanned Development metrics QA organization, etc. Methodologies used in development process

1. ;
IT Application Development
Research
Pini Cohen
EVP
pini@stki.info
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic

2. Agenda
• Major Trends and Issues
• Development and SOA
• ESM BSM CMDB
• DBMS and DATA
• Platforms – Servers
• Clients
• Storage Source: http://astonguild.org.uk/files/NEW_MENU_FRONT_RGB%5B1%5D.jpg
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic

3. STKI IT Development Survey
• The first IT development survey in Israel!
• Covering:
– Development vs. Maintenance budget
– Level of effort in development (requirements, design, code, testing,
etc.)
– Level of effort in maintenance (bugs, new enhancements, infra.
upgrades)
– Development internally, outsourced and package modification
– Regulation vs. planned vs. unplanned
– Development metrics
– QA organization, etc.
– Methodologies used in development process
Pini Cohen’s work Copyright 2011 @STKI 3
Do not remove source or attribution from any graphic or portion of graphic

4. STKI IT Development survey
• How should you use this data:
– Look at the fine print – not all graphs are the same
– including or excluding specific data according to
the industry
– If you do not have all data as stated look at the
ratios
Pini Cohen’s work Copyright 2011 @STKI 4
Do not remove source or attribution from any graphic or portion of graphic

5. New Initiatives (“development”) vs.
Running (“maintenance) Systems
New Running
Israeli IT Average Initiatives Systems
Finance - 73% 27%
Banking
Maint. Telecom 71% 29%
43% Finance - 53% 47%
Dev.
Insurance &
57%
Credit Cards
Health 48% 52%
Government 32% 68%
Source: STKI Survey
The actual question was: Which part from
your development budget is directed to
“development vs. maintenance”
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic

6. About Development vs.
Maintenance
• Different companies has different takes.
Example:
– “Development – 7 days or more with features that
contribute to company profitability (not
regulation)”
– “Everything is development except bugs
correction or training support. Updating tables is
development”
– Up to two months works is considered
maintenance.
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic

7. Development vs. Maintenance
definition
Pini Cohen’s work Copyright 2011 @STKI Source: STKI Survey
Do not remove source or attribution from any graphic or portion of graphic

8. Spending from IT budget on
developing new SW projects
All Data
Package
customization
not included
Pini Cohen’s work Copyright 2011 @STKI 8
Do not remove source or attribution from any graphic or portion of graphic Source: STKI Survey

9. STKI IT Development Survey
Requirements,
• Spending from IT Budget 8%
• Testing including
acceptance testing (effort
by IT)
• Development - means Testing, 21%
developers effort and not
necessarily just Design, 19%
development (might
include unit testing and
other testing)
Development,
52%
Source: STKI Survey
Pini Cohen’s work Copyright 2011 @STKI 9
Do not remove source or attribution from any graphic or portion of graphic

10. Per sector
Requirements Design Development Testing
16% 13%
25% 23% 21%
30%
53% 50% 59%
49%
46%
56%
24% 21% 24% 15%
17%
6% 12%
6% 7% 7% 7% 5%
Defence Health Telecom Finance (no Banking PublicGov.
Source: STKI Survey
Banks)
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic

11. STKI IT Development Report
• Public Government
– Spending more on the requirement stage
– Testing looks is much less mature.
– Therefore developers are dealing with testing
– Several organization are on the building stage of their testing
unitsprocedures
• Health: more effort on design. Less effort on Testing
• Telecom: heavy investment in testing without specific budget to
User Acceptance testing
• Finance (not banking): Heavy Management cost. Lots of
investment in Design
Pini Cohen’s work Copyright 2011 @STKI 11
Do not remove source or attribution from any graphic or portion of graphic

12. Requirement Mgmt. Attention
Public 12%
Telecom 7%
Health 7%
Finance (Insurance & Credit Cards) 7%
Defense 6%
Banking 5%
Source: STKI Survey
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic

13. Testing Focus
Factors Influencing QA: Telecom 30%
• Internal SW development
• SW is an integral part of Defense 25%
the org. services to
customers Finance 23%
(Insurance & Credit Cards)
• Regulation
• Established Banking 21%
methodologies (CMMI,
ISO) Health 16%
Public 13%
Source: STKI Survey
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic

14. What “contributes” to Software
maintenance:
• Not all “minor improvements” are the same:
• One company might put all improvement in “development budget” while the other
will put all “new development that is less than two weeks”
• “Infra” is SW adjustment to infrastructure changes – Win7, Oracle 11G, etc.
Source: STKI Survey
Pini Cohen’s work Copyright 2011 @STKI 14
Do not remove source or attribution from any graphic or portion of graphic

15. Who and what is developed?
All including PublicHealth PublicHealth
Source: STKI Survey
Pini Cohen’s work Copyright 2011 @STKI 15
Do not remove source or attribution from any graphic or portion of graphic

16. Development Budget Distribution
Sector Regulation Planned Unplanned
Regulation Finance 30% 47% 23%
15%
Banking 18% 60% 22%
Planned
Unplanned Projects Telecom 18% 56% 26%
Requests 60%
25% Public 15% 60% 25%
Health 6% 75% 19%
Defense 5% 60% 35%
Pini Cohen’s work Copyright 2011 @STKI
Source: STKI Survey
Do not remove source or attribution from any graphic or portion of graphic

17. Metrics used
Project Bugs Faults Depth of Development Other
Management Testing Operations
maturity
Source: STKI Survey
Each respondent could add several metrics
Pini Cohen’s work Copyright 2011 @STKI 17
Do not remove source or attribution from any graphic or portion of graphic

18. Metrics used – what is missing?
Project Bugs Faults Depth of Development Other
Management Testing Operations
maturity
Deployment Hatmaa metrics are missing!
Pini Cohen’s work Copyright 2011 @STKI 18
Do not remove source or attribution from any graphic or portion of graphic

19. Selected Project Management
metrics
• Resource, Time, Features. And keeping track of it.
• LOC – line of code
• Number of mission accomplished in a monthperiod
(normalized on mission size)
• Plan vs. actual in total effort spent per customer,
project
• Net resources vs. overhead (courses, safety, illness-
out of work)
Pini Cohen’s work Copyright 2011 @STKI 19
Do not remove source or attribution from any graphic or portion of graphic

20. Selected Quality (bugfault related)
metrics. All per severity level
• # of bugs in testing and per man year, per K
LOC
• # of escaping bugs (limited time in production)
and per man year, per K LOC
• # of escaping bugs per # of bugs found in
testing
• First time quality – in testing and in
production
Pini Cohen’s work Copyright 2011 @STKI 20
Do not remove source or attribution from any graphic or portion of graphic

21. Selected Quality (bugfault related)
metrics. All per severity level
• # of critical bugs per total bugs found in
testing and in production
• # of requiring bugs (bugs that were not fixed
at after first time)
• # of false positive bugs – bugs that were
reported but actually did not exist
Pini Cohen’s work Copyright 2011 @STKI 21
Do not remove source or attribution from any graphic or portion of graphic

22. Selected testing breath metrics
• Percent of code tested
• Percent of code with automatic tests
Pini Cohen’s work Copyright 2011 @STKI 22
Do not remove source or attribution from any graphic or portion of graphic

23. Selected dev.production process
metrics
• How many changes were introduced to a project
while being developed (not relevant to Agile)
• Did the project followed all procedures
• How many times the project was put to
production (if more than once- something went
wrong…)
• # of LOC developed per specific feature
(comparing two developing
environmentsupporting tools)
Pini Cohen’s work Copyright 2011 @STKI 23
Do not remove source or attribution from any graphic or portion of graphic

24. Other development metrics
• ISO metrics
• Maturity of reporting systems –how well
people report to the activitybug system (is
reporting accurate, how often the reporting
takes place, what is the delay between activity
and the reporting).
Pini Cohen’s work Copyright 2011 @STKI 24
Do not remove source or attribution from any graphic or portion of graphic

25. Organization Position of QA
Office of the
CIO
11%
QA
Development Department
Department 50%
39%
Source: STKI Survey
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic

26. Show me the QA money!
• When QA is part of development the testing
budget is lower – 38% difference! (at average)
Source: STKI Survey
Pini Cohen’s work Copyright 2011 @STKI 26
Do not remove source or attribution from any graphic or portion of graphic

27. Profile of your testing personnel
Rising interest in Israeli Nearshore due to their subsidization by the Gov.
Nearshore
19%
Internal
Staff
Professional 57%
Services
24%
Source: STKI Survey
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic

28. Which Methodology is used
• Counted even if used in few projects or
experimenting (for example “using iterative
only in Internet projects” or “experimenting
agile in small projects”
• CMMI is more common in Banking
• ISO certifications used are ISO9001:2008 and
9002.
Pini Cohen’s work Copyright 2011 @STKI 28
Do not remove source or attribution from any graphic or portion of graphic

29. Methodologies in use
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic
Source: STKI Survey

30. Thank you
Pini Cohen
Pini Cohen’s work Copyright 2011 @STKI 30
Do not remove source or attribution from any graphic or portion of graphic