This document discusses requirements engineering and its importance in software project success. It defines requirements engineering and outlines the key processes: elicitation, analysis, specification, verification and management. Case studies show that projects with strong requirements engineering in areas like user involvement, clear requirements and proper planning are more likely to succeed. The document concludes that requirements engineering impacts 7 of the top 10 attributes that determine project success.
3. Cobb’s Paradox
"We know why projects fail, we know how to
prevent their failure -- so why do they still fail?”
Martin Cobb
Treasury Board of Canada Secretariat
Ottawa, Canada
9. Top Ten Project Success Factors
1. user involvement
2. executive management support
3. clear statement of requirements
4. proper planning
5. realistic expectations
6. smaller project milestones
7. competent staff
8. ownership
9. clear vision and objectives
10. hard-working, focused staff
10. Properties of Challenged Projects
LackofExecutive
Support
12.8 12.3 11.8
7.5 7 6.4 5.9 5.3
4.3 3.7
23
0
5
10
15
20
25
Other
LackofUser
Involvement
Inc.
Requirements&Specs
Technology
Incompetence
Unrealistic
Expectation
LackofResources
Changing
Requirements&Specs
Unclear
ObjectivesUnrealisticTimeFrame
NewTechnology
11. Top Ten Challenged Project Factors
1. Lack of user involvement
2. Incomplete requirements and specifications
3. Changing requirements and specifications
4. Lack of executive support
5. Technology Incompetence
6. Lack of Resources
7. Unrealistic expectations
8. Unclear objectives
9. Unrealistic timeframe
10. New technology
13. Top Ten Impaired Project Factors
1. Incomplete requirements
2. Lack of user involvement
3. Lack of resources
4. Unrealistic Expectations
5. Lack of executive support
6. Changing requirements & specs
7. Lack of planning
8. Didn’t need anymore
9. Lack of IT management
10. Technology illiteracy
14. Case Studies
FAILED SUCCESS
SUCCESS CRITERIA POINTS DMV AA HYATT BANCO
1. User Involvement 19 NO (0) NO (0) YES (19) YES (19)
2. Executive Management Support 16 NO (0) YES (16) YES (16) YES (16)
3. Clear Statement of Requirements 15 NO (0) NO (0) YES (15) NO (0)
4. Proper Planning 11 NO (0) NO (0) YES (11) YES (11)
5. Realistic Expectations 10 YES (10) YES (10) YES (10) YES (10)
6. Smaller Project Milestones 9 NO (0) NO (0) YES (9) YES (9)
7. Competent Staff 8 NO (0) NO (0) YES (8) YES (8)
8. Ownership 6 NO (0) NO (0) YES (6) YES (6)
9. Clear Vision & Objectives 3 NO (0) NO (0) YES (3) YES (3)
10. Hard-Working, Focused Staff 3 NO (0) YES (3) YES (3) YES (3)
16. Systems Requirements Engineering Lifecycle
User
Requirements
System
Requirements
System
Architecture
User
Requirements
User
RequirementsComponent
Development
Integration
Test
Acceptance
Test
System
Development
Capability
Development
Component
Development
18. Requirements Engineering
R e q u ir e m e n t s E lic it a t io n R e q u ir e m e n t s A n a ly s is
R e q u ir e m e n t s S p e c ific a t io n R e q u ir e m e n t s V e r if ic a t io n
R e q u ir e m e n t s M a n a g e m e n t
R e q u ir e m e n ts E n g in e e r in g
19. Requirements Elicitation
Software
Requirements
Identify relevant sources of requirements
(usually customer)
Determine what information is needed.
Analyze the gathered information, looking for
implications, inconsistencies, or unresolved
issues
Confirm your understanding of the
requirements with the source
Synthesize appropriate statements of the
requirements
20. Outcome of good elicitation activities
–The buyer/customer fully explore and fully understand their
requirements.
–The buyer/customers are able to separate their wants from their
needs.
–The buyer/customers are able to understand the capabilities
and limitations of computer technology.
–The buyer/customers understand the alternative solutions and
impact of each alternative.
–The buyer/customers understand the impact of the
requirements on the developer and themselves.
–The developers are solving the right problem.
–The developers have confidence that the system to be delivered
is feasible to build.
–The developers have the trust and confidence of the customer.
–The developers gain domain knowledge of the system
21. Outcome of poor elicitation effort
–The customer probably will be dissatisfied.
–The customer and developer have to cope with
constantly changing requirements.
–The developer is solving the wrong problem.
–The developer has a difficult time building the system.
22. Requirements Elicitation
User Involvement Criteria2
13.9
13
9.6
8.2 7.7 7.2
5.3
2.9 2.4
13.9
15.9
0
2
4
6
8
10
12
14
16
18
Other
User
Involvem
en
t
Executive
Management
Support
Proper
PlanningRealistic
Expectation
CompetentStaff
SmallerProject
Milestones
ClearStatement
ofRequirements
OwnershipClearVision
andObjectivesHard-Working
FocusedStaff
Project Success Factors
Do I have the right user(s)?
Did I involve the user(s)early
and often?
Do I have a quality user(s)
relationship?
Do I make involvement
easy?
Did I find out what the
user(s) needs?
Software
Requirements
23. Requirements Analysis
Obtain Requirements from all possible
sources (include but not limited to):
–observation and measurements of the
current system
–interviews with customers
–current system documentation
–feasibility studies
–models and prototypes
–competitive analysis
Software
Requirements
24. Quality attributes
Correctness - Extent to which a program satisfies its specifications and fulfills the user’s mission objectives.
Reliability - Probability that the software will perform its logical operation in the specified environment without failure.
Efficiency - Degree of utilization of resources in performing functions.
Integrity - Extent to which unauthorized access to the software or data can be controlled.
Usability - Effort for training and software operation - familiarization, input preparation, execution, output, interpretation.
Survivability - Probability that the software will continue to perform or support critical functions when a portion of the system is
inoperable.
Maintainability - Average effort to locate and fix a software failure.
Verifiability - Effort to verify the specified software operation and performance.
Portability - Effort to convert the software for use in other operating environments.
Reusability - Effort to convert a software component for use in another application.
The analysis group has a tough task to take the huge picture of data and whittle down to the first 12 months without precluding
possibility of the big picture over time. Analysis process generates more questions which requires going back to the customer. More
efficient for analysts to go to customer rather than elicitation team redoing a part.
26. Statement of Requirements Criteria
Software
Requirements
Do I have a concise vision?
Do I have a functional analysis?
Do I have a risk assessment?
Do I have a business case?
Can I measure the project?
13.9
13
9.6
8.2 7.7 7.2
5.3
2.9 2.4
13.9
15.9
0
2
4
6
8
10
12
14
16
18
Project Success Factors
27. Requirements Verification
To identify and resolve software
problems and high risk issues early in
the software cycle.
The assurance that the software
requirement specification is in
compliance with the system
requirements, conforms to document
standards, and is an adequate basis for
the architectural design.
Integration &
Verification
28. Requirements Management
Basic responsibility is to keep project within
costs, within budget, and to meet customers
needs.
Estimate cost of system based on
requirements.
Control the volatility of the requirements.
Manage the requirements configuration of the
system
Negotiate requirement changes
Re-estimate cost of the system when
requirements change.
Software
Requirements
33. Requirements Engineering Conclusion
Software
Requirements
Architectural
design
Detailed design
& coding
Integration &
Verification
User
RequirementsUser
RequirementsUser
RequirementsComponent
Development
Software Requirements Engineering
includes:
–elicitation
–analysis
–specification
–verification and validation
–management of requirements
development process
Affects 7 of 10 attributes of
successful projects
34. References
1 The Standish Group, Chaos, January 1995
2 The Standish Group, Unfinished Voyages, September 1995
3 Software Quality Measurement for Distributed Systems,
RADC-TR-83-175
4 Requirements Engineering, Thayer, SMC 10/97, version 2
5 Richard Thayer, Software Requirements Engineering, IEEE,
1997
6 STEP, Operational Requirements for Automated Capabilities,
STEP, 1991
7 MBASE, “Avoiding the Software Model-Clash Spiderweb,”
IEEE Computer, November, 2000, pp. 120-122.
Editor's Notes
Like anything with sw eng, we are trying to solve a problem.
From Unfinished Voyages. There is a decently well-established set of success criteria. We know how to fix them but why don’t we? Requirements engineering satisfies quite a few of those areas.
Give background on where the following information came from. Resolution Type 1, or project success: The project is completed on-time and on-budget, with all features and functions as initially specified. Resolution Type 2, or project challenged: The project is completed and operational but over-budget, over the time estimate, and offers fewer features and functions than originally specified. Resolution Type 3, or project impaired: The project is canceled at some point during the development cycle. Overall, the success rate was only 16.2%, while challenged projects accounted for 52.7%, and impaired (canceled) for 31.1%. They looked at 365 respondents and represented 8,380 applications. They divided them into these categories, small medium and large companies small: less than $100,000 annual revenue large: greater than $500,000,000 medium: in between Included the gamut of applications and companies. Basically, the division was like this regardless of size. Large companies had more impaired than challenged; smaller ones had more challenged than impaired. Perhaps this was because the small company’s life depended on it and they had to deliver SOMEthing. Unfinished Voyages shows that more and more projects fall into challenged and impaired category.
Three controls we have -- Cost overruns for challenged and impaired projects. Over 50% chance of being over 50% over budget. Time overruns -- 50% chance of being 100% over the time you estimated. Functionality - only 7% of challenged projects deliver the original functionality. 50% chance or more that you are only going to deliver 50% of the originally planned functionality.
Previous slide’s comments repeated here: Three controls we have -- Cost overruns for challenged and impaired projects. Over 50% chance of being over 50% over budget. Time overruns -- 50% chance of being 100% over the time you estimated. Functionality - only 7% of challenged projects deliver the original functionality. 50% chance or more that you are only going to deliver 50% of the originally planned functionality.
See comment in notes, two slides ago. Three controls we have -- Cost overruns for challenged and impaired projects. Over 50% chance of being over 50% over budget. Time overruns -- 50% chance of being 100% over the time you estimated. Functionality - only 7% of challenged projects deliver the original functionality. 50% chance or more that you are only going to deliver 50% of the originally planned functionality .
One of the best things that came out of Chaos was the top ten reasons that made projects successful. Requirements engineering includes: (the list is on the next slide) 1. user involvement 2. executive management support 3. clear statement of requirements 4. proper planning 5. realistic expectations 6. smaller project milestones 7. competent staff 8. ownership 9. clear vision and objectives 10. hard-working, focused staff If you have all of these, high probability of success.
These correlate well as the opposite of the success factors. 1. Lack of user involvement 2. Incomplete requirements and specifications 3. Changing requirements and specifications 4. Lack of executive support 5. Technology Incompetence 6. Lack of Resources 7. Unrealistic expectations 8. Unclear objectives 9. Unrealistic timeframe 10. New technology
Almost the same list as “challenged” except for a greater emphasis on incomplete requirements and lack of user involvement. 1. Incomplete requirements 2. Lack of user involvement 3. Lack of resources 4. Unrealistic Expectations 5. Lack of executive support 6. Changing requirements & specs 7. Lack of planning 8. Didn’t need anymore 9. Lack of IT management 10. Technology illiteracy
Standish Group examined case studies. 1 California DMV automation of the driver’s license. The only one they met was realistic expectations. They didn’t get users involved, etc. The hardware and software didn’t work on the same system. $150M down the drain. American Airlines Early in 1994, American Airlines settled their lawsuit with Budget Rent-A-Car, Marriott Corp. and Hilton Hotels after the $165 million CONFIRM car rental and hotel reservation system project collapsed into chaos. This project failed because there were too many cooks and the soup spoiled. Executive management not only supported the project, they were active project managers. Of course, for a project this size to fail, it must have had many flaws. Other major causes included an incomplete statement of requirements, lack of user involvement, and constant changing of requirements and specifications. Hyatt Hotels While Marriott and Hilton Hotels were checking out of their failed reservation system, Hyatt was checking in. Today, you can dial from a cellular airplane telephone at 35,000 feet, check into your Hyatt hotel room, schedule the courtesy bus to pick you up, and have your keys waiting for you at the express desk. This new reservation system was ahead of schedule, under budget, with extra features -- for a mere $15 million of cold cash. They used modern, open systems software with an Informix database and the TUXEDO transaction monitor, on Unix-based hardware. Banco -- wanted to consolidate acquired banks’ banking systems into one system, approximately 19 systems. Had the criteria and were on time, on budget, and delivered. They used a prototyping technique rather than written SOR.
SYSTEMS Requirements Engineering Lifecycle Components may be hardware, may be software, may be sub-systems, may be smaller Talk through the arrows while mouse-clicking In incremental development, which portions of this “lifecycle” are executed?
In incremental development, which items are executed in an increment? How many times?
These are the five activities involved in software requirements engineering. Given what’s been said about incremental development, it should be obvious by now that these are all done throughout the project on various requirements.
Identify relevant sources of requirements: What are sources of requirements? customer -- various roles are all “the customer” maintenance test strategic planning for product family ease of demo etc. Analyze gathered information -- we’ll look at various techniques to highlight inconsistencies, missing items Confirm your understanding -- this step is done in various ways with various development methods. For example in various agile methods, there is an emerging product available to demo at the end of each increment. Synthesize appropriate statements
Requirements may change anyway -- but if the elicitation was done poorly, they will change constantly as you gradually understand each other, rather than changing for less frequent reasons such as market influences.
Getting the right requirements absolutely requires talking to the right users. Need to have ongoing relationship with the users, make them feel their input and time and trouble really made a difference in your findings. Must make a difference between the needs and wants. Sometimes you don’t know until the crisis happens. Takes skill to find what the real problem is and solve it in a way that allows them to do their job.
Software Quality Attributes and Metrics 3 Correctness - Extent to which a program satisfies its specifications and fulfills the user’s mission objectives. Reliability - Probability that the software will perform its logical operation in the specified environment without failure. Efficiency - Degree of utilization of resources in performing functions. Integrity - Extent to which unauthorized access to the software or data can be controlled. Usability - Effort for training and software operation - familiarization, input preparation, execution, output, interpretation. Survivability - Probability that the software will continue to perform or support critical functions when a portion of the system is inoperable. Maintainability - Average effort to locate and fix a software failure. Verifiability - Effort to verify the specified software operation and performance. Portability - Effort to convert the software for use in other operating environments. Reusability - Effort to convert a software component for use in another application. The analysis group has a tough task to take the huge picture of data and whittle down to the first 12 months without precluding possibility of the big picture over time. Analysis process generates more questions which requires going back to the customer. More efficient for analysts to go to customer rather than elicitation team redoing a part.
Correctness - Extent to which a program satisfies its specifications and fulfills the user’s mission objectives. Reliability - Probability that the software will perform its logical operation in the specified environment without failure. Efficiency - Degree of utilization of resources in performing functions. Integrity - Extent to which unauthorized access to the software or data can be controlled. Usability - Effort for training and software operation - familiarization, input preparation, execution, output, interpretation. Survivability - Probability that the software will continue to perform or support critical functions when a portion of the system is inoperable. Maintainability - Average effort to locate and fix a software failure. Verifiability - Effort to verify the specified software operation and performance. Portability - Effort to convert the software for use in other operating environments. Reusability - Effort to convert a software component for use in another application. The analysis group has a tough task to take the huge picture of data and whittle down to the first 12 months without precluding possibility of the big picture over time. Analysis process generates more questions which requires going back to the customer. More efficient for analysts to go to customer rather than elicitation team redoing a part.
Considerations for use of a Requirements Specification Standard. Why use a standard at all? Allow consumers and verification agents the ability to determine if the requirement specification meets their needs easier. The ability to repeat the process again. Should IEEE 830-93 be used? If there is nothing currently in place absolutely. There has been a great deal of time and effort by requirements professionals developing the standard. There should be a great deal of effort to avoid re-inventing the wheel. (Design constraints -- explicitly tells why must be done a certain way.)
For all of the success criteria, Unfinished Voyages report (which is a followup to the Chaos report) asked five questions to allow you to evaluate whether you are meeting the success criteria. Here’s my Requirements Specification -- which ones are risky? Probably the ones you have never done before because you may not have the right staff with the right competence. In requirements engineering, we are showing progress as one measure of the project. Requirements engineering is intertwined with project management.
You want to make sure that people can really use the requirements specification before you throw it over the wall to the developers. Many people involved in this don’t know when they are done. To identify and resolve sw problems and high risk issues early in the software cycle. early in the software cycle early in each increment and so ad infinitum
This is essential to the project manager being able to do the job of project management. With the requirements specification, you can do the kinds of things the project manager needs to be able to do -- cost estimation, etc. Once it’s been approved, there must be real proof that a change is necessary. (Consider cost as time impact on delivery date.) From “control the volatility” down to re-estimation -- those tasks need to be rolled into configuration management (change management).
This is an overall framework, not a methodology , for categorizing activities and showing progress. Requirements Elicitation : The process through which the customers and the developer of a software system discover, review, articulate, and understand the users’ needs and the constraints on the software and the development activity. 5 Requirements Analysis : The process of analyzing the customers’ and users’ needs to arrive at a definition of software requirements. 5 Requirements Specification : The development of a document or set of documents that clearly and precisely records each of the requirements of the software system. 5 Requirements Verification : The process of ensuring that the software requirements specification is in compliance with the system requirements, conforms to document standards of the requirements phase, and is an adequate basis for the architectural (preliminary) design phase. 5 Requirements Management: The planning and controlling of the requirements elicitation, specification, analysis, and verification process. 5 Three main activities drive the work. Validation and management support the three main ones. One team moves thru phases of the three main activities. Re-emphasize: This is an overall framework, not a methodology , for categorizing activities and showing progress.
It is important to deliver concentric circles of functionality where one circle is the foundation for the next release. With each circle, need to go back and validate that the requirements are correct. Requirements have a half life so you need to confirm that old requirements are still requirements.
If you try to deliver the project on 2 year boundaries, you tend to be a year late. So much stuff got put into that, expected to absorb so much, the 24 month original window became 36 months. Rule of thumb, after requirements are approved, first release should be 2*N units out and then add functionality every N units after that. In order to do this, you MUST prioritize the requirements into buckets. P1 -- without this, it is not sellable. P2 -- critical P3 -- important/should have P4 -- nice to have
What is NOT on that list from the top ten? 2. executive management support 4. proper planning 5. realistic expectations 6. smaller project milestones 9. clear vision and objectives 10. hard-working, focused staff -- requirements enable staff to focus and proceed; no interruptions or re-directions during an increment