Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The waterfall, a commonly misapprehended methodological concept

103 views

Published on

Waterfall as methodological concept; right usage of methodologies; purpose of phases; possibilities for flexibility, continuous improvement, scalability, fast delivery, iterations, continuous testing, advantages & disadvantages, ...

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

The waterfall, a commonly misapprehended methodological concept

  1. 1. The Waterfall A Commonly Misapprehended Powerful Methodological Concept This presentation describes a different picture of the widely spread caricature of the ‘waterfall’. By going deeper into this subject and by explaining right concepts and their right/better usage a glimpse of its true power is revealed. Axel Vanhooren Freelance Consultant Business Informatics & IS Methodologies https://be.linkedin.com/in/axelvanhooren Version 1.2 19/05/2016 Copyright©2016 Copyright ©2016
  2. 2. Content  Part 1: General Notes on Waterfall and Methodologies  Part 2: Phases  Part 3: Principles  Part 4: Structuring, Organising and Planning  Part 5: Various Aspects  Part 6: Critics and Myths  Part 7: Brief Evaluation 21-May-16Axel Vanhooren 2 For the latest version: http://www.taurus-ee.com/Publications/TEE - The Waterfall – A Commonly Misapprehended Methodological Concept.pdf
  3. 3. 21-May-16Axel Vanhooren 3 General Notes Background knowledge to draw a common context
  4. 4. What is the Waterfall ?  A methodology?  A standard?  A philosophy?  …. 21-May-16Axel Vanhooren 4
  5. 5. What is the Waterfall ?  No Official Standard !! (AFAK)  No official number of phases  No official names of phases  No official rules or principles  …. 21-May-16Axel Vanhooren 5  Since there is no standard, we have to be cautious with statements expressing rules, obligations, prohibitions emanating from “the waterfall”.  These rules, obligations, … simply don’t exist.
  6. 6. What is the Waterfall ? 1. Waterfall is a Methodological Concept  We know the model (= core idea of waterfall)  Many methodologies implement this model  Every methodology has its own structure, its own rules, …  Common element in these methodologies is the model/concept 2. Waterfall = type / family of methodologies Family of methodologies implementing the model 21-May-16Axel Vanhooren 6 Conclusion: WATERFALL = METHODOLOGICAL CONCEPT (is also the systems development lifecycle)
  7. 7. Right Usage of a Methodology  A methodology serves to facilitate (to guide, to save time, to make more efficient and effective, …) the project execution  The success of the project must take precedence over “following a methodology". Never follow a methodology blindly. A methodology is not a procedure.  Each projects is different, different environment, different type of system  will never fit a specific project  A methodology offers, suggests, proposes, …like a template  A methodology needs ALWAYS to be adapted to the project.  A methodology doesn’t bare the responsibility. People make choices and take decisions. They need to know what they do and act wisely. Do everything what is required to ensure project success, and only that.  A methodology NEVER obliges, dictates, imposes, prohibits, …  A methodology does in no way replace discipline knowledge  People need to master their discipline  The goal of a methodology IS NOT & CANNOT be to increase costs, to provide more work, to impose rules against the project success, to increase risks, .... NEVER !! If so, don’t throw it away, but adapt it, use it wisely and improve it. 21-May-16Axel Vanhooren 7
  8. 8. IT Challenges  Bunch of features  Small Single User Application  Website  waterfall methodology = unsuitable  Real challenges require a methodology  Critical systems  Larger, more complex systems for corporate environments 21-May-16Axel Vanhooren 8 No real challenge  no great need for a methodology
  9. 9. Different Systems – Different Approaches 21-May-16Axel Vanhooren 9 User Interface Arch. Business Logic Business Logic User Interface Architecture Technical logic Business logic Architecture User Interface The main emphasise is on: User Interface: Needs a lot of interaction with end-users, visualisation, … Architecture: Needs a vision based, top-down approach Technical logic: More a back-office development These are just a few examples…. Other factors influencing the choice of the approach/methodology: size of system, importance of information & information processes, complexity, security, networked system, agent-based systems, stability/variability of the environment, expected lifetime of the system, nature of problem to be solved, number of end-users groups, priorities and urgency, criticality, technology, …
  10. 10. 21-May-16Axel Vanhooren 10 PHASES Purpose of phases, type of phases, relation between phases and activity types
  11. 11. THE PHASES  To better understand the role of the phases, the following slides presents them  by starting to look at what is required to develop a small software application.  When the software applications become larger, developers will meet some problems. Phases are added to solve these specific problems. This reveals their purpose.  Phases are not presented in the order of execution. 21-May-16Axel Vanhooren 11
  12. 12. Programming  The actual activity of building the software application  Tiny software applications can be built by starting programming right away No formal analysis or design required 21-May-16Axel Vanhooren 12
  13. 13. Design  Problem: As the developer progresses in the programming of larger software applications, (s)he will find better ways to organise the screens, better ways to structure the databases, better ways to organise functions and features and to structure the source code organisation  Problem: a lot of rework, higher cost, longer development time Solution: Thinking about how the software application will do things and how this will be organised – Doing things Right  Programming w/o design isn’t not efficient for larger software applications.  Getting ideas to mature and to settle ideas before programming  To align opinions and obtaining general agreement before programming  The design defines the product  clear picture of the product to be built  Design helps to communicate  Allows to plan and to organise the project (resources, parallel development)  Design allows better estimates (better than the analysis does) 21-May-16Axel Vanhooren 13
  14. 14. Testing  Problem: In larger the software system the risk for wrong functional logic and bugs is real. Problems and bugs aren’t desired in production environments. Solution: Testing = “What we made, did we made it right ?”  Goal of the Test phase (in project): Last opportunity to prevent problems and bugs to be introduced in production. Verification if the system functions as it is functionally meant to. It’s about discovering technical issues: integration issues, programming bugs, performance, …  Testing happens at every step during the whole project. Testing happens also outside the “Test phase”. Testing starts even before the project is launched (business case, project proposal, …)  Not the goal of the Test Phase:  to know if the right problem is being solved or whether the right features have been implemented. This should have been tested much earlier. However, such errors can still be discovered, but this is late. Functional/conceptual errors should be detected earlier than the Test phase.  A “product presentation phase”. Presenting the product to the business stakeholders should be organised earlier and differently. 21-May-16Axel Vanhooren 14
  15. 15. Analysis  Problem: The design is about how to solve a problem. The design can be right. In complex environments, or complex problems, we may find out during the testing or later that we solved the wrong problem or that we even created new problems. Solution: Analysis ensures we do the right things.  This is about what has to be done. It ensures no wrong solution is being built.  If the wrong solution has been built or if new problems are created, it’s the analysis that failed ! (not necessarily the fault of the analyst)  Analysis is about  Identifying problems and opportunities  Studying the situation and the context  Determining  what the information system or information processes should do, how to organise data, …  the requirements and constraints  how will all this fit into the enterprise  how this will add value, contribute to the objectives, make the enterprise stronger, …  Analysis allows ideas to mature, to stabilise, to settle before building 21-May-16Axel Vanhooren 15
  16. 16. Analysis (2)  Analysis is not:  Decomposing something into parts and studying the parts and the relations among these parts  A set of activities  Translating a business demand into artefacts for developers  Refining a demand received from the business community  A way to make sure the business demand is developed  An interface between business and the IT project team ( a bridge)  Gathering and managing requirements (a Requirements Collector/Recorder/Manager)  Documenting software feature (feature specialist)  The translation of requirements into specifications (a translator)  Producing UML-models (an UML-ist; BPMN-ist, …) Everything above combined still misses the essence of the Analysis discipline 21-May-16Axel Vanhooren 16
  17. 17. Some Other Phases, Other Names  Other project phases can be added, split up, renamed:  Preliminary analysis  Architectural Design  Requirements Analysis  Business Analysis  Functional Analysis  Systems Design  Technical Design  Integration  Implementation  Deployment  Evaluation  … 21-May-16Axel Vanhooren 17
  18. 18. Activity, Set of Activities & Project Phase  The Activity  Analysis, Design, Programming, Testing, … can be seen as the very act of analysis, of creating the models, of writing code, ….  Example: Testing  Performing the test  test activity itself  mind is busy with testing.  Producing test data  supporting activity of testing  mind is busy with getting data, not with testing  Set of Activities  Perspective: Software Development Process  Performing the ‘Analysis’ consists of different activities like planning the analysis, gathering information, organising workshops, modelling, verifying the information and the understanding. These activities are required to obtain a certain outcome: “the analysis” or belong to a same discipline. Same for Design’, ‘Programming’, ‘Testing’ & ‘Implementation’  Activities are often grouped in the WBS  Can be called softw.dev. phases, methodology phases, SDLC phases  Project Phase  Perspective: Project Management  A phase is an organisational view on the project process defining periods during which the project’s main focus is on producing an certain outcome (analysis, design, ..) & indicating the main set of activities being performed to produce this outcome. They can be ‘separated’ by a gate, like test, validation or sign-off.  Phases are an organisational concept. They depict more or less the nature of activity the project is busy with, not really its progress. It is not a progress measuring tool! 21-May-16Axel Vanhooren 18
  19. 19. Activity, Set of Activities & Project Phase 21-May-16Axel Vanhooren 19 Analysis Real analysis activity Other activities of the Analysis discipline: • analysis planning • information gathering • interview • workshops • … Analysis TestingDesign Building Impl. Analysis Design Testing Project Phases Sets of activities Even though the project is in a phase, activities of another sets of activities can are executed. Example: During the project phase ‘Analysis’, some design activities and testing activities have already been started. Some analysis activities continue when the project is in the ‘Design’ phase. Or, if we call a “set/type of activity” a ‘phase’ (at softw.dev.level), then softw.dev phases are not the same as project phases. Set of activities
  20. 20. Historical Evolution 21-May-16Axel Vanhooren 20 Programmer = End-user Programmer End-User Programmer End-UserSystems Analyst Programmer End-UserSystems DesignerFunctional Analyst Analyst - Programmer End-User Programmer End-UserFunctional AnalystBusiness Analyst Systems Designer Process Analyst Information Architect Solution Architect IT Architect Information Analyst Enterprise Architect Application ArchitectTime Evolution: 1) Specialisation corresponding to the types of activities 2) Broadening upwards (with roles like ‘architect’) The evolution of the roles, the specialisation that happened, matched the ‘phases’ of the waterfall. The split up of roles into different roles comes from the need of specialisation. Is merging roles the way to go? However, collaboration among the roles is certainly useful. Note: The schema may not be 100% correct, but the essence is the tendency.
  21. 21. 21-May-16Axel Vanhooren 21 PRINCIPLES Explains the 2 most fundamental principles of the waterfall concept
  22. 22. Waterfall Based on 2 Main Principles  Principle 1 1. Know what problem to solve 2. Know what the solution should do to solve it 3. Know how the solution will solve it 4. Build the solution 5. Test 6. Implement  Principle 2:  Do the right things right from the first time 21-May-16Axel Vanhooren 22
  23. 23. Main Principle 1 21-May-16Axel Vanhooren 23 Analysis Design Building Testing Implementation Diagnose Learn Solve Evaluate Think Conceive Problem Solving Process Waterfall Process • Business/Functional Analysis also include the design of the conceptual solution (functional design) • The evaluation and the cycle are not included in the basic waterfall concept. • Evaluation is often done outside the project process. (see suggested waterfall process) • The improvement cycle is naturally performed by re-iterating the process. Waterfall concept mapped on the problem solving process Iterate if not solved correctly
  24. 24. The Fundamental Waterfall Concept 21-May-16Axel Vanhooren 24 Analysis Design Building Testing Implementation CLARIFICATION OF THE CONCEPT Every piece of logic should have passed through the sequence. • Something that hasn’t been tested, shouldn’t be implemented. • A piece of a software application that hasn’t been programmed, can’t be tested (this concerns software tests, and not, for example, testing requirements ) • It is not efficient to program a (larger) software application without having first a design. • It is ineffective to start design a solution if the cause hasn’t been diagnosed and the problem and its surrounding not understood.
  25. 25. The Fundamental Waterfall Concept  “Piece of logic”  For corporate systems, it has no sense to do the analysis feature by feature. This would increase the risk for bad overall design (architecture) and for inconsistency.  The project must ensure that each piece fits into the larger whole. Simply repeatedly adding pieces without global vision, global architecture and plan is, in the long term, usually a recipe for chaos.  The piece of logic is usually much larger (a whole system, a component, a set of processes, …)  During the process, the logic is expressed in source code. 21-May-16Axel Vanhooren 25
  26. 26. The Fundamental Waterfall Concept 21-May-16Axel Vanhooren 26 Analysis Design Building Testing Implementation Jumping back to a previous activity, respects the waterfall sequence Analysis  Design  Analysis  Design In the end, analysis always preceded design and the logic flown through the process in the right direction. Building  Testing  Building  Testing  Implementation This sequence is normal for correcting bugs. No untested or uncorrected software application will be put in production.
  27. 27. Main Principle 2  Do the right things right from the first time  The fastest and the cheapest way to solve a problem is to solve it correctly from the first time  Avoiding as much as possible change that could have been avoided.  Avoiding the volume of rework  Minimising rework  Rework should happen early in the project  Rework should happen in the cheaper activities 21-May-16Axel Vanhooren 27 Early detection of mistakes Avoid making mistakes
  28. 28. Main Principle 2  This principle is an unachievable goal  But the intention is to do a genuine effort  There will be change, there will be rework  Implication: Dealing with cases of imperfections (like aspects not well analysed, missed or wrong requirements, …) is part of the process  re-execute a former type of activity  How? This is a trade-off  Doing too little and superficial job  problems later, rework  Trying to reach perfection  paralysis  much higher cost, much longer delay, no benefit  Try to reach 80%, 90%, 95% & deal with the remainder 20% to 5%  This is where professionalism, experience, sound judgement, guts feeling come into play 21-May-16Axel Vanhooren 28
  29. 29. Not Doing the Right Things Right from the First Time (1) 21-May-16Axel Vanhooren 29 Analysis Design Building Testing Implementation Programming • Bug correction • Correction of impact of bugs • Re-testing the bugged/corrected areas Design • Re-Design (at least partly) • Reprogram the software application (at least partly) • Re-testing software application (at least partly) • Correction of impacts of the wrong solution • Testing to verify if the impacts have been well solved Analysis Can range from • correcting some source code; • re-design the solution; • to, in case of wrong diagnosis of root cause and wrong understanding of the problem, halting the project and restart a new one to solve the real problem Consequences if we don’t do the right thing in a set of activity This is the impact in work in softw. dev.. There is also an impact in project management, in budget, in cost/benefit, in reputation, inter-group relations, ...
  30. 30. Not Doing the Right Things Right from the First Time (2) 21-May-16Axel Vanhooren 30 Problem 1 Impact 1 Faulty Product Problem 2 Problem 3 Problem 4 delivers Impact 3 Impact 2 Impact 4 Impact 5 Problem 1 ? Problem 2 Problem 3 Problem 4 Impact 1 Impact 2 Impact 3 Impact 4 Impact 5 Right solution or faulty product ? delivers Rework & additional work First attempt Second attempt Softw. Dev. Process
  31. 31. Waterfall Models 21-May-16Axel Vanhooren 31 These are all waterfalls Analysis Design Building Testing Implementation time Analysis Design Building TestingImplementation Evaluation Analysis Design Building Testing Implementation
  32. 32. 21-May-16Axel Vanhooren 32 STRUCTURING, ORGANISING AND PLANNING Presents the richness of waterfall-type methodologies
  33. 33. Releases 21-May-16Axel Vanhooren 33 A D B T I A D B T I Release 1 Release 2 A D B T I A D B T I Release 1 Release 2 Longer delay between two implementation Shorter delay Release 3, … [ A ]nalysis [ D ]esign [ B ]uilding [ T ]esting [ I ]mplementation
  34. 34. Sub-Projects – Scaling the Waterfall 21-May-16Axel Vanhooren 34 A D B T I Parallel tracks can be sub-projects B T I A D B T Integration B T IT A B T Integration B T IT D D A B T Integration B T IT D D A A
  35. 35. Sub-Projects – Scaling the Waterfall 21-May-16Axel Vanhooren 35 These are a few combinations showed with only 2 sub-projects. More sub-projects are possible. Many other combinations are possible. Combinations with releases at sub-project-level and with additional project phases. A B T Integration B T IT D D A B T Integration B T IT D D A A D D
  36. 36. In which ‘phase’ is this project? 21-May-16Axel Vanhooren 36 Analysis Sub-project 1 Sub-project 2 Integration B T I Testing D A B T D Sub-project 1 Sub-project 2 X X X What is the Phase of the project at this point? • Analysis? • Design? • or Building?
  37. 37. Waterfall Doesn’t Concern Project Phases  For ease, projects are divided in phases with same names as the activities  Confusion  Project phases  project management  Waterfall  oriented to purpose of activities in Softw. Dev.  Tasks of different types are executed within a same phase  Example: “The project is in the design phase, but there are still some analysis activities going on and we already started some programming.”  Testing, W-model  testing activities during the whole project duration  A project can be divided into subprojects  Each sub-project has its own phases and these phases may differ from the overall project.  overlapping phases  Possibility to do exactly the same within a project without defining sub-projects. What matters is the nature of the practical activities, not project management concepts (project phases) 21-May-16Axel Vanhooren 37
  38. 38. Iterations in the Waterfall 21-May-16Axel Vanhooren 38 Analysis Design Intra-task Inter-phase Iteration Improvement Iteration Release 1 Release 2 Analysis Design Building TestingImplementation Evaluation Inter-task / Intra-phase Analysis Information Gathering (actually ‘set of activities’) Correcting a work Gradual work improvement Improving the product
  39. 39. Practical: How to perform a backward jump? A possible way of how to… by example During the design phase an issue is raised. The design is stuck. Some additional analysis is required. 1. The area surrounding the issue can temporarily be frozen. 2. Verification whether the scope needs to be changed. 3. The analyst will analyse the situation concerning the issue and resolve it. The impact of the change is determined. Some verifications are done to ensure coherence. 4. The corrections are reviewed and re-validated (if there was a validation previously). 5. Verification whether plans and estimations remain valid. 6. The design can continue.  The project remained in “Design Phase”.  People have decided to redo some previous activities.  The rework was limited and focused. Usually, the validation goes fast. Only the changed part is reviewed. Much of the subject is known and the rest of the analysis was already validated. Meanwhile the project team members continued working on another part of the project.  Project Manager may not like it if estimations and plans have to be reviewed or, worse, if (s)he has to renegotiate time and resources. But that’s the job. 21-May-16Axel Vanhooren 39
  40. 40. Various Techniques or Concepts Applicable in Waterfall Projects  Sub-projects  Phases  Work Packages  Iterations  Phased delivery  Phase overlaps  Proof of Concept  Prototyping  Scenarios  Simulations  Role Playing  Mock-up screens  Parallel run  V-model, W-model  Re-use  Objective change  Scope change  Rolling waves  Planning Reviews  Releases  Phased delivery  … 21-May-16Axel Vanhooren 40
  41. 41. Major PM sins “causing the waterfall to fail”  Product is more important than project  SIN #1: success is on delivery: time, on budget, on scope  ignoring product value  Benefits come from the product, not from the project  Project creates the product, but the product creates value  Lifetime project < lifetime of product  Project execution is the reality, planning is only an approximation  SIN #2: Team must work harder to respect planning and to meet deadline  Planning serves to facilitate the project execution  Project execution does not serve to prove the planning is right  If diff between plan and project execution is small, adapt the project execution. If the diff is big, adapt the plan.  Review and adapt the plans regularly  SIN #3: thinking the plan is stable, doesn’t need changes during the project execution and the project execution will be as planned.  Early in project, little is known. Only after the design the product is known and the plans become more reliable.  Estimations are approximations  Software development projects are dependent of many uncertainties and variable aspects !! You can’t control them, you can guide them. Need for planning reviews 21-May-16Axel Vanhooren 41
  42. 42. Suggested (General) Waterfall 21-May-16Axel Vanhooren 42 Diagnosis Preliminary Analysis High level Design / Architecture Analysis Design Building Testing Integration Deployment & Release 2X Normal flow Return to previous type of activity ‘Template concept’ that can be combined with releases, sub-projects, special development tracks, work packages, prototyping, PoC, …
  43. 43. Suggested (General) Waterfall 21-May-16Axel Vanhooren 43 Software Development Perspective • (1) Preliminary analysis is required to conceive the HL Design / Architecture • (2)Testing from day 1, in parallel. • (3) Every phase is in relation to testing. (4) Nothing is released without testing • (5) Continuous integration during building activities (integration // with building) • (6) Execute the process 2x. The 2nd time is important to remove teeth problems. The cycle is/can be/should be repeated several times for ‘continuous improvement’ / the deployment of different releases. Diagnosis Preliminary Analysis High level Design / Architecture Analysis Design Building Testing Integration Deployment & Release 2X (2) (1) (3)(6) (3)(3) (3) (3) (4) (5)
  44. 44. Suggested (General) Waterfall 21-May-16Axel Vanhooren 44 Project Management Perspective • HL Design / Architecture allows to do estimates and to organise the project, define and establish impact and collaboration, define and plan resource needs, communication, … • Estimates become more reliable once the solution is designed. • Plans & estimates should be reviewed min. 2 or 3 times, possibly more. Certainly 1 time after the design phase. Diagnosis Preliminary Analysis High level Design / Architecture Analysis Design Building Testing Integration Deployment & Release Project organisation & Planning Reliable estimates
  45. 45. Approaches and Perspectives  Goal oriented  Holistic  Systemic  Value based  Plan based  Priority based  Risk based  Vision based  Top-down approach  Multi-stakeholders environments  Multi-user  Cross functional approach  From inner core to peripheral areas  From foundation to top-layer  Follow-the-flow approach  Iterative  Incremental  Composite approaches  Multi-disciplinary collaboration  Structural integration  Product/Structural decomposition  Purpose oriented  Architecture centric  Process centric  Ontology based  Networked/ Interconnected systems  Component based systems  Agent based systems  Delocalised  Information sharing  Building Blocks, Re-Use  Continuous Improvement  Short & long term  … 21-May-16Axel Vanhooren 45 Allows many types of approaches & perspectives
  46. 46. 21-May-16Axel Vanhooren 46 VARIOUS ASPECTS Presents some common topics often discussed and evaluated.
  47. 47. Goal Oriented  A specific goal, an objective, a desired outcome drives waterfall projects  The project goal is (among others) derived from business objectives.  The product is the mean to reach or to contribute to the (business) goal  Building the product is the path to this goal  High level Design / Architecture  early in project  The goal is clearly established from the onset providing focus and direction  It defines the path towards the goal and guides the whole project from the beginning.  The goal is or should be (relatively) stable. No moving target 21-May-16Axel Vanhooren 47
  48. 48. High Visibility  Overall vision of product  High level Design / Architecture  early in project – target is known  “Simple” understandable global process / plan  General process is more or less known  The HL Design of the solution allows making a global plan covering the whole project (not confusing with neither a plan cut in stone nor with a detailed plan).  Maintaining visibility  Limited and controlled change  This visibility allows  Organising the product in systems, sub-systems, components, … product flexibility  Structuring, planning and organising the project  Organising the development in teams, parallel development, …  Project Management: Estimating the project, timely foreseeing resources  Effectively progressing towards the goal  Better understanding and study of the product, increased focus, better communication, better collaboration, ...  Avoidance of redundancy, (by allowing re-use), … 21-May-16Axel Vanhooren 48
  49. 49. Project Size - Scalability  Small  Apply the waterfall in a ‘lighter’ process and with lighter project management  Depends of how critical the project is and the required degree of control  Medium  Most methodologies support directly medium-size projects  Large  The larger the project, the more complex the initiative is  Upfront High-Level Design & Architecture allows scalability  Structure the initiative in programme, projects, sub-projects, releases, phases & work packages  Waterfall projects use more written communication. Written communication suits better the needs of larger and more complex projects than face-to-face (voice). 21-May-16Axel Vanhooren 49
  50. 50. Customer Involvement  The project is heavily dependent of the customer  Transfer of knowledge and information  (Business) Objectives, Complaints/Operational issues, Decision making, Feedback  Collaboration  Involve the customer throughout the project  Prototyping, proof of concept, mock-up screens, …  Showing the product in development (ex. demo’s) even if not finished is possible  The UAT is not a “product presentation phase” 21-May-16Axel Vanhooren 50
  51. 51. Change  Positive Side  Correction of wrong idea, concept, requirements, …  Improvement  Added (Business) Value  Insight evolves; new ideas may come up; ideas evolve, mature  Increased non-functional characteristics, decreased risks, …  May provide the feeling of progress  Negative Side  Cost  Rework – Additional delay – Impatience - Increased pressure for projects  Re-estimation, re-negotiation, re-planning  Synchronisation among projects/sub-projects  Risks for introducing new problems, impact elsewhere, coherence  Degradation of the system  Risk of confusion  Adaptation, fear, pressure, increased stress & conflicts  May give feeling of wasted time (in the past)  Frustration (people need some stability)  Deployment, Release Management, Version Management, Training 21-May-16Axel Vanhooren 51 Not all changes are good Acceptance of a change is a matter of benefits vs downsides Particularly when many changes occur partly simultaneously, partly at different dates
  52. 52. Change – Types of Changes  Unavoidable Change  Change that really couldn’t be foreseen  Limited analysis and insufficient reflection generate changes which can too easily be labelled as ‘unavoidable’. They aren’t.  Can be triggered externally  Avoidable Change  Changes due to ignoring the cause, an approximate understanding of the problem, a lack of insight or effort, bad choices, bad decisions, not knowing what to want, ignoring the real need, wrong or missing requirements, … Common causes: bad diagnosis, not enough ‘analysis’  Limit as much as possible the occurrence of such changes. 21-May-16Axel Vanhooren 52
  53. 53. Change – Handling Changes  Changes are Allowed  Changes Should Be Limited (as much as possible/reasonably)  Positive / Negative aspects of change  Grouping of changes  Process  Evaluate every change  Criteria: Value (benefit), Importance, Criticality, Urgency  Impact on product, environment, business  impact on time, budget, scope, project plan, project resources and project capability  Yes/No/Later/Partial/Taking future change already into account in present design  Plan change  Introduce through Change Control  Avoiding customer changes many times  Negotiating scope, budget and time  Review of plan  avoid project to fail  Dealing with pressure on project team  Be practical, not overly bureaucratic 21-May-16Axel Vanhooren 53
  54. 54. Flexibility  Process Flexibility  The waterfall concept provides a basic engineering concept  It is not a good idea to ignore engineering best practices  The softw. dev. process can be organised in many ways  Formalism doesn’t mean ‘no flexibility’ – formalised processes can be adapted  Process flexibility doesn’t guarantee / increase the product flexibility  Freedom can be foreseen even in a formalised environment  Product Flexibility  Obtained through the architecture  using (well-defined) building blocks  by the quality of the interfaces  Requires a picture of the global product architecture as early as possible in the process  Without formalism (structure, rules, norms, standards, agreements, …) no or lesser flexibility 21-May-16Axel Vanhooren 54
  55. 55. Continuous Improvement 21-May-16Axel Vanhooren 55 Analysis Design Building TestingImplementation Evaluation Product Improvement Process Improvement In methodologies, implemented as “Learned Lessons”
  56. 56. Quality  The project process focusses a lot on Analysis. This means on learning, understanding the business, learning the situation and the context.  The process supports learning, then thinking, then solving, then building and finally testing.  The project process isn’t satisfied with “what the customer wants or asks” which is very unreliable. The whole organisation, the enterprise, the context and already implemented information systems and IT infrastructure imposes its own requirements.  Verification throughout the project  The process is simple and manageable and thus controllable.  The process is known by anybody and artefacts are known.  The artefacts and the various techniques offer a solid base for communication and verification. (Written communication is preferred over face-to-face)  Verification is done through many activities and techniques. 21-May-16Axel Vanhooren 56
  57. 57. Speed of Development  Focussed to the Goal  Project objectives and global design early in process  Driven by “doing the right things right from the first time”  Avoiding wasting time coding wrong solutions  Reduction (not eliminating) of avoidable change, thus rework, thus delay and cost  Strive to obtain clear, mature and commonly agreed idea of the product before starting the coding  Detecting the major functional flaws early in the process before building  Effectiveness and Efficiency  Provides a manageable process and allows a good organisation  Since (to some extend) the product is known early, work can more easily be broken down, resources hired on time and assigned to tasks  HL design allows a good project organisation  Requires thinking and producing artefacts. This may give the idea of no progress. The progress is mental and becomes concrete once coding started.  Good support of re-use  Avoiding redundancy  Speeding up future development 21-May-16Axel Vanhooren 57
  58. 58. Delivery Frequency – Big Bang?  Deliver all at once at the end of the project  Phased delivery  The solution is delivered in useable parts  Releases  Regularly a new release is delivered.  Releases are different versions of the same.  Releases can be developed in parallel reducing the time between two deliveries.  Often, when creating a new system, the first release is lengthy. The subsequent releases are improvements and expansions. They can often be delivered more quickly.  Per sub-project  Delivery can be organised per sub-project (or per group of sub-projects) This depends of their content of these sub-projects. 21-May-16Axel Vanhooren 58
  59. 59. Limited Risks Risk is inherent to projects Waterfall projects reduce risks.  It is a ‘simple’ process (as simple as it can be; depending on how the PM structures and organises the project).  It’s a controllable environment / process  It supports risk management  Many aspects like upfront thinking, a lot of verification, prototyping, proof-of- concept, .. reduce risks  Possibility to deal with the most risky tasks/components first  The main risks are often related to  new technologies  assumptions about the product  dependencies from external factors like stakeholders 21-May-16Axel Vanhooren 59
  60. 60. Cost Efficiency It is effective, efficient, manageable and guidable:  Goal oriented  Doing the right things right from the first time (as much as possible)  Know what and why to do things  Avoiding avoidable changes & rework,  Finding errors early  Testing during the whole project  Structured and understandable approach allows efficient organisation  Objectives, early HL-design and process allows guidance improving the effectiveness 21-May-16Axel Vanhooren 60
  61. 61. Communication  Balance between written communication and verbal communication  Face-to-face  Fast, Interactive, Connection  Volatile, people need to be present, can be distorted, repeated vocal messages can be repeated differently  Problem if persons are not present or large groups  Suitable for small teams, interviews, workshops…  Written communication  Non-volatile, re-usable, distributable, shareable, …  Can be retrieved at any time (for re-reading, reworking, …)  Suitable for large groups and larger projects  Traceable  Slower, most people don’t like to write or read 21-May-16Axel Vanhooren 61
  62. 62. Documentation (1)  The waterfall itself doesn’t specify whether a lot of documentation has to be created.  Don’t limit or skip documentation because  not liking to document  because nobody reads it anyway.  Do not document because it has to be like that, because it is an obligation  acting purposeless  useless  Badly written documentation won’t be read  Don’t overly document  Don’t document too little  Don’t underestimate the need for documentation.  Writing forces to think! 21-May-16Axel Vanhooren 62
  63. 63. Documentation (2)  Documentation is an activity to help others doing their job  Make documentation practical : structure, style, readable  Document purposefully - only useful stuff  Consider future needs (testing, installation, end-users, maintenance, administration, disaster recovery, future software projects,…)  Don’t document obvious stuff  Analysis and design as documentation  Analysis and design artefacts, and possibly some others as well, are often considered as ‘documentation’. The point is not to get these documents written. The essence is about the thinking that is being done when making them. These artefacts are the result of this thinking. They support thinking, maturation of ideas, communication, collaboration & form the basis for future work in the project. They are also used for future projects adapting the system. Other documents and plans: record only what is necessary. Waterfall can be lightweight 21-May-16Axel Vanhooren 63
  64. 64. Innovation -[ This is a personal opinion ]-  Pressure, fear, forced obeisance, interdictions and imposed limitations, uncertainty, confusion, hopelessness, and other negative emotions kill or hinder creativity.  Focus, organisation, rules, structure don’t kill creativity and innovation.  Space, time, freedom, peace of mind are four essential ingredients for creativity.  Within an organised environment specific spaces, time and freedom to foster creativity can be created.  A mechanism is required to capture ideas, evaluate and to offer support to the good ones.  Methodologies based on the waterfall concept allow to handle pressure. And, if time and budget allows it, it is possible to expand the process with creativity moments and spaces. (Workshops, brainstorming, prototyping, …) 21-May-16Axel Vanhooren 64
  65. 65. 21-May-16Axel Vanhooren 65 Critics & Myths Many myths do exist and critics are uttered often based on assumptions, bad usage or a lack of insight. Answers are provided.
  66. 66. Myths and Critics - Iterations  Each phase must be completed and perfected before the next phase begins  Yes and no. It is the intention to the right things right before going starting the next set of activities. If, for example, during programming phase it appears that the design isn’t completely right, then the design must be corrected.  In waterfall, the project can’t return to an earlier phase.  The intention is to execute it sequentially way but Dr. Winston W. Royce’s model shows iterations and many other methodologies show/allow/foresee jumps to earlier ‘phases’  Cfr. slide “Usage of Methodologies”  Most often, the project don’t need to return to an earlier phase. Activities of different types can be executed during any phase.  From a methodological perspective, there can be iterations in the waterfall methodologies. It is not because water flows only downwards, that the methodological ‘waterfall’ can’t iterate.  If something goes wrong in an early step, the implementation phase will be a disaster. (Death March)  Waterfall methodologies seek to avoid mistakes and to detect mistakes as early as possible. If a mistake is detected, it is wise to correct it as soon as possible. It is more likely that mistakes detected later in the project are only minor mistakes. Many things can be done to prevent implementation disasters. 21-May-16Axel Vanhooren 66
  67. 67. Myths and Critics - Iterations  The whole project has to be moved to an earlier phase  It is not because an issue needs some more analysis or that a design aspect has to be reviewed that the whole project has “to be moved to an earlier phase”.  If the situation requires such a drastic move, it may be worthwhile to consider to hire a professional staff (or PM) or to reconsider the feasibility of the project.  All the work has to be thrown away and the phase (or even the project) must be restarted from scratch.  All the work to be thrown away? The whole project restarted from scratch? … no comment. 21-May-16Axel Vanhooren 67
  68. 68. Myths and Critics - Bureaucracy  “The Waterfall” is bureaucratic  Being “bureaucratic” is vague, subjective and imprecise term. It’s an emotional critic. Does it mean “administrative work” or “unnecessary administrative work”?  People may experience something as bureaucratic if they have to do paperwork that serves someone else. They are not always aware of how it serves.  Lesser documents can be produced to make a process lighter, evolve faster, deliver more and earlier. But this is pretty short-sighted if this documentation is needed for end-user training, disaster recovery, debugging, refactoring, future adaptation of the software system, replacement of team members, ensure coherence among systems or among processes, …  Much more system documentation is required than people often assume, particularly if not aware of someone else’s job or if considering only a successful delivery and not a successful product lifetime.  Methodologies don’t impose unnecessary work. People do !  Larger and more complex the project in larger environments require more paperwork is to manage the product and the project.  The waterfall, as a concept (or process lifecycle), can’t be bureaucratic, neither are methodologies. Waterfall projects shouldn’t be bureaucratic.  Understand what paperwork is required and useful and which isn’t. Leave out the unnecessary. 21-May-16Axel Vanhooren 68
  69. 69. Myths and Critics - Change  No change allowed or change is difficult  Regardless of the philosophy, approach or methodology, software development is a slow process and require some stability. This is the reason why changes should be selected and accepted wisely. By emphasis good analysis, the waterfall process allows to avoid a lot of changes caused by a lack of information and insight, caused by bad decisions or by ideas still maturing.  Resistance to change. Fear of change  People do have the tendency to fear change and thus to resist it. This has nothing to do with the waterfall. People may have some more the tendency to want to preserve their work from changes. They may not like having to change the analysis, the design the source code, the plans (even more) because everything has been thought through and it was a hell of a job. This is maybe a belief, an expectation, a norm and/or habit to be created.  Adjusting the scope of a project can kill the project  The scope delineates what the development will implement and what it won’t. A scope change means automatically a verification of time and budget and a review of plans. It is logic that if more work has to be done, it will require more time and budget. This is a project management/people issue, not a waterfall issue. 21-May-16Axel Vanhooren 69
  70. 70. Myths and Critics - Testing  Testing happens only at the end  The Test Phase is confused with testing activities. There is a Test phase at the end, but testing happens during the whole project. All the checks, verifications and validation analysts and stakeholders do are a form of testing. Testing analysis or design is done differently than the testing during the test phase. Different test methods are prototyping, proof of concept, simulation, scenarios, role playing, mock-up screens, reviews, requirements validation, …  The concept shows that the product has to be tested when it is built and before it is deployed in production environment. This doesn’t mean that these are the only tests.  Risk of Late Discovery of Mistakes Due to Late Testing  The learning and thinking work done before building and the checks and tests of analysis and design are done early to discover mistakes. The global visibility prevents also some mistakes and increases the chance to take everything into account and to maintain consistency.  Test Phase is Reduced  It happens that people underestimate the testing effort. Independently of the methodology, software systems require a lot of effort to be tested. And the more changes occurred, the more tests are needed.  Causes are: impatience, trust, short on time and budget.  This decision is not a methodological issue, but a people issue. 21-May-16Axel Vanhooren 70
  71. 71. Myths and Critics - Requirements  Need of big upfront requirements  There are upfront requirements. A genuine effort has to be done to get the all requirements. However, it doesn’t mean 100% of the requirements must be elicited, (you can’t know when you have 100% of the requirements) neither does it mean they may not be changed anymore. (validation ≠ freezing)  Need of big upfront design  There is an upfront design, called architecture. It allows to organise the project and to conduct and execute it more efficiently. An upfront design doesn’t mean that it can’t be changed if required.  Requirements need to be very well understood  Whatever the approach, requirements need always to be well understood.  Not suitable when requirements are at moderate to high risk of changing  The better the analysis has been done, the lesser the requirements will change. It’s quicker for someone to change his mind than for developers to develop a software application. Regardless of the approach, some stability is required. 21-May-16Axel Vanhooren 71
  72. 72. Myths and Critics – Project Size & Process  Doesn’t work for small projects  The waterfall concept contains basic steps which are suitable for any development project. It is up to the project manager and team to decide how ‘lightweight’ they make the methodology. Ex. The “implementation phase” of a very small project can be as simple as copying a file on a hard disk.  Doesn’t work for large projects  The waterfall concept allows the elaboration of methodologies useable for larger projects. The architecture/HL design allows to divide the project into sub-projects or other sub-units. This makes waterfall projects very scalable.  Heavy process  The waterfall concept depicts in fact a natural process. People decide how ‘heavy’ they make the process. The larger the project and/or the more control is required, the heavier the process will be. 21-May-16Axel Vanhooren 72
  73. 73. Myths and Critics - Delivery  Lengthy delay before delivery  Software development is a slow, complex and tedious work and many software systems are large. This is the nature of the job. The business community may underestimate this.  Doing a genuine effort to do the right things right in an organised and guided way is the fastest way to delivery.  There are ways to work faster, and thus to deliver faster, but this is not by increasing pressure, doing overtime or taking short-cuts on planning, documentation, …  Big Bang Delivery  More frequent delivery can be obtained by delivering in phases, in parts and in releases. This can be done by working with sub-projects and iterations. 21-May-16Axel Vanhooren 73
  74. 74. Myths About the Waterfall  Little customer involvement  This is not a methodological issue.  Customer discovers the product only at the end of the project  It is indeed a bad practice to show the product for the first time to the client during the testing (UAT). The waterfall is not the cause of this. The client can get prototypes, mock-up screens, scenarios, models and demo’s from software still in development. It is the team’s responsibility to organise this. There are plenty of methods and occasions to show the client what the product will look like or looks like. Nothing in the waterfall forbids this.  Since the client can get a working model of the software application only at the end of the project, he can’t inform the developers if what has been designed is what he asked for.  Reducing the journey of software systems development, and particularly when it concerns large and complex systems, to a client-developer story is a dangerous simplification. The client may verify if what is being developed is what he asked for. But how sure are we that what asked will really solve the problem? This concerns different disciplines other than business knowledge. It is really not a good idea to rely on this. Furthermore, there are plenty of other specialists involved like architects and analysts.  It is up to the team to do a good job and to put in place verification mechanisms detecting mistakes during the whole project. 21-May-16Axel Vanhooren 74
  75. 75. 21-May-16Axel Vanhooren 75 Brief Evaluation
  76. 76. Projects (not) Suiting the Waterfall  Suiting the Waterfall  Systems where information is important  Projects of any size  When guidance towards a relatively stable objective is important  Systems where internal structure and organisation is important  Systems where non-functional requirements are important  Systems with many end-users groups  Systems based on shared and/or reusable components  Projects with engineering-minded professionals  Lesser well-suiting the Waterfall  Systems where layout and features are most important  Projects with many changes, moving target  Projects guided by the preferences of end-users  Projects where a lot of ideas have to be experimented  Projects with creative team considering software development as an art or experimentation 21-May-16Axel Vanhooren 76
  77. 77. Advantages  Goal oriented  Corresponds to natural problem solving process  Suits many Corporate IT projects from small to large  Allows holistic approach  Process is “Easy” and can be structured in many ways  Flexible  Can be lightweight, as well as heavy  Cost efficiency: do the right things right from the first time – limiting rework, time, cost  Focus on long term, lifespan of systems  Organisable, manageable and controllable  Errors are prevented or detected early  Client involvement during the whole process is possible  Motivated, responsible, professional team members  Suitable with many stakeholders, different groups and/or at different locations  And many many many others 21-May-16Axel Vanhooren 77
  78. 78. Disadvantages The following points are serious risks.  Developers work more individually, instead of as team. The analysis and design are “on paper”, this knowledge can be shared. But then work is divided. Each developer is assigned to do a work. Each developer knows the piece of the software system he works on. This knowledge is rarely shared among the developers. Each developer knows his source code. The code tends to be more personal. (is not unsolvable)  Lesser creativity and social interaction in the developers job. The phases in waterfall-based methodologies facilitates job specialisation. Creativity and interaction with the client is moved away from the developers. Their job become lesser creative and with reduced social interactions. This interaction is also more limited to technical issues. An important share of input for their job is (partly) happens through documents (bureaucracy?).  The process may support and “guide” the team a little bit too much. The team may too much trust “the methodology” and rely on it. The team may come into a kind of automatic mode of operations in which they just follow a process. Instead, during the whole course of the project, they should understand and be aware of where they are in the project and what is the true situation of the project. They should continuously reflect about “what should we do and what not and why (not)” and then taking decisions accordingly. An overly formalised methodology may prevent this awareness and learning. This leads to many issues like sticking to the plan made early in the project, producing too much documents, spending more attention to deadlines rather than to the product or need, … and certainly failing to diagnose project issues and seeing the first signs of a coming disaster. Constant vigilance is required. 21-May-16Axel Vanhooren 78
  79. 79. The Waterfall  Methodological Concept which  corresponds with Problem Solving Process  corresponds with Development Lifecycle  supports engineering disciplines: IS Engineering, Software Engineering, …  Suits many types and sizes of projects  Concept is used for decades  Robust and proven 21-May-16Axel Vanhooren 79
  80. 80. 21-May-16Axel Vanhooren 80 GOOD LUCK AXEL VANHOOREN Freelance Consultant Business Informatics & IS Methodologies Linkedin: be.linkedin.com/in/axelvanhooren/en Other publications: www.taurus-ee.com/Publications

×