Process Vision
Black Book description:
   “… a discipline that governs the identification, control,
    status accounting, and auditing of a given entity, such as a
    software program or system, and the components that
    makeup that entity.” (Berlack, H., “Software Configuration
    Management”, ©1992)
The main goal of SCM is to identify, control, maintain,
 and verify the versions of software configuration items.
                         Configuration      Configuration
                         Management            Audits


        Configuration      Configuration      Configuration
        Identification       Control           Accounting
Policy and procedures the Company is implementing
 are taken from Industry “Best Practices” along with
 recommendations
   Auditing
   Branching Strategy
   Rebasing Projects
   Build process for traceability, completeness,
    repeatability, reliability
   Environment
     Build machine
 Any time we make a change to production, there is the risk that something will go
   wrong. Risk is a fact. The only way to eliminate risk is to never do anything. (But of
   course, that is not an option.) So we are left with actions we can take to mitigate
   those risks.
 CM helps us to avoid many failures, and to recover quickly from those few that we
   can't avoid.
6 Questions
 Both auditing and traceability are mechanisms that are designed to answer the six
   questions: who, what, when, where, why, and how. For example:
      Who made the change? Who authorized it? Who knew about it?
      What exactly was changed? And what was not changed?
      When was the change made (especially relative to other activities)?
      Where was the change made (e.g. what platform, repository, location)?
      Why was the change made? What triggered the change or motivated the person?
      How was the change made? What precisely was done and not done? How was
       information about the change captured and communicated?
 Who?
   The "who" of change primarily revolves around the availability of
    pertinent information. Were the "right" people involved?
   Did the person who made the change have the information that
    was necessary to make an informed choice about doing it?
   The same question can be asked about the person who approved
    the change.

 What?
   The "what" of change points us to issues of completeness.
   Were changes made to all of the things that should have been
    changed?
   What things were missed, and why were they missed?
 When?
   The "when" of change deals with synchronization.
   Were there activities that should have been completed before the
    change was made?
   Were there activities that should not have been done before the
    change?
   Did this change affect other changes?

 Where?
   The "where" of a change leads us to distribution issues.
   Was the change made in all of the places that it should have been
    made?
   Was it distributed to all of the people who should have received
    it?
   And was it timely installed in each place?
 Why?
   What are the reasons behind the change, and what might have been the
    argument against making it?
   Were all of the pertinent facts considered?
   Did the right questions get asked?
   Were the risks understood by those who decided to make the change?

 How?
   Was the failure simply a human one?
   Did the person skip steps, do things incorrectly, or cut corners?
   Did the people have the requisite skills and training to be able to do the
    right things?
   Do they understand why the change procedures exist and the value of
    following each step?
   Do they have the tools to help them to be consistent and efficient as they
    do their parts?
 Utilizing ’s recommended branching
   “In a “Project Based Branching” model, the content (features, enhancements, and defects) tends to be more
   dynamic in nature with constant additions and removal of projects (comprising of features, enhancements, and
   defects) until the last minute. Furthermore, the development package in increments can be released into a
   production environment. Usually, this type of development does not release the entire version after the initial
   rollout, and tends to have more frequent updates to production as frequently as every week.”

 Projects at the Company mandate Parallel Development
   Development which diverges from a common starting point
   Multiple concurrent “current configurations”
   Also implies the possible need to converge again
   WHY?
   Post-release maintenance
   Different product variants from same code base

 There is no checking Development code into the main branch!
 the Company has the policy of branching as needed.
   When a Project does not contain the variant needed to make changes to, a request is to be submitted to get the variant created.
 Source Code starts with a common Line of
  Development
   With parallel development (projects), branches
    are created from that common line
   Keeping branches in sync with each other with
    fixes from the common line to the branch is
    considered a rebase.

  Taking from the base and bringing to the branch
 The build process is more than just a compilation of the code that is
  done on a developer's workstation
 A build process for an application would build the installer,
  documentation, licenses etc. for the entire product.
Update the build numbers in code and
                                       Build the documentation.
   documentation.
                                       Generate release notes
Get the latest sources from source
   control.                            Build the installers.
Tag the code in source control.        Copy the artifacts to a folder for archiving
                                          or to the machine that is used for
Build the code.
                                          archiving the builds. Make sure that
Run automated unit tests.                 the folder are named appropriately.
Run automated functional tests.        Deploy the product to a test server.
Run automated regression tests         Send out build status email.
                                       Clean up and temp files and folders.
 Traceability and Completeness
    knowing throughout the complete software development lifecycle why you are doing
     what you are doing and that it contains all of what you intended.
    A build process can help with traceability by automatically capturing and reporting
     on the changes (new features, defects and so on) that have gone into the build. This
     information is critical for the builds consumers, for example the System Test needs
     to know which of the defects have been included in the build.
 Repeatability and Reliability
    being able to do the same thing over and over again and it being correct each time.
    A build process should snapshot everything at the moment it is created, including
      source file version, compiler settings and the operating system environment itself.
      This information is critical for being able to reproduce an environment for fixing
      defects after a product has been released.
 Agility and Speed
    having a build process in which changes can be integrated quickly or as and when
      needed and that completes in as short a time as possible.
The build machine is a dedicated physical or virtual
 machine whose sole purpose is to build your product.
It should not be used for development or QA activities.
Management of the SCM process
Based on the context, constraints, and guidelines a sound SCM plan (SCMP) should be created.


Software Configuration Identification (CI)
     Typically a CI is a collection of objects related to the specific functionality of a larger system. Examples
        of these objects may be requirements, code, documentation, models and other files.



Software Configuration Control
     This phase concerns the management of changes to configuration items during the software product
        life cycle. The process starts when new changes are identified with a change request (Defect or
        enhancement). During the whole software life cycle, change requests can be initiated by users and
        developers of the software. A standard process for handling change requests should be created. This
        document should describe the formal procedures for handing in and recording change requests,
        evaluate costs and impact, and accept, modify or reject the request. When the change request is
        approved by the Software Configuration Control Board, an implementation plan is created. This plan
        describes the procedures how the change requests are converted to a tangible solution.
Management of the SCM process (Con’t)
Software Configuration Status Accounting
     To control the configuration items, it is essential to record and report about the current state of affairs. An automated
      system will be used to capture and report all relevant information during the life cycle of a software configuration item.

     The collected information will be used as input for the development team, the maintenance team, project management,
      and quality assurance activities. Reports can be periodic or ad-hoc. This feedback can be used to improve the system by
      directed recommendations.

Software Configuration Auditing & Validation
     Software is subject to many regulations, guidelines, constraints, plans, and procedures. On a regular base this
      compliance should be checked by a software audit. This activity assures an independent evaluation of the conformance of
      the configuration item under study. Audits are conducts following standard (detailed) process descriptions.

     There are three fields of interest for audits: functional configuration, physical configuration, and in-process audits of the
      software baseline. The functional configuration audit concerns the question if the item is similar to the prevailing
      specifications. The physical audit considers if the technical documentation is consistent with the built software. The last
      form of auditing, in-process of baseline, investigates the status of several elements of the configuration. A purpose of this
      last audit is to check if the current baseline meets the specifications.
Management of the SCM process (Con’t)
Software Release Management & Delivery
   This last phase of the SCM process is dedicated to the release of a software version. The correct
     baseline, composed of baseline versions of configuration items, is compiled to an executable program.
     This executable can be delivered to any recipient, for purposes of testing or distribution to customer.
     Specific build instructions are needed to guarantee that the build steps are taken and that they are
     performed in the right sequence. Sometimes different versions of the same product should be built (for
     platform, customer, functionality, etc.). The area of software engineering concerned with this process is
     build management.

   When an executable program is created, the program should be delivered. This can occur to external or
     internal customers. Internal customers are just interested in the file and technical documentation. For
     external distribution some additional steps should be taken, such as inserting a manual, release notes,
     and put it in a box. The topic on release management elaborates on this activity in more detail.
Engineering team should be using only those tools that
 have been approved by management.
   This means that everything you are using for your
    development environment MUST have been approved by
    Software Engineering Manager
     Java
     Eclipse
     Derby
     Apache
   If you find a tool you wish to try, send out an e-mail to see
    what feedback is received or go through management.

General SCM

  • 1.
  • 2.
    Black Book description:  “… a discipline that governs the identification, control, status accounting, and auditing of a given entity, such as a software program or system, and the components that makeup that entity.” (Berlack, H., “Software Configuration Management”, ©1992) The main goal of SCM is to identify, control, maintain, and verify the versions of software configuration items. Configuration Configuration Management Audits Configuration Configuration Configuration Identification Control Accounting
  • 3.
    Policy and proceduresthe Company is implementing are taken from Industry “Best Practices” along with recommendations  Auditing  Branching Strategy  Rebasing Projects  Build process for traceability, completeness, repeatability, reliability  Environment Build machine
  • 4.
     Any timewe make a change to production, there is the risk that something will go wrong. Risk is a fact. The only way to eliminate risk is to never do anything. (But of course, that is not an option.) So we are left with actions we can take to mitigate those risks.  CM helps us to avoid many failures, and to recover quickly from those few that we can't avoid. 6 Questions  Both auditing and traceability are mechanisms that are designed to answer the six questions: who, what, when, where, why, and how. For example:  Who made the change? Who authorized it? Who knew about it?  What exactly was changed? And what was not changed?  When was the change made (especially relative to other activities)?  Where was the change made (e.g. what platform, repository, location)?  Why was the change made? What triggered the change or motivated the person?  How was the change made? What precisely was done and not done? How was information about the change captured and communicated?
  • 5.
     Who?  The "who" of change primarily revolves around the availability of pertinent information. Were the "right" people involved?  Did the person who made the change have the information that was necessary to make an informed choice about doing it?  The same question can be asked about the person who approved the change.  What?  The "what" of change points us to issues of completeness.  Were changes made to all of the things that should have been changed?  What things were missed, and why were they missed?
  • 6.
     When?  The "when" of change deals with synchronization.  Were there activities that should have been completed before the change was made?  Were there activities that should not have been done before the change?  Did this change affect other changes?  Where?  The "where" of a change leads us to distribution issues.  Was the change made in all of the places that it should have been made?  Was it distributed to all of the people who should have received it?  And was it timely installed in each place?
  • 7.
     Why?  What are the reasons behind the change, and what might have been the argument against making it?  Were all of the pertinent facts considered?  Did the right questions get asked?  Were the risks understood by those who decided to make the change?  How?  Was the failure simply a human one?  Did the person skip steps, do things incorrectly, or cut corners?  Did the people have the requisite skills and training to be able to do the right things?  Do they understand why the change procedures exist and the value of following each step?  Do they have the tools to help them to be consistent and efficient as they do their parts?
  • 8.
     Utilizing ’srecommended branching “In a “Project Based Branching” model, the content (features, enhancements, and defects) tends to be more dynamic in nature with constant additions and removal of projects (comprising of features, enhancements, and defects) until the last minute. Furthermore, the development package in increments can be released into a production environment. Usually, this type of development does not release the entire version after the initial rollout, and tends to have more frequent updates to production as frequently as every week.”  Projects at the Company mandate Parallel Development Development which diverges from a common starting point Multiple concurrent “current configurations” Also implies the possible need to converge again WHY? Post-release maintenance Different product variants from same code base  There is no checking Development code into the main branch!  the Company has the policy of branching as needed. When a Project does not contain the variant needed to make changes to, a request is to be submitted to get the variant created.
  • 9.
     Source Codestarts with a common Line of Development  With parallel development (projects), branches are created from that common line  Keeping branches in sync with each other with fixes from the common line to the branch is considered a rebase. Taking from the base and bringing to the branch
  • 10.
     The buildprocess is more than just a compilation of the code that is done on a developer's workstation  A build process for an application would build the installer, documentation, licenses etc. for the entire product. Update the build numbers in code and Build the documentation. documentation. Generate release notes Get the latest sources from source control. Build the installers. Tag the code in source control. Copy the artifacts to a folder for archiving or to the machine that is used for Build the code. archiving the builds. Make sure that Run automated unit tests. the folder are named appropriately. Run automated functional tests. Deploy the product to a test server. Run automated regression tests Send out build status email. Clean up and temp files and folders.
  • 11.
     Traceability andCompleteness  knowing throughout the complete software development lifecycle why you are doing what you are doing and that it contains all of what you intended.  A build process can help with traceability by automatically capturing and reporting on the changes (new features, defects and so on) that have gone into the build. This information is critical for the builds consumers, for example the System Test needs to know which of the defects have been included in the build.  Repeatability and Reliability  being able to do the same thing over and over again and it being correct each time.  A build process should snapshot everything at the moment it is created, including source file version, compiler settings and the operating system environment itself. This information is critical for being able to reproduce an environment for fixing defects after a product has been released.  Agility and Speed  having a build process in which changes can be integrated quickly or as and when needed and that completes in as short a time as possible.
  • 12.
    The build machineis a dedicated physical or virtual machine whose sole purpose is to build your product. It should not be used for development or QA activities.
  • 13.
    Management of theSCM process Based on the context, constraints, and guidelines a sound SCM plan (SCMP) should be created. Software Configuration Identification (CI)  Typically a CI is a collection of objects related to the specific functionality of a larger system. Examples of these objects may be requirements, code, documentation, models and other files. Software Configuration Control  This phase concerns the management of changes to configuration items during the software product life cycle. The process starts when new changes are identified with a change request (Defect or enhancement). During the whole software life cycle, change requests can be initiated by users and developers of the software. A standard process for handling change requests should be created. This document should describe the formal procedures for handing in and recording change requests, evaluate costs and impact, and accept, modify or reject the request. When the change request is approved by the Software Configuration Control Board, an implementation plan is created. This plan describes the procedures how the change requests are converted to a tangible solution.
  • 14.
    Management of theSCM process (Con’t) Software Configuration Status Accounting  To control the configuration items, it is essential to record and report about the current state of affairs. An automated system will be used to capture and report all relevant information during the life cycle of a software configuration item.  The collected information will be used as input for the development team, the maintenance team, project management, and quality assurance activities. Reports can be periodic or ad-hoc. This feedback can be used to improve the system by directed recommendations. Software Configuration Auditing & Validation  Software is subject to many regulations, guidelines, constraints, plans, and procedures. On a regular base this compliance should be checked by a software audit. This activity assures an independent evaluation of the conformance of the configuration item under study. Audits are conducts following standard (detailed) process descriptions.  There are three fields of interest for audits: functional configuration, physical configuration, and in-process audits of the software baseline. The functional configuration audit concerns the question if the item is similar to the prevailing specifications. The physical audit considers if the technical documentation is consistent with the built software. The last form of auditing, in-process of baseline, investigates the status of several elements of the configuration. A purpose of this last audit is to check if the current baseline meets the specifications.
  • 15.
    Management of theSCM process (Con’t) Software Release Management & Delivery  This last phase of the SCM process is dedicated to the release of a software version. The correct baseline, composed of baseline versions of configuration items, is compiled to an executable program. This executable can be delivered to any recipient, for purposes of testing or distribution to customer. Specific build instructions are needed to guarantee that the build steps are taken and that they are performed in the right sequence. Sometimes different versions of the same product should be built (for platform, customer, functionality, etc.). The area of software engineering concerned with this process is build management.  When an executable program is created, the program should be delivered. This can occur to external or internal customers. Internal customers are just interested in the file and technical documentation. For external distribution some additional steps should be taken, such as inserting a manual, release notes, and put it in a box. The topic on release management elaborates on this activity in more detail.
  • 16.
    Engineering team shouldbe using only those tools that have been approved by management.  This means that everything you are using for your development environment MUST have been approved by Software Engineering Manager Java Eclipse Derby Apache  If you find a tool you wish to try, send out an e-mail to see what feedback is received or go through management.

Editor's Notes

  • #4 What holds everything together? It is the processes used in your organization. Processes allow you to align the way you do business. They allow you to address scalability and provide a way to incorporate knowledge of how to do things better. Processes allow you to leverage your resources and to examine business trends.
  • #5 SCM Audit is performed to ensure that the code that has been changed for fixes are actually picked up in the builds. Also – since we have experienced issues with Synergy not always picking up the correct revisions, this helps us make certain that this is minimized.
  • #6 ISO is an "audit standard" and that the CMMI (Capability Maturity Model Integration) is a "process model," so conceptually they're quite a bit different. Think of the CMMI as a large set of related "best practices" gathered from many product engineering and software development organizations. ISO can be very helpful, especially for marketing, HR, and quality, but it is not specifically focused on engineering and project management as is the CMMI. So the depth of CMMI in these areas is far greater. Who And because any change can impact on other activities, a similar question can be posed. Did the people to whom this change was pertinent know about it when they needed that information? What The converse concerns changing more than should have been changed. Was the changer overzealous? Did he or she slip in other changes that were not authorized?
  • #7 When The "when" may also deal with the change relative to the business use of the system that was changed. Was there some business activity that should not have been interrupted to make the change? Or was there a reason to postpone the change until after a special event (like month-end closing)? Learning from "Where?" The "where" of a change leads us to distribution issues. Was the change made in all of the places that it should have been made? (All Variants) Was it distributed to all of the people who should have received it? And was it timely installed in each place? Conversely, were there places where the change should not have been made? Did it go beyond its intended target? How far a field did it go?
  • #8 The "why" of a change can be the most important question. What are the reasons behind the change, and what might have been the argument against making it? Were all of the pertinent facts considered? Did the right questions get asked? Were the risks understood by those who decided to make the change? Learning from "How?" Finally, people make mistakes. Was the failure simply a human one? Did the person skip steps, do things incorrectly, or cut corners? Did the people have the requisite skills and training to be able to do the right things? Do they understand why the change procedures exist and the value of following each step? Do they have the tools to help them to be consistent and efficient as they do their parts? Learning From the 6 Questions If we are not going to use this information to place blame, then what will we use it for? To learn! While we should strive to never make a mistake, we must always view mistakes as an opportunity to improve. At least one of the six questions (likely more than one) will point us to shortcomings in our CM processes. Perhaps some procedures need to be more specific than they are. Maybe someone who should be involved was overlooked. Training may be called for. Perhaps some checks and balanced are needed. The more information we have about the problem at hand, the more we can learn from it, and the better our improved process will be. Which brings us back to the necessity of audit trails and traceability. These are our primary source of this goldmine of information. With little or no information, we are left with people's memories and imaginations to identify the root causes of problems. With hard data, a clear path to an improved process is likely to emerge. So in the end, audit trails and traceability really are risk mitigation mechanisms. Although they don't directly help us to avoid risks, they do position us to learn from each failure, and hopefully avoid repeating it in the future!