2nd International Colloquium
               On

High Maturity Best Practices 2011
             (HMBP 2011)



           Presenting a Paper
                   On



            M∆ - MDelta
         Quantifying Maturity


                 By
       Sushant Mukund Hublikar


      E-mail: shublika@cisco.com
         Ph: +91-9886752443


     Cisco Systems India Pvt. Ltd.,
  Salarpuria Hallmark, Kadubeesanahalli,
     Opposite to New Horizon College,
             Outer Ring Road.
          BANGALORE – 560061
Page |2


                                         Abstract

       The main intention of this paper is to bring into focus, a new model developed to
gauge the maturity levels of the organizations or the independent business units within
organizations and to provide a simple method of evaluating focus areas to achieve higher
maturity levels and sustain at those levels with continuous improvement. In this paper, we
would be mainly talking about MDelta, the model being proposed, its need, applications and
outcome.



                                         Prologue
                             Models, Maturity Levels and M∆

Delta (∆) is a Greek alphabet which signifies finite differences between functions, basically
used in mathematics and computational analysis.

In this paper, we have proposed a new model appropriately called as MDelta (M∆). This
model basically constitutes methods/techniques of evaluating “differences” between the
immediate maturity levels of the CMMI Model.

CMMI best practices are published in documents called models, each of which addresses a
different area of interest. CMMI currently addresses three areas of interest:

   1. Product and service development - CMMI for Development (CMMI-DEV),
   2. Service establishment, management, and delivery - CMMI for Services (CMMI-
      SVC),
   3. Product and service acquisition - CMMI for Acquisition (CMMI-ACQ).

The current release of CMMI is Version 1.3, developed in November 2010.

Using the improvement path known as the staged representation enables you to achieve
“maturity levels”. Maturity levels apply to an organization’s process improvement
achievement across multiple process areas. These levels are a means of improving the
processes corresponding to a given set of process areas (i.e., maturity level). There are 5
maturity levels numbered 1 through 5.

   •   The maturity levels are measured by the achievement of the specific and
       generic goals associated with each predefined set of process areas.

CMMI Levels 4 and 5 are known as the High Maturity Levels.

Achieving maturity level 4 involves implementing all process areas for maturity levels 2, 3,
and 4. Likewise, achieving maturity level 5 involves implementing all process areas for
maturity levels 2, 3, 4, and 5.

   •   M∆ attempts to quantify the maturity and ascertains the progress required
       in the process areas of the lower maturity levels to elevate to the high
       maturity levels.

                                                                               HMBP 2011 | M∆
Page |3


Endeavouring to quantify the maturity indicates the need of the organizations to be
“appraised” to higher maturity levels and maintain a stable process improvement cycle.

Adhering to the focus of this year's Colloquium, "High Maturity: A Practical Approach", this
paper makes a sincere effort to address on “How to Implement” rather than “What Activities
To Implement”.

   •   CMMI for Development (CMMI-DEV) model has been used as the model for reference
       in this paper to demonstrate the usage of M∆.

Changing trends in the market currently are at the highest pace and so are the customer
needs. Therefore, there is a need for organizations to adhere to the customer desires by
bringing out quality products without any slippage in time and at the same time maintaining
standard processes and standards throughout the lifecycle. This can be gauged better with
the help of a quantifiable entity which would recognize the areas needing enhancements.

M∆ attempts to do the same.




                                                                             HMBP 2011 | M∆
Page |4


                                      Let’s Achieve!
                                 Implementation of M∆


Let us consider the process areas of the High Maturity levels and the immediate lower
Maturity level 3:




                                        CAR - Causal Analysis and Resolution
                   Maturity Level 5
                    Optimizing
                                        OPM - Organizational Performance Management
High Maturity
   Levels          Maturity Level 4     OPP - Organizational Process Performance
                   Quantitatively
                     Managed            QPM - Quantitative Project Management

                                        DAR - Decision Analysis and Resolution

                                        IPM - Integrated Project Management

                                        OPD - Organizational Process Definition

                                        OPF - Organizational Process Focus

                                        OT - Organizational Training
Lower Maturity     Maturity Level 3
    Level                               PI - Product Integration
                      Defined
                                        RD - Requirements Development

                                        RSKM - Risk Management

                                        TS - Technical Solution

                                        VAL – Validation

                                        VER – Verification

                           Table 1: Immediate Maturity Levels




                                                                             HMBP 2011 | M∆
Page |5


Maturity Level 3 (Defined) to Maturity Level 4 (Quantitatively Managed):

To achieve High Maturity Level 4, we need to understand the Maturity Level 3 areas where
the organizations/business units require regulation.

Therefore, we are suggesting a simple method where we can divide the nine process areas
of the ML 3 into groups equal to as many process areas of ML 4 i.e. a set of 4 process areas
(Group 1) in ML 3 has been directly mapped to the Organizational Process Performance
(OPP) PA of ML 4 and the remaining 7 (Group 2) have been mapped to the Quantitative
Project Management (QPM) PA of the ML 4 respectively.

This grouping or classification has been done based on the specific goals associated with
each of the PA of ML 3 which closely satisfies the goals of the corresponding PA of ML 4.

To elaborate, let us consider OPP => Organizational Process Performance PA of ML 4.

The purpose of OPP is to

   •   Establish and maintain a quantitative understanding of the performance of
       selected processes in the organization’s set of standard processes in
       support of achieving quality and process performance objectives

   •   To provide process performance data, baselines, and models to
       quantitatively manage the organization’s projects.

Now, scanning the PAs of ML 3 which closely relate to OPP, we have the following:

OPD =>         Organizational Process Definition involves establishing and maintaining a
usable set of organizational process assets, work environment standards, and rules and
guidelines for teams.

OPF    =>     Organizational Process Focus which involves planning, implementing, and
deploying organizational process improvements based on a thorough understanding of
current strengths and weaknesses of the organization’s processes and process assets.

OT     =>     Organizational Training involves developing skills and knowledge of people so
they can perform their roles effectively and efficiently.

Similarly, let us consider QPM => Quantitative Project Management PA of ML 4.

The purpose of QPM is to

   •   Quantitatively manage the project to achieve the project’s established
       quality and process performance objectives.

The PAs of ML 3 which closely relate to QPM are:

DAR =>        Decision Analysis and Resolution involves analyzing possible decisions using a
formal evaluation process that evaluates identified alternatives against established criteria.


                                                                              HMBP 2011 | M∆
Page |6


IPM    =>      Integrated Project Management involves establishing and managing the
project and the involvement of relevant stakeholders according to an integrated and defined
process that is tailored from the organization’s set of standard processes.

PI      =>     Product Integration involves assembling the product from the product
components, ensuring that the product possesses the required functionality and quality
attributes, and delivering the product.

RD    =>      Requirements Development involves analyzing, and establishing customer,
product, and product component requirements.

RSKM =>        Risk Management involves identifying potential problems before they occur so
that risk handling activities can be planned and invoked as needed across the life of the
product or project to mitigate adverse impacts on achieving objectives.

TS     =>    Technical Solution involves selecting, designing, and implementing solutions
to requirements.

VAL     =>     Validation involves demonstrating that a product or product component fulfils
its intended use when placed in its intended environment.

VER    =>     Verification involves ensuring that selected work products meet their specified
requirements.

Referring to the figure below, we have:

M∆     =      ML 4 – ML 3

M∆1    =      f (OPP) - f (OPD + OPF + OT)

M∆2    =      f (QPM) – f (DAR + IPM + PI + RD + RSKM + TS + VAL + VER)
                                          ML 3
M∆     =      M∆1 + M∆2

This would help to quantify or single out the areas which need focus and improvements to
                                           achieve higher maturity.
                    DAR

                    IPM
                                                                                          ML 4
                    OPD

                    OPF

                     OT

                     PI

                    RD

                   RSKM

                     TS                                                              HMBP 2011 | M∆

                    VAL

                    VER                          Figure 1: Mapping PAs of ML 3 to ML 4
Page |7




                                                               OPP

                                                              QPM




Maturity Level 4 (Quantitatively Managed) to Maturity Level 5 (Optimizing):

To achieve High Maturity Level 5, the levels of maturity level 4 need to be “sustained” and
constantly improved upon.

M∆       =     ML 5 – ML 4

M∆       =     f (CAR + OPM) – f (OPP + QPM)

Let us consider CAR => Causal Analysis and Resolution PA of ML 5

The purpose of CAR is to

     •   Identify causes of selected outcomes and take action to improve process
         performance.

     •   Improving quality and productivity by preventing the introduction of defects
         or problems and by identifying and appropriately incorporating the causes
         of superior process performance.

The QPM PA of ML 4 relates closely to the CAR PA of ML 5 due to the following similarities:

CAR mainly deals with the identifying and analyzing causes of selected outcomes and taking
actions to remove causes, prevent the recurrence of defects and problems in the future, and
proactively analyze data to identify potential problems and prevent them from occurring
whereas one of the specific goals of QPM involves performing root cause analysis of selected
issues to address deficiencies in achieving the project’s quality and process performance
objectives.
                                                                             HMBP 2011 | M∆
Page |8



Similarly, the purpose of OPM => Organizational Performance Management PA of ML 5 is

     •   Proactively manage the organization’s performance to meet its business
         objectives.

     •   Iteratively analyzing aggregated project data, identifying gaps in
         performance against the business objectives, and selecting and deploying
         improvements to close the gaps.

Similarly the OPM PA of ML 5 is satisfied by the combined achievement of OPP and QPM PAs
of ML 4.

The OPM PA addresses improved product quality (e.g., functionality, quality attributes),
increased productivity, increased process efficiency and effectiveness, increased consistency
in meeting budget and schedule, greater customer and end-user satisfaction, shorter
development or production time to change functionality, add new features, or adapt to new
technologies, and improved use of resources across the organization while the OPP PAs
satisfy them at the same level by addressing: organizational quantitative quality and
process performance objectives based on business, establishing definitions of the measures
to be used in process performance analyses, establishing process performance baselines and
process performance models along with QPM which involves selecting sub-processes and
attributes critical to understanding performance and that help to achieve the project’s
quality and process performance objectives, selecting measures and analytic techniques to
be used in quantitative management, monitoring the performance of selected sub-processes
using statistical and other quantitative techniques, managing the project using statistical
and other quantitative techniques to determine whether or not the project’s objectives for
quality and process performance are being satisfied.



                                     ML 4                        ML 5

                                     OPP                          CAR

                                     QPM                          OPM



                                   Figure 2: Mapping PAs of ML 4 to ML 5




Therefore,

M∆       =     ML 5 – ML 4

M∆1      =     f (CAR) - f (QPM)

                                                                             HMBP 2011 | M∆
Page |9


M∆2    =      f (OPM) – f (OPP + QPM)

M∆     =      M∆1 + M∆2

High Maturity Process Areas require more of improvements than implementation. The
processes are there and standards are set but the focus is more towards how well the
processes are sustained and how actively are the standards followed to make it a regular
way of doing business.

In the next section, we will look a few case studies for the application and results of M∆.




                                                                               HMBP 2011 | M∆
P a g e | 10


                                        Case Study
                              Applications and Results of M∆


Scenario 1:

A software development organization currently appraised at CMMI Maturity Level 3 is aiming
for a High Maturity appraisal. It has established and implemented all the required processes
and achieved the specific/generic goals of Level 3. However, though the products being
developed by the organization are delivered on time to the market, there is gradual
decrease in the quality of the products and the customers are circumspect about this.
The organization stands at a precarious position where it may be demoted from the Level 3
appraisal during the next audit cycle. The board of directors, the engineering heads and the
senior management set up a meeting to discuss the areas they are falling behind and how
to overcome and improve the processes followed.

Approach:

   •   Identify the Process Areas of Level 3 where the processes are not being followed as
       recommended starting from the gathering the requirements.

         i.    Are the product managers analyzing the customer needs and providing the
               proper requirements? => RD

         ii.   Has the correct solution for the requirement, design and development been
               selected to build the product? => TS

        iii.   Are the required product components (infrastructure/environment) being used
               correctly to build the right product to possess the required functionality and
               quality attributes? => PI

        iv.    Does the final product fulfil its intended use when placed in its intended
               environment? => VAL

        v.     Does the final product meet the specified requirements? => VER

        vi.    Are the correct project management processes implemented for the product
               development followed with the involvement of the relevant stakeholders? =>
               IPM

       vii.    Are the risks encountered during the project handled effectively and on
               priority? => RSKM

       viii.   Are the organizational process assets, work environment standards, rules and
               guidelines employed as per the standards set by the organization? => OPD

        ix.    Has the organization identified the correct processes to be followed based on
               the thorough understanding of its current strengths and weaknesses? => OPF

        x.     Are the resources well-trained enough to perform their roles effectively and
               efficiently? => OT

                                                                                HMBP 2011 | M∆
P a g e | 11


       xi.     Are correct decisions being taken at the right time that has a minimal effect
               on the ability of the project to achieve its objectives? => DAR


Deduce:

  •   According to the scenario, the product is delivered “on time” but the quality is
      decreasing. Considering this situation, we can shortlist RD, TS and PI as the three
      areas which need improvement.

      M∆ = ML 4 – ML 3

      M∆1 = OPP – (OPD + OPF + OT) = No differences.

      M∆2 = QPM – (DAR + IPM + PI + RD + RSKM + TS + VAL + VER) = PI + RD + TS

      M∆ = M∆1 + M∆2 = PI + RD + TS

How To Implement:

  •   Now, analyse the specific goals of each of these PAs:

          i.   RD: Are the product managers analyzing the customer needs and providing
               the proper requirements?

               No    > Analyse the market and find out the trends and the changing
               customer needs
               Yes   > The requirements gathered are correct.

       ii.     TS: Has the correct solution for the requirement, design and development
               been selected to build the product?

               No     > The solution suggested is incorrect because of ambiguous
               requirements.
               Yes    > The solution decided upon is correct.

       iii.    PI: Are the required product components (infrastructure/environment) being
               used correctly to build the right product to possess the required functionality
               and quality attributes?

               No     > The product components are causing a change in the functionality.
               Yes    > The product components used are correct

      Any of these three areas would have failed for the decrease in quality. Reworking on
      them will fetch good results and thereby identifies how to prevent such deviations in
      the future.

Sustenance:

  •   Sustaining at Level 3 and to continually improve the processes to appraise to Level 4,
      the organizations need to perform this activity, frequently, say, once in a quarter, on
      all the projects that are delivered during that time period.
                                                                               HMBP 2011 | M∆
P a g e | 12



       How?

                Perform a M∆ for each project.
                Ensure there are no process areas identified for improvements.
                Ensure the specific process area goals are achieved.

                  Ex: In the above scenario, in the next release of the product, validating the
                  requirements with the customer needs by direct interactions, sharing the
                  solution to verify the requirements and frequent display of the product
                  prototypes will immensely help to get a clear picture on the final solution.
                  Similarly, in the coming release, following the same process leads to the
                  evolution of an organizational standard which every project would follow.

Conclusion:

       Once these areas have been taken care of, the organization remains CMMI Level 3
       appraised but has a good chance to be appraised to Level 4, as all the other process
       areas are taken care of.

       QPM = No delta.

       OPP = No delta


Scenario 2:

       A small-scale organization has done well to reach CMMI Level 3 and is looking for an
       appraisal towards CMMI Level 4. Customers are readily offering projects to the
       organization and the need for more human resources has been answered by an
       urgent recruitment drive. The hasty process has ended up in candidates with false
       experience joining the organization. This has dealt a severe blow to the confidence
       level of the customers with their projects delivered with a huge number of faults and
       the standard processes to be followed have also deviated in order to meet the
       deadlines. The management are in a fix to identify the crux of this issue.


Approach:

   •   Identify the Process Areas of Level 3 where the processes are not being followed as
       recommended starting from the gathering the requirements.

         i.       Are the correct project management processes implemented for the product
                  development followed with the involvement of the relevant stakeholders? =>
                  IPM
        ii.       Are the risks encountered during the project handled effectively and on
                  priority? => RSKM

        iii.      Are the organizational process assets, work environment standards, rules and
                  guidelines employed as per the standards set by the organization? => OPD



                                                                                  HMBP 2011 | M∆
P a g e | 13


       iv.     Has the organization identified the correct processes to be followed based on
               the thorough understanding of its current strengths and weaknesses? => OPF

       v.      Are the resources well-trained enough to perform their roles effectively and
               efficiently? => OT

       vi.     Are correct decisions being taken at the right time that has a minimal effect
               on the ability of the project to achieve its objectives? => DAR


Deduce:

  •   According to the scenario, the processes have been deviated and the resources are
      not “well-trained” to perform their roles efficiently. Considering this situation, we can
      shortlist OPD, OPF, OT, IPM and RSKM as the areas which need improvement.

      M∆ = ML 4 – ML 3

      M∆1 = OPP – (OPD + OPF + OT) = OPD + OPF + OT

      M∆2 = QPM – (DAR + IPM + PI + RD + RSKM + TS + VAL + VER) = IPM + RSKM

      M∆ = M∆1 + M∆2 = OPD + OPF + OT + IPM + RSKM + DAR

How to Implement:

  •   Analysing the specific goals of the PAs

          i.   Are the correct project management processes implemented for the product
               development followed with the involvement of the relevant stakeholders? =>
               IPM

               No     > The processes were not followed as recommended. The last minute
               rush to delivery has heavily impacted the adherence to the processes.
               Yes    > The standard processes have been followed effectively.

       ii.     Are the risks encountered during the project handled effectively and on
               priority? => RSKM

               No      > The correct risks were not identified and overall the project was
               delivered open to risks.
               Yes     > The risks were identified and the appropriate mitigation plan were
               put in place.

       iii.    Are the organizational process assets, work environment standards, rules and
               guidelines employed as per the standards set by the organization? => OPD

               No     > The infrastructure/environment required for the under-trained
               resources and the time (and in turn, money) spent to make them acquaint
               with the project was heavily utilized thus violating the threshold of their
               usage. There were also cases of attrition because of the pressure involved
               thereby the deteriorating the work environment.

                                                                               HMBP 2011 | M∆
P a g e | 14


                Yes    > The assets, standards were efficiently followed and there was
                professionalism maintained at the workplace.

       iv.      Has the organization identified the correct processes to be followed based on
                the thorough understanding of its current strengths and weaknesses? => OPF

                No      > The organization failed in the process of staffing the right candidates
                required for the job. There was no background verification done on the
                candidates and hence had to bear the embarrassment for the poor project
                delivery.
                Yes     > The organization strengthened it stronger areas of operation and
                utilized minimal of its weakest areas.

                Ex: Being a small scale organization, the new candidates faced the shortage
                of owning work place desks for themselves. Therefore, the organization
                decided to depute them at the client’s place directly. This will provide more
                interaction with the client and work getting done faster.

       v.       Are the resources well-trained enough to perform their roles effectively and
                efficiently? => OT

                No      > The recruited candidates were not enough experienced to handle the
                projects as soon as they were on the job. There was a sufficient amount of
                training required thereby using up much important time on the same.
                Yes     > The candidates were well suited for the job and the right resources
                have been employed to handle the projects on the go.

      Again, failure in any of these areas would have led to the deviations in the processes.

Sustenance:

  •   Re-define the organizational processes frequently and communicate the same to
      each and every employee, including new hires. Any new hire would obviously require
      some time to get acclimatized to the work and the work place but ensuring that the
      time is within the consumption of the overall project is a challenge to maintain. This
      can be achieved only by setting up a standard documentation for every new hire and
      revisit them often for changes in line with the organizational standards.
      Also, revisit the organizational processes of the recruiting/staffing unit and make
      certain that the correct practices as followed.


      How?

              Perform a M∆ after each medium-to-large scale recruitment drive.
              Ensure there are no process areas identified for improvements.
              Ensure the specific process area goals are achieved.

                Ex: In the above scenario, in the next event of recruitment, identifying
                candidates who satisfy the requirements completely or who are certified in
                the technology required for the organization would be a good practice rather
                than accepting candidates who just closely match the requirements. Also,
                putting in place an entire new hire training unit department with a detailed

                                                                                 HMBP 2011 | M∆
P a g e | 15


                outlined courses on the technologies needed would help the organization save
                time on training and thereby utilizing the candidates immediately on the job
                and delivering the projects on time.
                       Following the same process during every hiring campaign leads to the
                evolution of an organizational standard which forms the core of organizational
                processes.

Conclusion:

       Once these areas have been taken care of, the organization remains CMMI Level 3
       appraised but has a good chance to be appraised to Level 4, as all the other process
       areas are taken care of.

       QPM = No delta.

       OPP = No delta

Scenario 3:

        An IT bellwether appraised at CMMI Level 4 is pursuing CMMI Level 5 achievement. It
has well-laid organizational standards, efficient project specific processes, best practices in
the fields of Human Resources and technology. Customers are very keen on doing business
with the organization as they are aware of the kind of quality with which the projects are
delivered with and their promises being kept. However, sustaining the improvements within
the processes has been a minor hiccup with the company. The external auditing firm has
listed the areas which need “further” improvement.

Approach:


   •   Identify the Process Areas of Level 4 where the processes require more precision and
       the projects where the standards need to be maintained.

          i.    Are the organization’s set of standard processes in support of achieving
                quality and process performance objectives, and to provide process
                performance data, baselines maintained? => OPP

          ii.   Are the projects managed quantitatively to achieve the project’s established
                quality and process performance objectives? => QPM
Deduce:

   •   According to the scenario, the organization has well-defined standards and has
       created a stronghold for itself in the market with respect to the quality of work
       delivered.

In this case,

       M∆       =      ML 5 – ML 4

       M∆1      =      CAR - QPM

       M∆2      =      OPM – (OPP + QPM)
                                                                               HMBP 2011 | M∆
P a g e | 16


      M∆        =      M∆1 + M∆2 = OPP + QPM

How to Implement:

  •   Analyzing the specific goals of the PAs

      i.        Are the organization’s set of standard processes in support of achieving
                quality and process performance objectives, and to provide process
                performance data, baselines maintained? => OPP

                No      > The products were not delivered within budget and on time, the
                productivity was not improved, the customer ratings did not increase and they
                remained the same.
                Yes     > The products were delivered on time and were of high quality and
                even the customer ratings have shot up. Time-to-market has improved by a
                specified percent.

      ii.       Are the projects managed quantitatively to achieve the project’s established
                quality and process performance objectives? => QPM

                No      > The products performance attributes such as Usability, Reliability,
                and Maintainability were not improved. The cycle time and the percentage of
                rework time did not reduce. Schedule slippage did not improve.
                Yes     > The products function much better now, the mean time between
                failures is at an all-time low, the total lifecycle cost has been reduced by a
                specified percent.

      Stagnation in any of the above areas would have led to the deviations in the
      processes.

Sustenance:

  •   Constantly improving the processes set by the organization and regularly monitoring
      them at equal intervals of a time period assists in recognizing the best practices
      being followed within the business units as well as the organizational level. Drill down
      to the individual teams of the business units and the processes they follow to carry
      out their roles efficiently. Ensure the same is being done at the unit level, being in
      sync with the set standards.

      How?

             Perform a M∆ at the individual team levels.
             Check the process areas which need “further” improvement.
               Ensure the specific process area goals are achieved

                Ex: A team comprising of certified testers who were also genuinely
                experienced in their domain followed the process of improving their cycle time
                by constantly automating the test cases developed for the previous release.
                By doing so, they were able to reduce to time taken for manual testing on the
                existing test cases and concentrated only on the newer features. This led to a

                                                                                HMBP 2011 | M∆
P a g e | 17


              decrease in the total time for of the testing phase and at the same time
              quality was also kept in check.

          Perform a M∆ at a unit level. Repeat the same steps further up the hierarchy.

Conclusion:

      Once these areas have been taken care of, the organization remains CMMI Level 3
      appraised but has a good chance to be appraised to Level 4, as all the other process
      areas are taken care of.

      CAR = No delta.

      OPM = No delta


                                            Epilogue
                                          Accomplished

      In this paper, there has been an endeavour to bring out a simple method which
      organizations can follow to determine the maturity level differences in their processes
      and standards in pursuit of High Maturity. The M∆ approach can be applied to CMMI
      for Services (CMMI-SVC), CMMI for Acquisition (CMMI-ACQ) models as well and
      evaluate the process areas which would need improvement and assist the
      organizations to achieve High Maturity.
      Hopefully, M∆ at the least provides a way of singling out processes or standards
      which need to re-worked upon. This has been an earnest endeavour to put forth the
      ideas presented in the paper. Organizations can validate the ideas put forth here and
      find out if it really helps.




                                                                             HMBP 2011 | M∆
P a g e | 18


                              References


1. CMMI® for Development, Version 1.3. (http://www.sei.cmu.edu)
2. Self Ideation
3. Wikipedia


                         Author’s Biography

    Possessing 5.5+ yrs of experience in Software Testing/Quality Assurance.
      Currently employed with Cisco Systems India Pvt Ltd., as a QA Engineer.
      Advanced Level ISTQB Certified (Test Analyst)
      Presented a White Paper at Software Testing Conference 2010 conducted by
       QAI.
      A Passionate Tester.




                                  Appendix

CAR    -      Causal Analysis and Resolution
CMMI   -      Capability Maturity Model Integration
DAR    -      Decision Analysis and Resolution
IPM    -      Integrated Project Management
M∆     -      MDelta (Maturity Delta)
ML     -      Maturity Level
OPD    -      Organizational Process Definition
OPF    -      Organizational Process Focus
OPM    -      Organizational Performance Management
OPP    -      Organizational Process Performance
OT     -      Organizational Training
PA     -      Process Area
QPM    -      Quantitative Project Management
PI     -      Product Integration
RD     -      Requirements Development
RSKM   -      Risk Management
TS     -      Technical Solution
VAL    -      Validation
VER    -      Verification




                                                                  HMBP 2011 | M∆

MDelta-Quantifying Maturity

  • 1.
    2nd International Colloquium On High Maturity Best Practices 2011 (HMBP 2011) Presenting a Paper On M∆ - MDelta Quantifying Maturity By Sushant Mukund Hublikar E-mail: shublika@cisco.com Ph: +91-9886752443 Cisco Systems India Pvt. Ltd., Salarpuria Hallmark, Kadubeesanahalli, Opposite to New Horizon College, Outer Ring Road. BANGALORE – 560061
  • 2.
    Page |2 Abstract The main intention of this paper is to bring into focus, a new model developed to gauge the maturity levels of the organizations or the independent business units within organizations and to provide a simple method of evaluating focus areas to achieve higher maturity levels and sustain at those levels with continuous improvement. In this paper, we would be mainly talking about MDelta, the model being proposed, its need, applications and outcome. Prologue Models, Maturity Levels and M∆ Delta (∆) is a Greek alphabet which signifies finite differences between functions, basically used in mathematics and computational analysis. In this paper, we have proposed a new model appropriately called as MDelta (M∆). This model basically constitutes methods/techniques of evaluating “differences” between the immediate maturity levels of the CMMI Model. CMMI best practices are published in documents called models, each of which addresses a different area of interest. CMMI currently addresses three areas of interest: 1. Product and service development - CMMI for Development (CMMI-DEV), 2. Service establishment, management, and delivery - CMMI for Services (CMMI- SVC), 3. Product and service acquisition - CMMI for Acquisition (CMMI-ACQ). The current release of CMMI is Version 1.3, developed in November 2010. Using the improvement path known as the staged representation enables you to achieve “maturity levels”. Maturity levels apply to an organization’s process improvement achievement across multiple process areas. These levels are a means of improving the processes corresponding to a given set of process areas (i.e., maturity level). There are 5 maturity levels numbered 1 through 5. • The maturity levels are measured by the achievement of the specific and generic goals associated with each predefined set of process areas. CMMI Levels 4 and 5 are known as the High Maturity Levels. Achieving maturity level 4 involves implementing all process areas for maturity levels 2, 3, and 4. Likewise, achieving maturity level 5 involves implementing all process areas for maturity levels 2, 3, 4, and 5. • M∆ attempts to quantify the maturity and ascertains the progress required in the process areas of the lower maturity levels to elevate to the high maturity levels. HMBP 2011 | M∆
  • 3.
    Page |3 Endeavouring toquantify the maturity indicates the need of the organizations to be “appraised” to higher maturity levels and maintain a stable process improvement cycle. Adhering to the focus of this year's Colloquium, "High Maturity: A Practical Approach", this paper makes a sincere effort to address on “How to Implement” rather than “What Activities To Implement”. • CMMI for Development (CMMI-DEV) model has been used as the model for reference in this paper to demonstrate the usage of M∆. Changing trends in the market currently are at the highest pace and so are the customer needs. Therefore, there is a need for organizations to adhere to the customer desires by bringing out quality products without any slippage in time and at the same time maintaining standard processes and standards throughout the lifecycle. This can be gauged better with the help of a quantifiable entity which would recognize the areas needing enhancements. M∆ attempts to do the same. HMBP 2011 | M∆
  • 4.
    Page |4 Let’s Achieve! Implementation of M∆ Let us consider the process areas of the High Maturity levels and the immediate lower Maturity level 3: CAR - Causal Analysis and Resolution Maturity Level 5 Optimizing OPM - Organizational Performance Management High Maturity Levels Maturity Level 4 OPP - Organizational Process Performance Quantitatively Managed QPM - Quantitative Project Management DAR - Decision Analysis and Resolution IPM - Integrated Project Management OPD - Organizational Process Definition OPF - Organizational Process Focus OT - Organizational Training Lower Maturity Maturity Level 3 Level PI - Product Integration Defined RD - Requirements Development RSKM - Risk Management TS - Technical Solution VAL – Validation VER – Verification Table 1: Immediate Maturity Levels HMBP 2011 | M∆
  • 5.
    Page |5 Maturity Level3 (Defined) to Maturity Level 4 (Quantitatively Managed): To achieve High Maturity Level 4, we need to understand the Maturity Level 3 areas where the organizations/business units require regulation. Therefore, we are suggesting a simple method where we can divide the nine process areas of the ML 3 into groups equal to as many process areas of ML 4 i.e. a set of 4 process areas (Group 1) in ML 3 has been directly mapped to the Organizational Process Performance (OPP) PA of ML 4 and the remaining 7 (Group 2) have been mapped to the Quantitative Project Management (QPM) PA of the ML 4 respectively. This grouping or classification has been done based on the specific goals associated with each of the PA of ML 3 which closely satisfies the goals of the corresponding PA of ML 4. To elaborate, let us consider OPP => Organizational Process Performance PA of ML 4. The purpose of OPP is to • Establish and maintain a quantitative understanding of the performance of selected processes in the organization’s set of standard processes in support of achieving quality and process performance objectives • To provide process performance data, baselines, and models to quantitatively manage the organization’s projects. Now, scanning the PAs of ML 3 which closely relate to OPP, we have the following: OPD => Organizational Process Definition involves establishing and maintaining a usable set of organizational process assets, work environment standards, and rules and guidelines for teams. OPF => Organizational Process Focus which involves planning, implementing, and deploying organizational process improvements based on a thorough understanding of current strengths and weaknesses of the organization’s processes and process assets. OT => Organizational Training involves developing skills and knowledge of people so they can perform their roles effectively and efficiently. Similarly, let us consider QPM => Quantitative Project Management PA of ML 4. The purpose of QPM is to • Quantitatively manage the project to achieve the project’s established quality and process performance objectives. The PAs of ML 3 which closely relate to QPM are: DAR => Decision Analysis and Resolution involves analyzing possible decisions using a formal evaluation process that evaluates identified alternatives against established criteria. HMBP 2011 | M∆
  • 6.
    Page |6 IPM => Integrated Project Management involves establishing and managing the project and the involvement of relevant stakeholders according to an integrated and defined process that is tailored from the organization’s set of standard processes. PI => Product Integration involves assembling the product from the product components, ensuring that the product possesses the required functionality and quality attributes, and delivering the product. RD => Requirements Development involves analyzing, and establishing customer, product, and product component requirements. RSKM => Risk Management involves identifying potential problems before they occur so that risk handling activities can be planned and invoked as needed across the life of the product or project to mitigate adverse impacts on achieving objectives. TS => Technical Solution involves selecting, designing, and implementing solutions to requirements. VAL => Validation involves demonstrating that a product or product component fulfils its intended use when placed in its intended environment. VER => Verification involves ensuring that selected work products meet their specified requirements. Referring to the figure below, we have: M∆ = ML 4 – ML 3 M∆1 = f (OPP) - f (OPD + OPF + OT) M∆2 = f (QPM) – f (DAR + IPM + PI + RD + RSKM + TS + VAL + VER) ML 3 M∆ = M∆1 + M∆2 This would help to quantify or single out the areas which need focus and improvements to achieve higher maturity. DAR IPM ML 4 OPD OPF OT PI RD RSKM TS HMBP 2011 | M∆ VAL VER Figure 1: Mapping PAs of ML 3 to ML 4
  • 7.
    Page |7 OPP QPM Maturity Level 4 (Quantitatively Managed) to Maturity Level 5 (Optimizing): To achieve High Maturity Level 5, the levels of maturity level 4 need to be “sustained” and constantly improved upon. M∆ = ML 5 – ML 4 M∆ = f (CAR + OPM) – f (OPP + QPM) Let us consider CAR => Causal Analysis and Resolution PA of ML 5 The purpose of CAR is to • Identify causes of selected outcomes and take action to improve process performance. • Improving quality and productivity by preventing the introduction of defects or problems and by identifying and appropriately incorporating the causes of superior process performance. The QPM PA of ML 4 relates closely to the CAR PA of ML 5 due to the following similarities: CAR mainly deals with the identifying and analyzing causes of selected outcomes and taking actions to remove causes, prevent the recurrence of defects and problems in the future, and proactively analyze data to identify potential problems and prevent them from occurring whereas one of the specific goals of QPM involves performing root cause analysis of selected issues to address deficiencies in achieving the project’s quality and process performance objectives. HMBP 2011 | M∆
  • 8.
    Page |8 Similarly, thepurpose of OPM => Organizational Performance Management PA of ML 5 is • Proactively manage the organization’s performance to meet its business objectives. • Iteratively analyzing aggregated project data, identifying gaps in performance against the business objectives, and selecting and deploying improvements to close the gaps. Similarly the OPM PA of ML 5 is satisfied by the combined achievement of OPP and QPM PAs of ML 4. The OPM PA addresses improved product quality (e.g., functionality, quality attributes), increased productivity, increased process efficiency and effectiveness, increased consistency in meeting budget and schedule, greater customer and end-user satisfaction, shorter development or production time to change functionality, add new features, or adapt to new technologies, and improved use of resources across the organization while the OPP PAs satisfy them at the same level by addressing: organizational quantitative quality and process performance objectives based on business, establishing definitions of the measures to be used in process performance analyses, establishing process performance baselines and process performance models along with QPM which involves selecting sub-processes and attributes critical to understanding performance and that help to achieve the project’s quality and process performance objectives, selecting measures and analytic techniques to be used in quantitative management, monitoring the performance of selected sub-processes using statistical and other quantitative techniques, managing the project using statistical and other quantitative techniques to determine whether or not the project’s objectives for quality and process performance are being satisfied. ML 4 ML 5 OPP CAR QPM OPM Figure 2: Mapping PAs of ML 4 to ML 5 Therefore, M∆ = ML 5 – ML 4 M∆1 = f (CAR) - f (QPM) HMBP 2011 | M∆
  • 9.
    Page |9 M∆2 = f (OPM) – f (OPP + QPM) M∆ = M∆1 + M∆2 High Maturity Process Areas require more of improvements than implementation. The processes are there and standards are set but the focus is more towards how well the processes are sustained and how actively are the standards followed to make it a regular way of doing business. In the next section, we will look a few case studies for the application and results of M∆. HMBP 2011 | M∆
  • 10.
    P a ge | 10 Case Study Applications and Results of M∆ Scenario 1: A software development organization currently appraised at CMMI Maturity Level 3 is aiming for a High Maturity appraisal. It has established and implemented all the required processes and achieved the specific/generic goals of Level 3. However, though the products being developed by the organization are delivered on time to the market, there is gradual decrease in the quality of the products and the customers are circumspect about this. The organization stands at a precarious position where it may be demoted from the Level 3 appraisal during the next audit cycle. The board of directors, the engineering heads and the senior management set up a meeting to discuss the areas they are falling behind and how to overcome and improve the processes followed. Approach: • Identify the Process Areas of Level 3 where the processes are not being followed as recommended starting from the gathering the requirements. i. Are the product managers analyzing the customer needs and providing the proper requirements? => RD ii. Has the correct solution for the requirement, design and development been selected to build the product? => TS iii. Are the required product components (infrastructure/environment) being used correctly to build the right product to possess the required functionality and quality attributes? => PI iv. Does the final product fulfil its intended use when placed in its intended environment? => VAL v. Does the final product meet the specified requirements? => VER vi. Are the correct project management processes implemented for the product development followed with the involvement of the relevant stakeholders? => IPM vii. Are the risks encountered during the project handled effectively and on priority? => RSKM viii. Are the organizational process assets, work environment standards, rules and guidelines employed as per the standards set by the organization? => OPD ix. Has the organization identified the correct processes to be followed based on the thorough understanding of its current strengths and weaknesses? => OPF x. Are the resources well-trained enough to perform their roles effectively and efficiently? => OT HMBP 2011 | M∆
  • 11.
    P a ge | 11 xi. Are correct decisions being taken at the right time that has a minimal effect on the ability of the project to achieve its objectives? => DAR Deduce: • According to the scenario, the product is delivered “on time” but the quality is decreasing. Considering this situation, we can shortlist RD, TS and PI as the three areas which need improvement. M∆ = ML 4 – ML 3 M∆1 = OPP – (OPD + OPF + OT) = No differences. M∆2 = QPM – (DAR + IPM + PI + RD + RSKM + TS + VAL + VER) = PI + RD + TS M∆ = M∆1 + M∆2 = PI + RD + TS How To Implement: • Now, analyse the specific goals of each of these PAs: i. RD: Are the product managers analyzing the customer needs and providing the proper requirements? No > Analyse the market and find out the trends and the changing customer needs Yes > The requirements gathered are correct. ii. TS: Has the correct solution for the requirement, design and development been selected to build the product? No > The solution suggested is incorrect because of ambiguous requirements. Yes > The solution decided upon is correct. iii. PI: Are the required product components (infrastructure/environment) being used correctly to build the right product to possess the required functionality and quality attributes? No > The product components are causing a change in the functionality. Yes > The product components used are correct Any of these three areas would have failed for the decrease in quality. Reworking on them will fetch good results and thereby identifies how to prevent such deviations in the future. Sustenance: • Sustaining at Level 3 and to continually improve the processes to appraise to Level 4, the organizations need to perform this activity, frequently, say, once in a quarter, on all the projects that are delivered during that time period. HMBP 2011 | M∆
  • 12.
    P a ge | 12 How?  Perform a M∆ for each project.  Ensure there are no process areas identified for improvements.  Ensure the specific process area goals are achieved. Ex: In the above scenario, in the next release of the product, validating the requirements with the customer needs by direct interactions, sharing the solution to verify the requirements and frequent display of the product prototypes will immensely help to get a clear picture on the final solution. Similarly, in the coming release, following the same process leads to the evolution of an organizational standard which every project would follow. Conclusion: Once these areas have been taken care of, the organization remains CMMI Level 3 appraised but has a good chance to be appraised to Level 4, as all the other process areas are taken care of. QPM = No delta. OPP = No delta Scenario 2: A small-scale organization has done well to reach CMMI Level 3 and is looking for an appraisal towards CMMI Level 4. Customers are readily offering projects to the organization and the need for more human resources has been answered by an urgent recruitment drive. The hasty process has ended up in candidates with false experience joining the organization. This has dealt a severe blow to the confidence level of the customers with their projects delivered with a huge number of faults and the standard processes to be followed have also deviated in order to meet the deadlines. The management are in a fix to identify the crux of this issue. Approach: • Identify the Process Areas of Level 3 where the processes are not being followed as recommended starting from the gathering the requirements. i. Are the correct project management processes implemented for the product development followed with the involvement of the relevant stakeholders? => IPM ii. Are the risks encountered during the project handled effectively and on priority? => RSKM iii. Are the organizational process assets, work environment standards, rules and guidelines employed as per the standards set by the organization? => OPD HMBP 2011 | M∆
  • 13.
    P a ge | 13 iv. Has the organization identified the correct processes to be followed based on the thorough understanding of its current strengths and weaknesses? => OPF v. Are the resources well-trained enough to perform their roles effectively and efficiently? => OT vi. Are correct decisions being taken at the right time that has a minimal effect on the ability of the project to achieve its objectives? => DAR Deduce: • According to the scenario, the processes have been deviated and the resources are not “well-trained” to perform their roles efficiently. Considering this situation, we can shortlist OPD, OPF, OT, IPM and RSKM as the areas which need improvement. M∆ = ML 4 – ML 3 M∆1 = OPP – (OPD + OPF + OT) = OPD + OPF + OT M∆2 = QPM – (DAR + IPM + PI + RD + RSKM + TS + VAL + VER) = IPM + RSKM M∆ = M∆1 + M∆2 = OPD + OPF + OT + IPM + RSKM + DAR How to Implement: • Analysing the specific goals of the PAs i. Are the correct project management processes implemented for the product development followed with the involvement of the relevant stakeholders? => IPM No > The processes were not followed as recommended. The last minute rush to delivery has heavily impacted the adherence to the processes. Yes > The standard processes have been followed effectively. ii. Are the risks encountered during the project handled effectively and on priority? => RSKM No > The correct risks were not identified and overall the project was delivered open to risks. Yes > The risks were identified and the appropriate mitigation plan were put in place. iii. Are the organizational process assets, work environment standards, rules and guidelines employed as per the standards set by the organization? => OPD No > The infrastructure/environment required for the under-trained resources and the time (and in turn, money) spent to make them acquaint with the project was heavily utilized thus violating the threshold of their usage. There were also cases of attrition because of the pressure involved thereby the deteriorating the work environment. HMBP 2011 | M∆
  • 14.
    P a ge | 14 Yes > The assets, standards were efficiently followed and there was professionalism maintained at the workplace. iv. Has the organization identified the correct processes to be followed based on the thorough understanding of its current strengths and weaknesses? => OPF No > The organization failed in the process of staffing the right candidates required for the job. There was no background verification done on the candidates and hence had to bear the embarrassment for the poor project delivery. Yes > The organization strengthened it stronger areas of operation and utilized minimal of its weakest areas. Ex: Being a small scale organization, the new candidates faced the shortage of owning work place desks for themselves. Therefore, the organization decided to depute them at the client’s place directly. This will provide more interaction with the client and work getting done faster. v. Are the resources well-trained enough to perform their roles effectively and efficiently? => OT No > The recruited candidates were not enough experienced to handle the projects as soon as they were on the job. There was a sufficient amount of training required thereby using up much important time on the same. Yes > The candidates were well suited for the job and the right resources have been employed to handle the projects on the go. Again, failure in any of these areas would have led to the deviations in the processes. Sustenance: • Re-define the organizational processes frequently and communicate the same to each and every employee, including new hires. Any new hire would obviously require some time to get acclimatized to the work and the work place but ensuring that the time is within the consumption of the overall project is a challenge to maintain. This can be achieved only by setting up a standard documentation for every new hire and revisit them often for changes in line with the organizational standards. Also, revisit the organizational processes of the recruiting/staffing unit and make certain that the correct practices as followed. How?  Perform a M∆ after each medium-to-large scale recruitment drive.  Ensure there are no process areas identified for improvements.  Ensure the specific process area goals are achieved. Ex: In the above scenario, in the next event of recruitment, identifying candidates who satisfy the requirements completely or who are certified in the technology required for the organization would be a good practice rather than accepting candidates who just closely match the requirements. Also, putting in place an entire new hire training unit department with a detailed HMBP 2011 | M∆
  • 15.
    P a ge | 15 outlined courses on the technologies needed would help the organization save time on training and thereby utilizing the candidates immediately on the job and delivering the projects on time. Following the same process during every hiring campaign leads to the evolution of an organizational standard which forms the core of organizational processes. Conclusion: Once these areas have been taken care of, the organization remains CMMI Level 3 appraised but has a good chance to be appraised to Level 4, as all the other process areas are taken care of. QPM = No delta. OPP = No delta Scenario 3: An IT bellwether appraised at CMMI Level 4 is pursuing CMMI Level 5 achievement. It has well-laid organizational standards, efficient project specific processes, best practices in the fields of Human Resources and technology. Customers are very keen on doing business with the organization as they are aware of the kind of quality with which the projects are delivered with and their promises being kept. However, sustaining the improvements within the processes has been a minor hiccup with the company. The external auditing firm has listed the areas which need “further” improvement. Approach: • Identify the Process Areas of Level 4 where the processes require more precision and the projects where the standards need to be maintained. i. Are the organization’s set of standard processes in support of achieving quality and process performance objectives, and to provide process performance data, baselines maintained? => OPP ii. Are the projects managed quantitatively to achieve the project’s established quality and process performance objectives? => QPM Deduce: • According to the scenario, the organization has well-defined standards and has created a stronghold for itself in the market with respect to the quality of work delivered. In this case, M∆ = ML 5 – ML 4 M∆1 = CAR - QPM M∆2 = OPM – (OPP + QPM) HMBP 2011 | M∆
  • 16.
    P a ge | 16 M∆ = M∆1 + M∆2 = OPP + QPM How to Implement: • Analyzing the specific goals of the PAs i. Are the organization’s set of standard processes in support of achieving quality and process performance objectives, and to provide process performance data, baselines maintained? => OPP No > The products were not delivered within budget and on time, the productivity was not improved, the customer ratings did not increase and they remained the same. Yes > The products were delivered on time and were of high quality and even the customer ratings have shot up. Time-to-market has improved by a specified percent. ii. Are the projects managed quantitatively to achieve the project’s established quality and process performance objectives? => QPM No > The products performance attributes such as Usability, Reliability, and Maintainability were not improved. The cycle time and the percentage of rework time did not reduce. Schedule slippage did not improve. Yes > The products function much better now, the mean time between failures is at an all-time low, the total lifecycle cost has been reduced by a specified percent. Stagnation in any of the above areas would have led to the deviations in the processes. Sustenance: • Constantly improving the processes set by the organization and regularly monitoring them at equal intervals of a time period assists in recognizing the best practices being followed within the business units as well as the organizational level. Drill down to the individual teams of the business units and the processes they follow to carry out their roles efficiently. Ensure the same is being done at the unit level, being in sync with the set standards. How?  Perform a M∆ at the individual team levels.  Check the process areas which need “further” improvement.  Ensure the specific process area goals are achieved Ex: A team comprising of certified testers who were also genuinely experienced in their domain followed the process of improving their cycle time by constantly automating the test cases developed for the previous release. By doing so, they were able to reduce to time taken for manual testing on the existing test cases and concentrated only on the newer features. This led to a HMBP 2011 | M∆
  • 17.
    P a ge | 17 decrease in the total time for of the testing phase and at the same time quality was also kept in check.  Perform a M∆ at a unit level. Repeat the same steps further up the hierarchy. Conclusion: Once these areas have been taken care of, the organization remains CMMI Level 3 appraised but has a good chance to be appraised to Level 4, as all the other process areas are taken care of. CAR = No delta. OPM = No delta Epilogue Accomplished In this paper, there has been an endeavour to bring out a simple method which organizations can follow to determine the maturity level differences in their processes and standards in pursuit of High Maturity. The M∆ approach can be applied to CMMI for Services (CMMI-SVC), CMMI for Acquisition (CMMI-ACQ) models as well and evaluate the process areas which would need improvement and assist the organizations to achieve High Maturity. Hopefully, M∆ at the least provides a way of singling out processes or standards which need to re-worked upon. This has been an earnest endeavour to put forth the ideas presented in the paper. Organizations can validate the ideas put forth here and find out if it really helps. HMBP 2011 | M∆
  • 18.
    P a ge | 18 References 1. CMMI® for Development, Version 1.3. (http://www.sei.cmu.edu) 2. Self Ideation 3. Wikipedia Author’s Biography  Possessing 5.5+ yrs of experience in Software Testing/Quality Assurance.  Currently employed with Cisco Systems India Pvt Ltd., as a QA Engineer.  Advanced Level ISTQB Certified (Test Analyst)  Presented a White Paper at Software Testing Conference 2010 conducted by QAI.  A Passionate Tester. Appendix CAR - Causal Analysis and Resolution CMMI - Capability Maturity Model Integration DAR - Decision Analysis and Resolution IPM - Integrated Project Management M∆ - MDelta (Maturity Delta) ML - Maturity Level OPD - Organizational Process Definition OPF - Organizational Process Focus OPM - Organizational Performance Management OPP - Organizational Process Performance OT - Organizational Training PA - Process Area QPM - Quantitative Project Management PI - Product Integration RD - Requirements Development RSKM - Risk Management TS - Technical Solution VAL - Validation VER - Verification HMBP 2011 | M∆