Process Improvement: Based on Models, Implemented in Reality

Uploaded on


  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. Process Improvement: Based on Models, Implemented in Reality Better Software Conference and EXPO: W7 Wednesday, September 29, 2004, 11:30 AM By Barbara Ainsworth CSQA, CSTE Process Plus International, LLC Phone: 618.749.2080 Email:
  • 2. Introduction Disclaimer and Limitation of Liability: The author has used best efforts in designing and developing this presentation. There are no representations or warranties with respect to accuracy or completeness of the contents of this publication and specifically disclaim any implied warranties or merchantability or fitness for any particular purpose and shall in no event be liable for any loss of profit or any other commercial damage, including but not limited to special, incidental,consequential, or other damages Experience demonstrates that no matter the end product — financial services, insurance, beer, construction, etc. — the foundation for and focus on implementing process improvement remains consistent. Using models as the basis for improving processes makes success more likely; however, no single model has all the right answers. • Overview and comparison of popular models • Determine aspects of the available models that are right for you • Examples of starting and reinvigorating process improvement initiatives Acknowledgements: Thank you to Peer Review participants: Tom Ainsworth, Brad Carroll, Inger Dooley , Quince Hedley, Kimi Ziemski. Additional thanks to Brad Carroll for his UML expertise and willingness to share for the benefit of the community.
  • 3. Overview: Terms & Premise Process Improvement: An effort to improve the processes used in an organization to accomplish its business Model [Framework]: Structured collection of elements that describe characteristics of effective processes Model-Based Process Improvement: A process improvement effort that uses a model to appraise the quality of its current processes, identify and prioritize needed changes, and guide its improvement activities Process Management Premise: The quality of a system is highly influenced by the quality of the process used to acquire, develop and maintain it System Boundaries: The focus of process improvement must have defined boundaries that surround and contain the problem to be addressed by the PI initiative. May be entire organization, department, product, project, or function.
  • 4.
    • Business objectives are traceable to deliverables
    • Internal operational efficiency; lower costs; less rework
    • Metrics indicate bottom line impact
    • Greater customer retention and satisfaction, increased market share, and improved profitability
      • Evidence suggests a long-term link between the use of models and improved business performance, growth, and prosperity in the world marketplace
    • Some models are a means to earn certifications or awards
      • Business contracts may require certifications based on models
      • Some organizations use resultant certifications/awards as marketing tools
    Overview: Why Use Models?
  • 5.
    • Provide answers to important questions related to organization’s current maturity
    • Assess maturity of entire or specific parts of the organization; identify strengths & areas for improvement
    • Promote organizational maturity awareness among senior management
    • Attribute organizational success to process management
    • Better employee relations, higher productivity
    • Manage development, acquisition, and contractors/outsourcing processes
    • Cohesive, comprehensive approach to guiding individuals, managing projects and achieving organizational strategies
  • 6.
    • Numerous models from various organizations
    • Membership and/or Public Information
    • Some models are related; some content “matches”, overlaps, and/or links
    • Models provide starting place, benefit of experiences, common language/shared vision, framework for prioritizing actions, guide to define “improvement”
    • Support measurement; framework for assessment
    • Accepted widely across the US and around the world
    • Models improve over time
    Overview: About Models
  • 7.
    • Risks
    • There is no “silver bullet”
      • “ All models are wrong; some are useful” - George Box
    • Need to expand the depth and breadth in order to successfully implement
      • Some provide high level guidelines - the “what”; others provide more details - the “how”
      • Still need to address crucial project success issues:
        • professional judgement; appropriate model interpretation
        • expertise in particular application domains;
        • determination of specific software technologies;
        • selection/hiring/motivating/and retaining competent people
    • Some provide cross functional focus; others maintain stovepipes
    • Need STRONG implementation management
    Overview: About Models
  • 8.
    • Software Engineering Institute (SEI)
      • IDEAL  Model
      • Software Capability Maturity Model (SW_CMM  )
      • Capability Maturity Model Integrated (CMMI  )
      • Personal Software Process (PSP SM )
      • Team Software Process (TSP SM )
    • International Standards Organization (ISO) Family of Standards
    • Quality Assurance Institute’s (QAI) Implementation Approach Model and Bodies of Knowledge
    • Project Management Institute’s (PMI) Models and Body of Knowledge
    • Malcolm Baldrige National Quality Award (MBNQA)
    • Six Sigma
    • Agile
    • Unified Modeling language (UML)
    Overview: Models In Review
  • 9. Set Context Build Sponsorship Charter Infrastructure Characterize Current & Desired States Develop Recommendations Set Priorities Develop Approach Plan Actions Create Solution Pilot/Test Solution Refine Solution Implement Solution Analyze and Validate Propose Future Actions Initiating Diagnosing Establishing Acting Learning Stimulus for Change Do work according to plan Learn from experience & improve ability to adopt new technologies Lay groundwork for successful improvement Determine where you are relative to where you want to be Plan specifics how to reach destination * Overview: SEI IDEAL  Model Software Engineering Institute’s IDEAL  Model
  • 10. Overview: SEI SW_CMM  Level Focus Key Process Areas Result 5 Optimizing 4 Managed 3 Defined 2 Repeatable 1 Initial Continuous process improvement Product and process quality Engineering processes & organization processes defined; Performance more predictable Project management in place, individual performance repeatable Process informal and ad-hoc, unpredictable performance Defect prevention Technology innovation Process change management Process measurement and analysis Quality management Organization process focus Organization process defn. Peer reviews Training program Inter-group coordination Software product engineering Integrated software mgt. Requirements Mgt. Software Project planning Software Project Tracking & Oversight Software Quality Assurance Software Configuration Mgt. Software Subcontract Mgt. Software Test Mgt. [DRAFT] Productivity & Quality Risk SEI Software Capability Maturity Model v 1.1 *
  • 11. Overview: SEI CMMI  5 Optimizing 4 Quantitatively Managed 3 Defined 2 Managed 1 Initial Quality Productivity Risk / Rework SEI Capability Maturity Model Integrated v1.1: Staged Process Areas Requirements Management Project Planning Project Monitoring & Control Supplier Agreement Management Measurement & Analysis Process and Product Quality Assurance Configuration Management Requirements Development Technical Solution Product Integration Verification Validation Organization Process Focus Organization Process Definition Organizational Training Integrated Project Management Risk Management Integrated Teaming Integrated Supplier Management Decision Analysis and Resolution Organizational Environment for Integration Organizational Process Performance Quantitative Project Management Organizational Innovation and Deployment Causal Analysis and Resolution Focus Continuous Process Improvement Quantitative Management Process Standardization Basic Project Management 2002-2003 by Carnegie Mellon University Introduction to CMMI ® - Staged V 1.1 Module 9-090103 Pg. 12 Level
  • 12. Overview: SEI TSP SM /PSP SM PSP SM Skill-building TSP SM Team-building TSP SM Team-working Personal measures Process discipline Estimating & planning Quality management Project goals Team roles Team process Project plan Balanced plan Risk analysis Team communication Team coordination Status tracking Project reporting Team Members Team Management Integrated Product Teams * Nov 2003 Team Software Process SM & Personal Software Process SM Team Disciplines
  • 13. Overview: ISO
    • International Standards Organization
    • Quality Management System (QMS) has essential features where relevant ISO 9000/ISO 14000 families of standards provide requirements
    • THE ISO 9000: 2000_12_15 FAMILY OF STANDARDS - ISO 9000: Quality Mgmt. Systems - Fundamentals and Vocabulary ISO 9001: Quality Mgmt. Systems - Requirements ISO 9004: Mgmt. Systems - Guidelines for Performance Improvements ISO19011 is applicable to both quality and environmental mgmt system audits - The 20 elements in ISO 9001 contain five main clauses titled: Quality Mgmt.System; Mgmt.Responsibility; Resource Mgmt.; Product Realization; Measurement, Analysis & Improvement - ISO 9001 “Quality Management System” (QMS) Contents (free samples available) Overview; Manual Preparation; QMS Quality Manual; QMS Quality Procedures; EMS Quality Procedures; Reports and Forms; Index - The new standards have been aligned with the existing standards
    • ISO/IEC for Software [“software” search = 71 standards 1994 -2001] - ISO/IES 90003:2004: Guidelines for applying ISO 9001:2000 to software - ISO/IEC 12207: IT SW Life Cycle Processes - ISO/IEC 15504: IT SW Process Assessment Requires 3rd Party Certification
  • 14. Overview: QAI
    • Quality Assurance Institute’s Implementation Approach Model
    • Bodies of Knowledge containing Skill Categories for:
      • Quality Assurance, Testing, and Project Management
  • 15.
    • Project Management Institute
    • PROJECTS - Project Management Body of Knowledge (PMBOK)-
      • Initiation, Scope, Time, Cost, Quality, Human Resources, Communication, Risk, Procurement
    • PEOPLE - Project Manager Competency Framework (PMCF)-2002
      • Improving performance of project personnel
    • ORGANIZATIONS - Organizational Project Management (OPM3)-’03
      • a standard for organizational project management and organization project management maturity
      • describes an organization's overall ability to select and manage projects in a way that supports its strategic goals
      • Elements: Knowledge Foundation, Assessment & Improvement
      • use in concert with PMBOK and PMCDF
    Overview: PMI
  • 16.
    • Malcolm Baldrige National Quality Award
    • The Baldrige performance excellence criteria categories & focus:
    • Leadership- Senior executives guidance; organization’s responsibilities to the public; practices good citizenship
    • Strategic planning- Setting strategic directions; key action plans
    • Customer and market focus -Customer and market requirements & expectations; customer relationships
    • Measurement, analysis, and knowledge management -Data management; key processes & performance mgmt. system support
    • Human resource focus- Workforce development & alignment with business objectives
    • Process management- Design, management, and improvement of key production/delivery and support processes
    • Business results- Key business areas performance & improvement; performance relative to competitors
    Overview: MBNQA
  • 17. Overview: Six Sigma
    • Highly disciplined process to focus on developing and delivering near-perfect products and services
    • There are three key elements of quality: Customers: Delighting Customers; Process: Outside-In Thinking; Employee: Leadership Commitment
    • All GE employees are trained in the strategy, statistical tools and techniques of Six Sigma quality. Training courses are offered at various levels
    • Six Sigma Quality Approaches and Models
      • Design for Six Sigma (DFSS): a systematic methodology utilizing tools, training & measurements to enable products and processes design that meets customer expectations & be produced at Six Sigma quality levels.
      • Define, Measure, Analyze, Improve and Control (DMAIC): a systematic, scientific, fact based, closed-loop continuous improvement process
      • Six Sigma – A vision of quality which equates with only 3.4 defects per million opportunities for each product or service transaction. Strives for perfection.
  • 18.
    • Agile Software Development
    • A Philosophy - 4 value statements and 12 principles (Agile Manifesto)
    • A Set of methods - over 10 methods & method families
    • A collection of practices - e.g., 19 engineering practices in eXtreme Programming (XP) alone
    • Be Agile: Slash the budget; If it doesn't work, kill it; Keep requirements to a minimum; Build on success, not hope; Keep your development teams small; Assign non-IT executives to software projects
    Overview: Agile Note cautions at comparison slides
  • 19.
    • Unified Modeling Language™
    • Object Management Group - Independent non-profit organization that researches, maintains and publishes UML, Common Warehouse Metamodel (CWM™), MOF™ 2.0 XMI® ( XML Metadata Interchange) and other standards
    • Nine UML Models – Use Case, Class Diagram, Sequence Diagram, Activity Diagram, State Diagram, Collaboration Diagram, Component and Deployment
    • Universality – Commonly accepted and used throughout software industry, domestically and internationally, to model and communicate software functional specifications, architecture and designs
    Overview: UML
  • 20. Comparison: Commonality
    • Terminology can be industry common or model unique
    • Process Focus
      • Required processes and process content differ by model
      • Processes,procedures,practices,documentation,gates/status indicators, etc.
    • Tools,Training,Support,Assessments,Metrics, Continuous Improvement
    • Improved over time
    • Numerous references, tools, and other guidance [interpreting, documenting, training, FAQs and answers, survival guides, business mapping techniques, support links, sample documentation, style guidelines, workbooks, etc.]
    • Certified companies
    • Compared to other models
    • Can be successfully used; or disastrous
    Input Tasks, Tools & Techniques Output
  • 21. Comparison: SEI CMM/CMMI ® Level CMMI ® Process Areas 5 Optimizing 4 Managed 3 Defined 2 Repeatable Quantitative Process Management Software Quality Management Organization Process Focus Organization Process Definition Training Program Integrated Software Management Software Product Engineering Intergroup Coordination Peer reviews SW_CMM ® v1.1 Key Process Areas Defect Prevention Technology Change Management Process Change Management Requirements Management Software Project Planning Software Project Tracking & Oversight Software Subcontract Management Software Quality Assurance Software Configuration Management Software Test Management [DRAFT] Requirements Management Project Planning Project Monitoring & Control Supplier Agreement Management Product and Process Quality Assurance Configuration Management Measurement & Analysis Organization Process Focus Organization Process Definition Organizational Training Integrated Project Management Risk Management Requirements Development Technical Solution Product Integration Verification Validation Decision Analysis and Resolution Organizational Process Performance Quantitative Project Management Causal Analysis and Resolution Organizational Innovation & Deployment 2003 Carnegie Mellon University Software Engineering Institute
  • 22. Comparison: CMMI ® /TSP SM /PSP SM CMMI ® Builds organizational capability TSP SM Builds quality products on cost and schedule PSP SM Builds Individual skill and discipline *
  • 23. Comparison: CMM/ISO9000 * The Capability Maturity Model: Guidelines for Improving the Software Process : CMU SEI pg.417 *Software Engineering Institute’s Technical Report: CMU/SEI-94-TR-12; See appendices re detailed listing - what practices are addressed where. SEI SW_CMM ® compared to ISO-9001 Compliant Organization
    • SEI CMMI ® c ompared to ISO-9001:2000
      • Similarities, differences, synergies
      • CMMI ® missing some ISO items
  • 24. Comparison: PMI/SEI/QAI Sample Model Comparison - Not intended to be comprehensive
  • 25.
    • MBNQA to ISO 9000 - ISO 9000 registration covers less than 10% of Baldrige Award criteria - Responsible for making quality a national priority & disseminating best practices across USA - For organizations utilizing Six Sigma Quality, TQM, ISO9000 and process improvement - MB feedback report is arguably the best bargain in consulting in USA; improve organization or business overall health
    • CMMI ® To OTHER MODELS - CMMI ® to Six Sigma to Achieve Levels 4 & 5: DMAIC process aligns with CMMI ® practices - Six sigma process is applied to CMMI ® process improvement optimizing processes at Level 5 - Using CMMI ® to Balance Agile and Plan-Driven Methods
  • 26.
    • Agile to ISO and CMM - Some common CMM & ISO practices at odds with Agile value set - CMM relies on processes, plans, and delivering to original contract; ISO requirements not met - If need CMM certification; less interest in Agile value proposition - If need agility for business reasons; less interest in CMM or ISO 9000 certification
    • TSP SM Compared to Agile
      • Part compatible with TSP SM /PSP SM ; not all TSP SM /PSP SM met
      • In practice, the two approaches tend to align with different : - Philosophies on design - Degrees of formalism in design verification - Levels of team data gathering and analysis
    Comparison: Caution: - Agile Development has hit a nerve. Some argue for it, some strongly aginst it, and others mix agile &plan-driven approaches. - “Considered Harmful for Reliable Software Development” - 6/2/02
  • 27. Comparison: UML / SW_CMM ® Sample Model Comparison - Not intended to be comprehensive
  • 28. Comparison: Chart
    • Mappings are subjective
    • Data shows that some models/frameworks:
      • measure for Awards, Certifications, Classifications
      • meet and exceed requirements of others
      • partially meet/provide high level guidance for others
      • support/can be used to help achieve requirements of others
      • have some conflict with others
      • lack sufficient data for comparison
    Which fit your organization’s criteria?
  • 29.
    • Symptoms Indicating Need for Process Improvement
    • Slow to market product development
    • Difficulty translating strategic vision to operational reality
    • High failure rate of software projects
    • No defined process owners
    • Depend upon heroics of one or more people to get things done
    • Organizational silos inhibit communication
    • Overly dependent upon outside experts for product or project knowledge
    Determine Right Model: Factors
  • 30.
    • Categories of factors in your decision
    • Business - How well known are the business objectives? How mature are the current processes? How strongly are processes mapped to the business objectives? Will improvements be made across the organization or within a product line? Which business objectives will be supported by the Process Improvement Program (PIP)?
    • Culture - What is the organization’s ability to deploy a PIP? Is the organization process based and experienced in PI? Does a specific process need improvement quickly?
    • Legacy - Is the organization experienced with a model? Has the organization invested resources and deployed processes across the organization using a model?
    Determine Right Model: Factors *CMMI ® Guidelines for Process Integration and Product Improvement
  • 31.
    • Review empirical data of organizations using the model
      • SEI data: SEIR is repository of data; must register to use; no cost; can re-register and de-register; not as much data on CMMI ® as SW_CMM ®
      • MBNQA organizations are required to communicate/share
      • ISO certified companies are listed
      • Conference materials are on the internet
    • Match business needs criteria to model
      • What is expected ROI?
    • Define the process improvement system boundaries
    • Criteria considerations:
      • Standard yet scalable, flexible and customizable
      • culture, staff size, personnel, system critically [consequences of error], application domain, size, optimized quality, dynamism, innovativeness h ttp://
    Determine Right Model: Tips
  • 32. Where are you? What’s Missing? Elements of Change: Effects of Missing Links SEI Visitors Day, June 20, 1996 - Transition Enabling, Mike Phillips from Ambrose, 1987 Start/Re-Energize PI: Current Status? What’s in place?
  • 33.
    • Organizational Commitment
    • Choose your approach to process improvement
      • Models are the foundation; need to expand depth and breadth in order to successfully implement
      • Ensure Trainers, Consultants are “in-the-trench” experienced
    • Deploy Project Management practices on PI Initiative
    • Team Approach
    • Address “What’s in it for me”
    • Implementation is Key - Institutionalization is Critical !
      • Only Some “Model Experts” are needed when the processes are compliant to model requirements
      • Drivers & influencers; access to all; training; compliance; small wins; measure & improve Manage Resistance
    Start/Re-Energize PI: Tips
  • 34.
    • Example: Use Adaptive or Predictive Process? Consider the following...
    • 1.) (a) Factors suggest adaptive process:
      • Uncertain or volatile requirements
      • Responsible and motivated developers
      • Customer who understands & will get involved
    • (b) Factors suggest predictive process:
      • A team of over a hundred
      • Fixed price, or more correctly a fixed scope, contract
    • 2.) [need] developers and business execs alike—who can follow ...principles of flexibility, teamwork, trust, & reflection. ...these environments...few & far between.
    • 3.)(a) Combine TSP SM /Agile advantages based on org’s & project characteristics (b) Criteria considerations: culture, staff size, personnel, system critically [consequences of error], application domain, size, optimized quality, dynamism, innovativeness h ttp://
    Not intended as a comprehensive list of considerations. If “adaptive” then...? If “predictive then…? Start/Re-Energize PI: Example
  • 35.
    • Implementation Examples
    • Malcolm Baldrige National Quality Award
      • 1988-2003 Recipient’s Contacts & Profile
      • Motorola
        • First MBNQA winner; Implemented "Motorola University, Six Sigma, Total Quality Management (TQM), in early 1990's.
        • Includes rear-end software to track measurements
    • Six Sigma
      • General Electric
        • Originated Six Sigma;” the way they do business”; everyone responsible & trained
      • Raython
        • Six Sigma Process and Its Application for CMMI®
    • ISO
      • Certified Companies search by ISO CERTIFIED COMPANIES provides specific listings, directories of all listings, etc.
    Start/Re-Energize PI: Examples
  • 36.
    • Implementation Examples
    • IDEAL ® , SW_CMM ® , CMMI ® ,QAI, PMI by Barbara Ainsworth [email_address]
      • Focus on Process Improvement using the IDEAL ® in concert with
        • Level 2 and Level 3 KPAs; transition to CMMI ® Level 2 & 3
        • Project Management, Quality Assurance, and Testing principles and practices
    • IDEAL® & ISO & CMMI ® by Hughes & Bearing Point: 2003
      • Focus on Process Improvement using the IDEAL® problem solving process and ISO 9001:2000 and CMMI ® synergy
    • CMMI ® Level 2 to Level 5 by David F. Rico
      • Implementation Plan
      • Work Breakdown Structure
      • Policies and Procedures (>500 pgs.)
      • Cost Model
    Start/Re-Energize PI: Examples
  • 37. Summary
    • Base process improvement on business objectives; trace through implementation; measure ROI
    • Models/frameworks are the foundation; need to expand depth and breadth in order to successfully implement
    • Choose and use models/frameworks wisely
    • Implementation is key; implement compliant processes and measure compliance
    • Change takes time, commitment, resources
    Questions? Need help? Contact info is on title page...
  • 38. References
  • 39. References
  • 40. References
  • 41. References