• Like
  • Save


Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.


A Method Assessment Framework

A Method Assessment Framework

Published in Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads


Total Views
On SlideShare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. A Method Assessment Framework Tom McBride Brian Henderson-Sellers School of Software University of Technology, Sydney
  • 2. There is a need for a method assessment framework
    • There has been little discussion of how the quality of a method might be assessed, or when
    • A growing number of communities are developing methods without the aid of method engineers
    • We believe that such initiatives can be supported by an assessment framework that separates the concerns of different phases in the life cycle of a method (design, enactment, performance)
  • 3. A method can be assessed at design, enactment and performance
    • Method design
      • When a method is created in order to achieve a specific purpose but in a range of circumstances
      • e.g. a method suited to developing life critical software using medium to large teams under tight schedule and budget constraints
    • Method enactment
      • When a method is tailored to the specific circumstances of a project
      • Project planning once the project requirements, schedule, team, budget, resources, environment have been decided.
    • Method performance
      • When the method is being used
  • 4. Process design and enactment
  • 5. What is method quality
    • Is the method suitable for its intended purpose in the circumstances for which is intended?
    • Is the method complete?
      • Work products missing, created but not used, used but not created
    • Is the method actually needed for the particular application
  • 6. Method design
    • Software development methods are created to address a range of circumstances
      • Waterfall development
      • Spiral development
      • SCRUM
    • Not intended to be used without modification.
    • Although necessarily general, their usefulness and ability to achieve their primary purpose of developing software in the claimed circumstances should be assessed.
    • For example: If the method is claimed to be suited to developing life critical software applications, is it?
  • 7. Different approaches suit different contingencies
    • Large projects with dispersed teams without tight collaboration generally favours plan-based methods (Waterfall)
    • Exploratory systems with high risk generally favour a Spiral, iterative or incremental approach.
    • When business architect and software developer must jointly explore problem and develop the system, SCRUM or other agile method is favoured
  • 8. The primary purpose of a method is to achieve some objective
    • The method must contain the processes that can conceivably achieve the intended purpose e.g.
      • Develop, deploy, maintain software
      • Deploy, operate and manage IT services
    • Such primary processes have been the focus of many proposed situational method engineering methods
  • 9. Consider issues for performance management
    • A method should include the means to set and manage performance goals
    • Performance goals must be set
      • Usually the agreed requirements, schedule, budget and other project specific objectives
      • Over-arching organizational goals
      • Statutory requirements
    • Performance must be monitored and managed
    • Can be achieved through including specific process fragments
  • 10. Resources must be deployed, work coordinated
    • Performance goals do not determine how they will be met
    • Resources must be deployed to their best effect
    • Work must be coordinated through the processes
      • Static coordination through plans, work product flows
      • Dynamic through meetings, reviews and other exchanges
    • Usually achieved through project management processes
    • Resource deployment and work coordination processes must be adequate for their intended range of circumstances
    • Are they?
  • 11. Method enactment
    • Each project has considerable knowledge of the specific project contingencies, constraints and performance objectives
    • Each project must decide precisely how the resources will be deployed to perform the specific activities necessary to achieve the specific objectives in the specific circumstances
      • Shift activities among processes to compensate for personnel weaknesses or to take advantage of strengths
      • Any resource excess or shortage also influences how the method is enacted
  • 12. Existing methods have little to say about distributed software development
    • How are processes enacted to deal with the problems of distributed development
    • Existing software development methods have little to say about distributed development yet they are used on such projects
    • e.g. verification could be performed by the developer on exit from a process or by the acquirer on entry to the next process in the workflow
  • 13. Method assessment at enaction
    • Helps avoid any tendency to use a familiar method that is not fully appropriate to the particular situational constraints
    • Helps to facilitate a more objective judgement about the appropriateness of the method
  • 14. Method performance
    • An enacted method should achieve the best possible outcome in practice
    • Does it?
    • Need to gather evidence to prove or disprove that the method is the best possible
    • Two distinct issues
      • Right method, poorly performed
      • Acceptable performance, wrong method
    • Dominant assumption is that the method is right, but poorly performed
    • How would anyone detect a wrong method?
  • 15. Existing process assessment methods range from the formal to informal
    • Formal process assessment methods
      • SPICE, CMMI, Six Sigma, ISO 9001
      • Not specific to the circumstances
      • Rely on finding common problems
    • Informal assessments are done all the time
      • SCRUM has a retrospective
      • People learn and adjust what they do
      • BUT this doesn’t provide guidance on what to look for
  • 16. Measures of method performance
    • Some interest in measuring process effectiveness, efficiency, alignment, coordination, governance BUT no agreed measures
    • May be difficult to determine whether these attributes are being achieved
    • May be easier and more possible to determine the effect of their absence – defects, schedule overruns, other failures
    • Difficult to establish links between detected faults and their causes
  • 17. Advantages of this framework
    • Separates the concerns of each phase (design, enactment, performance)
    • Identifies what can reasonably be achieved and assessed at each phase
    • Provides guidance for development of assessment techniques and tools
  • 18. Future Research
    • Linkages between actual software development practices and their tailoring to SME body of knowledge
    • Development of metrics appropriate to each of the three lifecycle phases (design, enactment, performance)
  • 19. Conclusion
    • Proposed a framework to enable assessment of methods at different phases of their life cycle
    • The framework allows method engineers to assess the congruence of their developed, enacted or performed method according to the claimed range of known constraints and contingencies
    • The framework highlights some fruitful areas for research
    • The framework underlines the need to better link actual practices and their tailoring to method engineering