Me2011 Method Assessment by Henderson-Sellers


Published on

A Method Assessment Framework

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Me2011 Method Assessment by Henderson-Sellers

  1. 1. © T.McBride and B.Henderson-Sellers, 2011 A Method Assessment Framework Tom McBride Brian Henderson-Sellers School of Software University of Technology, Sydney
  2. 2. © T.McBride and B.Henderson-Sellers, 2011 There is a need for a method assessment framework • There has been little discussion of how the quality of a method might be assessed, or when • A growing number of communities are developing methods without the aid of method engineers • We believe that such initiatives can be supported by an assessment framework that separates the concerns of different phases in the life cycle of a method (design, enactment, performance) ME'11 2
  3. 3. © T.McBride and B.Henderson-Sellers, 2011 A method can be assessed at design, enactment and performance • Method design – When a method is created in order to achieve a specific purpose but in a range of circumstances e.g. a method suited to developing life critical software using medium to large teams under tight schedule and budget constraints • Method enactment – When a method is tailored to the specific circumstances of a project – Project planning once the project requirements, schedule, team, budget, resources, environment have been decided. • Method performance – When the method is being used ME'11 3
  4. 4. © T.McBride and B.Henderson-Sellers, 2011 Process design and enactment Process Model or Method Metamodel Process Tailored process Performed process As enacted by real people on a specific project As documented As standardized Method Design Method Enactment Process Model or Method Metamodel Process Tailored process Performed process As enacted by real people on a specific project As documented As standardized Process Model or Method Metamodel ProcessProcess Tailored process Performed process As enacted by real people on a specific project As documented As standardized Method Design Method Enactment
  5. 5. © T.McBride and B.Henderson-Sellers, 2011 What is method quality • Is the method suitable for its intended purpose in the circumstances for which is intended? • Is the method complete? – Work products missing, created but not used, used but not created • Is the method actually needed for the particular application ME'11 5
  6. 6. © T.McBride and B.Henderson-Sellers, 2011 Method design • Software development methods are created to address a range of circumstances – Waterfall development – Spiral development – SCRUM • Not intended to be used without modification. • Although necessarily general, their usefulness and ability to achieve their primary purpose of developing software in the claimed circumstances should be assessed. • For example: If the method is claimed to be suited to developing life critical software applications, is it? ME'11 6
  7. 7. © T.McBride and B.Henderson-Sellers, 2011 Different approaches suit different contingencies • Large projects with dispersed teams without tight collaboration generally favours plan-based methods (Waterfall) • Exploratory systems with high risk generally favour a Spiral, iterative or incremental approach. • When business architect and software developer must jointly explore problem and develop the system, SCRUM or other agile method is favoured ME'11 7
  8. 8. © T.McBride and B.Henderson-Sellers, 2011 The primary purpose of a method is to achieve some objective • The method must contain the processes that can conceivably achieve the intended purpose e.g. – Develop, deploy, maintain software – Deploy, operate and manage IT services • Such primary processes have been the focus of many proposed situational method engineering methods ME'11 8
  9. 9. © T.McBride and B.Henderson-Sellers, 2011 Consider issues for performance management • A method should include the means to set and manage performance goals • Performance goals must be set – Usually the agreed requirements, schedule, budget and other project specific objectives – Over-arching organizational goals – Statutory requirements • Performance must be monitored and managed • Can be achieved through including specific process fragments ME'11 9
  10. 10. © T.McBride and B.Henderson-Sellers, 2011 Resources must be deployed, work coordinated • Performance goals do not determine how they will be met • Resources must be deployed to their best effect • Work must be coordinated through the processes – Static coordination through plans, work product flows – Dynamic through meetings, reviews and other exchanges • Usually achieved through project management processes • Resource deployment and work coordination processes must be adequate for their intended range of circumstances • Are they? ME'11 10
  11. 11. © T.McBride and B.Henderson-Sellers, 2011 Method enactment • Each project has considerable knowledge of the specific project contingencies, constraints and performance objectives • Each project must decide precisely how the resources will be deployed to perform the specific activities necessary to achieve the specific objectives in the specific circumstances – Shift activities among processes to compensate for personnel weaknesses or to take advantage of strengths – Any resource excess or shortage also influences how the method is enacted ME'11 11
  12. 12. © T.McBride and B.Henderson-Sellers, 2011 Existing methods have little to say about distributed software development • How are processes enacted to deal with the problems of distributed development • Existing software development methods have little to say about distributed development yet they are used on such projects e.g. verification could be performed by the developer on exit from a process or by the acquirer on entry to the next process in the workflow ME'11 12
  13. 13. © T.McBride and B.Henderson-Sellers, 2011 Method assessment at enaction Helps avoid any tendency to use a familiar method that is not fully appropriate to the particular situational constraints Helps to facilitate a more objective judgement about the appropriateness of the method
  14. 14. © T.McBride and B.Henderson-Sellers, 2011 Method performance • An enacted method should achieve the best possible outcome in practice • Does it? • Need to gather evidence to prove or disprove that the method is the best possible • Two distinct issues – Right method, poorly performed – Acceptable performance, wrong method • Dominant assumption is that the method is right, but poorly performed • How would anyone detect a wrong method? ME'11 14
  15. 15. © T.McBride and B.Henderson-Sellers, 2011 Existing process assessment methods range from the formal to informal • Formal process assessment methods – SPICE, CMMI, Six Sigma, ISO 9001 – Not specific to the circumstances – Rely on finding common problems • Informal assessments are done all the time – SCRUM has a retrospective – People learn and adjust what they do – BUT this doesn’t provide guidance on what to look for ME'11 15
  16. 16. © T.McBride and B.Henderson-Sellers, 2011 Measures of method performance • Some interest in measuring process effectiveness, efficiency, alignment, coordination, governance BUT no agreed measures • May be difficult to determine whether these attributes are being achieved • May be easier and more possible to determine the effect of their absence – defects, schedule overruns, other failures • Difficult to establish links between detected faults and their causes ME'11 16
  17. 17. © T.McBride and B.Henderson-Sellers, 2011 Advantages of this framework • Separates the concerns of each phase (design, enactment, performance) • Identifies what can reasonably be achieved and assessed at each phase • Provides guidance for development of assessment techniques and tools ME'11 17
  18. 18. © T.McBride and B.Henderson-Sellers, 2011 Future Research • Linkages between actual software development practices and their tailoring to SME body of knowledge • Development of metrics appropriate to each of the three lifecycle phases (design, enactment, performance)
  19. 19. © T.McBride and B.Henderson-Sellers, 2011 Conclusion • Proposed a framework to enable assessment of methods at different phases of their life cycle • The framework allows method engineers to assess the congruence of their developed, enacted or performed method according to the claimed range of known constraints and contingencies • The framework highlights some fruitful areas for research • The framework underlines the need to better link actual practices and their tailoring to method engineering ME'11 19