Sustainability Evaluation of Software Architectures: A Systematic Review

2,849 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,849
On SlideShare
0
From Embeds
0
Number of Embeds
1,686
Actions
Shares
0
Downloads
23
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Sustainability Evaluation of Software Architectures: A Systematic Review

  1. 1. Sustainability Evaluationof Software ArchitecturesHeiko KoziolekIndustrial Software SystemsABB Corporate Research, Germany1
  2. 2. 2 2
  3. 3. • Technology 2004 • Technology 2011 – GUI: Microsoft Visual Basic – GUI: Microsoft WPF – Middleware: Classic OPC – Middleware: OPC UA – OS: Windows XP – OS: Windows 7• Functionality (select.) • Functionality (select.) – Basic alarm management – Rich alarm management – No system integration – Multi-system integration – No online updates – Online updates• Code & Documentation • Code & Documentation – Defined APIs and layers – Some layer violations – Design rationale present – Some design rationale lost – Up-to-date documentation – Outdated documentation3
  4. 4. Measuring sustainability?4
  5. 5. State of practice:Techniques for architecture evaluation Experience 83% Prototyping 70% Scenarios 54% Checklists 40% Simulation 17%Questionaires 16% Metrics 15% Others 11% Math. models 5% [Source: AliBabar & Gorton 2009, survey with 88 architects]5
  6. 6. Research Questions• RQ1: Scenario-based methods & sustainability evaluation?• RQ2: What kind of architecture-level metrics?• RQ3: Implications for industry & research?6
  7. 7. Search• 14 conferences, 12 journals, 10 books• Included only methods/metrics from renowned venues• Excluded dormant or unvalidated methods/metrics7
  8. 8. Research Questions• RQ1: Scenario-based methods & sustainability evaluation?• RQ2: What kind of architecture-level metrics?• RQ3: Implications for industry & research?8
  9. 9. Scenario-based Methods• SAAM (5%) • ALMA (1%)• ATAM (5%) • ESAAMI Practitioners• ARID • SAAF applying the method• SAAMER • ASAAM [AliBabar &• PASA • SALUTA Gorton2009]• SBAR • SACAM• SAAMCS dormant • DOSAM• ISAAMCR • SBAR9
  10. 10. Sustainability Evaluation Criteria Change scenario Ripple effect Variation point specification analysis analysis Architecture Existing tools & Tradeoff analysis improvement artifacts
  11. 11. Sustainability Evaluation Criteria Change scenario Ripple effect Variation point specification analysis analysis Architecture Existing tools & Tradeoff analysis improvement artifacts
  12. 12. ATAM ALMAChange Sustainability evaluationscenario spec.Ripple effectanalysisVariation pointanalysisArchitectureimprovementTradeoffanalysisExistingartifacts/toolsReturn onInvestment12
  13. 13. ATAM ALMAChange Sustainability evaluation Quality attribute Top-down, bottom-up,scenario spec. scenario template no templateRipple effect Manual, based on Manual, based onanalysis experience experienceVariation point n/a n/aanalysisArchitecture Modifiability Tactics, n/aimprovement StylesTradeoff Utility tree with n/aanalysis stakeholder preferencesExisting No support No supportartifacts/toolsReturn on Cost: 30-70 person days Cost: unknownInvestment Benefit: not quantified Benefit: not quantified13
  14. 14. Findings RQ1: Scenarios• ATAM: more features• ALMA: more specific for sustainability• Manual ripple effect analysis• No reverse engineering• No tool integration• Return on Investment unknown• In practice: use best of both methods14
  15. 15. Research Questions• RQ1: Scenario-based methods & sustainability evaluation?• RQ2: What kind of architecture-level metrics?• RQ3: Implications for industry & research?15
  16. 16. Architectural metrics categories Similarity of Independent Encapsulation Purpose Compil. & Testing Acyclic [Sarkar2007] Size16 Dependencies
  17. 17. Architectural metrics categories Similarity of Independent Encapsulation Purpose Compil. & Testing Acyclic [Sarkar2007] Size17 Dependencies
  18. 18. Architecture MetricsSimilarity ofPurposeEncapsulationIndependentCompilation& TestingAcyclicDependenciesSize18
  19. 19. Architecture MetricsSimilarity of Concept Domination Metric, Concept Coherency Metic, API Function Usage Index, Concern Diffusion over Arch. Components,Purpose Concern Scope, Concern OverlapEncapsulation Ratio of Cohesive Interactions, Import Coupling, Export Coupling, Modularization Quality, Afferent Couplings, Efferent Couplings, Component-level Interlacing Between Concerns, Lack of Concern-based Cohesion, Module Interaction Index, Non-API Function Closedness Index, Implicit Dependency Index, Base class fragility index, Inheritance-based inter-module coupling, Not-programming-to-interfaces Index, Association-induced coupling, State Access Violation Index, Entropy of an architectural slicing, Architecture Slicing Cohesion, Decision VolatilityIndependent Cumulative Component Dependency, Average Cumulative Component Dependency, Normalized Cumulative Component Dependency,Compilation Coupling of a module, Intra-module coupling of a module, Cohesion of a& Testing module, Abstractness, Instability, Distance from the Main Sequence, Module Interaction Stability Index, Normalized Testability Dependency Metric, Plugin Pollution Index, Change impact, Independence LevelAcyclic Package Dependency Cycles, Cyclic Dependencies Index, Layer Organization Index, Excessive Structural ComplexityDependenciesSize Module Size Boundness Index, Module Size Uniformity Index19
  20. 20. Architecture MetricsSimilarity of Concept Domination Metric, Concept Coherency Metic, API Function Usage Index, Concern Diffusion over Arch. Components,Purpose Concern Scope, Concern OverlapEncapsulation Ratio of Cohesive Interactions, Import Coupling, Export Coupling, Modularization Quality, Afferent Couplings, Efferent Couplings, Component-level Interlacing Between Concerns, Lack of Concern-based Cohesion, Module Interaction Index, Non-API Function Closedness Index, Implicit Dependency Index, Base class fragility index, Inheritance-based inter-module coupling, Not-programming-to-interfaces Index, Association-induced coupling, State Access Violation Index, Entropy of an architectural slicing, Architecture Slicing Cohesion, Decision VolatilityIndependent Cumulative Component Dependency, Average Cumulative Component Dependency, Normalized Cumulative Component Dependency,Compilation Coupling of a module, Intra-module coupling of a module, Cohesion of a& Testing module, Abstractness, Instability, Distance from the Main Sequence, Module Interaction Stability Index, Normalized Testability Dependency Metric, Plugin Pollution Index, Change impact, Independence LevelAcyclic Package Dependency Cycles, Cyclic Dependencies Index, Layer Organization Index, Excessive Structural ComplexityDependenciesSize Module Size Boundness Index, Module Size Uniformity Index20
  21. 21. Example: Module Interact. Idx call call call ������ ������ ������������ ∈ ������1 … ������������ ������������������������ (������ ������ ) ������������������ ������ = ������������������������ (������) 1+1 = ≈ 0.67 API Non-API 3 ������ [Source: Sarkar2007 IEEE TSE]21
  22. 22. Example: Comparing Systems [Source: Sarkar2007 IEEE TSE]22
  23. 23. Findings RQ2• >40 architecture-level source code metrics in literature• Measures for architecture and modularization quality• Success factors for application: – Mix of metrics – Combination with class-level / process metrics• Limitations: – Empirical validation – Tool support23
  24. 24. Research Questions• RQ1: Scenario-based methods & sustainability evaluation?• RQ2: What kind of architecture-level metrics?• RQ3: Implications for industry & research?24
  25. 25. Findings RQ3: Implications• More empirical research• Method integration• Effective formal models• Codify experiences• Explore other approaches (simulation?)25
  26. 26. Conclusions• Sustainability Evaluation of Architectures – change scenarios: basic support for sustainability criteria – architectural code metrics: measure modularization, lack empirical evidence• Research opportunities – combined methods – empirical research26
  27. 27. 27
  28. 28. Existing surveys• Surveys on scenario- • Surveys / papers based method on code metrics – Dobrica 2002 – McCabe 1976 – Ali Babar 2004 – Halstead 1977 – Kazman 2005 – Chidamber 1994 – Breivold 2010 – Riaz 2009 – ... – ...28 28
  29. 29. ATAM ALMAGoals Sensitivity & Tradeoff Change impact analysis, Analysis maint. effort predictionArchitectural Process, data-flow, uses, Anydescription physical & module viewProcess Comprehensive LimitedsupportTool support n/a n/aKnowledge Recommended n/arepositoryValidation > 20 industrial case 7 industrial case studies studies documented (before 2004)29

×