Download to read offline
![49
Software Architecture Evaluation Methods
In the last years, several methods for evaluating software
architectures have been defined
» Scenario-based Software Architecture Evaluation Methods
– SAAM, ATAM, ALPSM and ALMA, SBAR, SALUTA, SAAMCS, ESAAMI,
etc.
» Mathematical Model-based Software Architecture Evaluation
– Software Architecture-based Reliability Analysis (e.g., Cheung [30])
– Software Architecture-based Performance Analysis (e.g., BIM, CM, ABI)
» Metrics-based Software Architecture Evaluation Methods
– Early evaluation: QuADAI
– Late evaluation: TLC, FA, MNS, SSC](https://image.slidesharecdn.com/sda-6-chaptersix-230109041313-66e7abc2/75/SDA-6-Chapter-Six-pdf-49-2048.jpg)
SDA - 6 -Chapter Six.pdf
- 1. 1 Software Design andArchitecture By Ashenafi C. AASTU, 2022
- 2. 2 Chapter Six Documenting andEvaluating Software Architecture
- 3. 3 Documenting Software Architecture Architecture serves as the blueprint for the system, and the project that develops it. » It defines the work assignments. » It is the primary carrier of quality attributes. » It is the best artifact for early analysis. » It is the key to post-deployment maintenance. » It is the conceptual glue that holds every phase of the project together for all of its many stakeholders. It must be described in sufficient detail, without ambiguity, and organized so that all stakeholders can quickly find the information they need. Documentation speaks for the architect, today and 20 years from today.
- 4. 4 Cont’d Even thebest architecture will be useless if the people who need it, » do not know what it is; » cannot understand it well enough to use, build, or modify it; » misunderstand it and apply it incorrectly. All of the effort, analysis, hard work, and insightful design on the part of the architecture team will have been wasted. Architecture documentation must » be sufficiently transparent and accessible to be quickly understood by new employees. » be sufficiently concrete to serve as a blueprint for construction. » have enough information to serve as a basis for analysis.
- 5. 5 Cont’d Architecture documentationis both prescriptive and descriptive. » For some audiences, it prescribes what should be true, placing constraints on decisions yet to be made. » For other audiences, it describes what is true, recounting decisions already made about a system’s design. Understanding stakeholder uses of architecture documentation is essential.
- 6. 6 Stakeholders Who Usethe Architectural Documentation Architect and requirements engineers who represent the customer(s) Architect and designers of constituent parts Implementers Testers and integrators Maintainers Designers of other systems with which this one must interoperate Quality attribute specialists Managers Product line managers Quality assurance team etc.
- 7. 7 Three Uses forArchitecture Documentation Education » Introducing people to the system – New members of the team – External analysts or evaluators – New architect Primary vehicle for communication among stakeholders » Especially architect to developers » Especially architect to future architect! Basis for system analysis and construction » Architecture tells implementers what to implement. » Each module has interfaces that must be provided and uses interfaces from other modules. » Documentation can serve as a receptacle for registering and communicating unresolved issues. » Architecture documentation serves as the basis for architecture evaluation.
- 8. 8 Notations Informal notations »Views are depicted (often graphically) using general-purpose diagramming and editing tools. » The semantics of the description are characterized in natural language. » They cannot be formally analyzed. Semiformal notations » Standardized notation that prescribes graphical elements and rules of construction. » Lacks a complete semantic treatment of the meaning of those elements. » Rudimentary analysis can be applied. » UML is a semiformal notation in this sense.
- 9. 9 Cont’d Formal notations »Views are described in a notation that has a precise (usually mathematically based) semantics. » Formal analysis of both syntax and semantics is possible. » Architecture description languages (ADLs). » Support automation through associated tools.
- 10. 10 Choosing Notations Tradeoffs »Typically, more formal notations take more time and effort to create and understand, but offer reduced ambiguity and more opportunities for analysis. » Conversely, more informal notations are easier to create, but they provide fewer guarantees. Different notations are better (or worse) for expressing different kinds of information. » UML class diagram will not help you reason about schedulability, nor will a sequence chart tell you very much about the system’s likelihood of being delivered on time. » Choose your notations and representation languages knowing the important issues you need to capture and reason about.
- 11. 11 “Views and Beyond”Approach to Documentation Views let us divide a software architecture into a number of interesting and manageable representations of the system. Views give us our first principle of architecture documentation: Document the relevant views, and then add information that applies to more than one view.
- 12. 12 Which Views areRelevant? Kruchten’s 4+1 views » Logical view: supports behavioral requirements. Key abstractions, which are objects or object classes » Process view: addresses concurrency and distribution. Maps threads to objects. » Development view: organization of software modules, libraries, subsystems, units of development. » Physical view: maps other elements onto processing and communication nodes. » “Plus one” view: Maps the other views onto important use cases to show how they work.
- 13. 13 Cont’d Siemens Four-Views(Hofmeister, Nord, Soni, Applied Software Architecture, 2000): » Conceptual view » Module interconnection view » Execution view » Code view Herzum & Sims (Business Component Factory, 1999): » Technical architecture » Application architecture » Project management architecture » Functional architecture
- 14. 14 Cont’d Software CostReduction method (Parnas, et al., 1980s) » Module view: shows modules as units of encapsulation; used to isolate changes and achieve modifiability. » Process view: shows processes and how they synchronize and communicate at run-time; used to achieve performance. » Uses view: shows programs and how they depend on each other; used to achieve incremental development and the ability to quickly field subsets.
- 15. 15 Cont’d Which viewsare relevant? It depends on: » who the stakeholders are » how they will use the documentation. Three primary uses for architecture documentation: » Education -- introducing people to the project » Communication -- among stakeholders » Analysis -- assuring quality attributes
- 16. 16 What views areavailable? Plenty! Too many! Already, we’ve seen 15 different views, and many more are available, based on examining the literature. An architect needs a way to choose the useful ones. One thing that would help is to organize the views into broad categories.
- 17. 17 Cont’d An architectmust consider the system in three ways: » How is it structured as a set of code units? – e.g Module views (module viewtype) » How is it structured as a set of elements that have run-time behavior and interactions? – e.g Component-and-connector views (C&C viewtype) » How does it relate to non-software structures in its environment? – e.g Allocation views (allocation viewtype)
- 18. 18 Cont’d How manyviews do we need in our documentation package? » Each view comes with a cost and benefit. » Planning a view set requires understanding the needs of the stakeholders, and the resources available.
- 19. 19 Choosing the Views You can determine which views are required, when to create them, and how much detail to include if you know the following: » What people, and with what skills, are available » Which standards you have to comply with » What budget is on hand » What the schedule is » What the information needs of the important stakeholders are » What the driving quality attribute requirements are » What the approximate size of the system is?
- 20. 20 Cont’d At aminimum, expect to have » at least one module view, » at least one C&C view, and » for larger systems, at least one allocation view in your architecture document.
- 21. 21 Method for Choosingthe Views Step 1. Build a stakeholder/view table. » Rows: List the stakeholders for your project’s software architecture documentation » Columns: Enumerate the views that apply to your system. – Use the structures discussed in Chapter 1, the views discussed in this chapter, and the views that your design work in ADD has suggested as a starting list of candidates. – Include the views or view sketches you have as a result of your design work so far. » Some views (such as decomposition, uses, and work assignment) apply to every system, while others (various C&C views, the layered view) only apply to some systems. » Fill in each cell to describe how much information the stakeholder requires from the view: none, overview only, moderate detail, or high detail.
- 22. 22 Cont’d Step 2.Combine views to reduce their number » Look for marginal views in the table; those that require only an overview, or that serve very few stakeholders. » Combine each marginal view with another view that has a stronger constituency. » These views often combine naturally: – Various C&C views - Because C&C views all show runtime relations among components and connectors of various types, they tend to combine well. – Deployment view with either SOA or communicating-processes views - An SOA view shows services, and a communicating-processes view shows processes. In both cases, these are components that are deployed onto processors. – Decomposition view and any of work assignment, implementation, uses, or layered views - The decomposed modules form the units of work, development, and uses. In addition, these modules populate layers.
- 23. 23 Cont’d Step 3.Prioritize and stage. » The decomposition view (one of the module views) is a particularly helpful view to release early. – High-level decompositions are often easy to design – The project manager can start to staff development teams, put training in place, determine which parts to outsource, and start producing budgets and schedules. » You don’t have to satisfy all the information needs of all the stakeholders to the fullest extent. – Providing 80 percent of the information goes a long way, and this might be “good enough” so that the stakeholders can do their job. – Check with the stakeholder if a subset of information would be sufficient. » You don’t have to complete one view before starting another. – People can make progress with overview-level information – A breadth-first approach is often the best.
- 24. 24 Building the DocumentationPackage Documentation package consists of » Views » Documentation beyond views
- 25. 25 Documenting a View Section 1: The Primary Presentation. » The primary presentation shows the elements and relations of the view. » The primary presentation should contain the information you wish to convey about the system—in the vocabulary of that view. » The primary presentation is most often graphical. – It might be a diagram you’ve drawn in an informal notation using a simple drawing tool, or it might be a diagram in a semiformal or formal notation imported from a design or modeling tool that you’re using. – If your primary presentation is graphical, make sure to include a key that explains the notation. – Lack of a key is the most common mistake that we see in documentation in practice. » Occasionally the primary presentation will be textual, such as a table or a list. – If that text is presented according to certain stylistic rules, these rules should be stated or incorporated by reference, as the analog to the graphical notation key.
- 26. 26 Cont’d Section 2:The Element Catalog. » The element catalog details at least those elements depicted in the primary presentation. – For instance, if a diagram shows elements A, B, and C, then the element catalog needs to explain what A, B, and C are. – If elements or relations relevant to this view were omitted from the primary presentation, they should be introduced and explained in the catalog. » Parts of the catalog: – Elements and their properties- This section names each element in the view and lists the properties of that element. Each view introduced in Chapter 1 listed a set of suggested properties associated with that view. – Relations and their properties- Each view has specific relation types that it depicts among the elements in that view. – Element interfaces- This section documents element interfaces. – Element behavior- This section documents element behavior that is not obvious from the primary presentation.
- 27. 27 Cont’d Section 3:Context Diagram. » A context diagram shows how the system or portion of the system depicted in this view relates to its environment. » The purpose of a context diagram is to depict the scope of a view. » Entities in the environment may be humans, other computer systems, or physical objects, such as sensors or controlled devices. Section 4: Variability Guide. » A variability guide shows how to exercise any variation points that are a part of the architecture shown in this view. Section 5: Rationale. » Rationale explains why the design reflected in the view came to be. » The goal of this section is to explain why the design is as it is and to provide a convincing argument that it is sound. » The choice of a pattern in this view should be justified here by describing the architectural problem that the chosen pattern solves and the rationale for choosing it over another.
- 28.
- 29. 29 Documenting Information BeyondViews Document control information. » List the issuing organization, the current version number, date of issue and status, a change history, and the procedure for submitting change requests to the document. » Usually captured in the front matter.
- 30. 30 Cont’d Section 1:Documentation Roadmap. The documentation map tells the reader what information is in the documentation and where to find it. » Scope and summary - Explain the purpose of the document and briefly summarize what is covered. » How the documentation is organized - For each section in the documentation, give a short synopsis of the information that can be found there. » View overview - Describes the views that the architect has included in the package. For each view:: – The name of the view and what pattern it instantiates, if any. – A description of the view’s element types, relation types, and property types. – A description of language, modeling techniques, or analytical methods used in constructing the view. » How stakeholders can use the documentation: – This section shows how various stakeholders might use the documentation to help address their concerns. – Include short scenarios, such as “A maintainer wishes to know the units of software that are likely to be changed by a proposed modification.” – To be compliant with ISO/IEC 42010-2007, you must consider the concerns of at least users, acquirers, developers, and maintainers.
- 31. 31 Cont’d Section 2:How a View is Documented. » Explain the standard organization you’re using to document views—either the one described in this chapter or one of your own. Section 3: System Overview. » Short prose description of the system’s function, its users, and any important background or constraints. » Provides your readers with a consistent mental model of the system and its purpose. » This might be a pointer to your project’s concept-of-operations document for the system.
- 32. 32 Cont’d Section 4:Mapping Between Views. » Helping a reader understand the associations between views will help that reader gain a powerful insight into how the architecture works as a unified conceptual whole. » The associations between elements across views in an architecture are, in general, many-to-many. » View-to-view associations can be captured as tables. – The table should name the correspondence between the elements across the two views. – Examples “is implemented by” for mapping from a component-and-connector view to a module view “implements” for mapping from a module view to a component-and-connector view “included in” for mapping from a decomposition view to a layered view
- 33. 33 Cont’d Section 5:Rationale. » Documents the architectural decisions that apply to more than one view. – Documentation of background or organizational constraints or major requirements that led to decisions of system-wide import. – Decisions about which fundamental architecture patterns are used. Section 6: Directory. » Set of reference material that helps readers find more information quickly. – Index of terms – Glossary – Acronym list.
- 34. 34 Documenting Behavior Behaviordocumentation complements each views by describing how architecture elements in that view interact with each other. Behavior documentation enables reasoning about » a system’s potential to deadlock » a system’s ability to complete a task in the desired amount of time » maximum memory consumption » and more Behavior has its own section in our view template’s element catalog.
- 35. 35 Notations for DocumentingBehavior Trace-oriented languages » Traces are sequences of activities or interactions that describe the system’s response to a specific stimulus when the system is in a specific state. » A trace describes a particular sequence of activities or interactions between structural elements of the system. » Examples – use cases – sequence diagrams – communication diagrams – activity diagrams – message sequence charts – timing diagrams – Business Process Execution Language
- 36. 36 Cont’d Comprehensive languages »Comprehensive models show the complete behavior of structural elements. » Given this type of documentation, it is possible to infer all possible paths from initial state to final state. » The state machine formalism represents the behavior of architecture elements because each state is an abstraction of all possible histories that could lead to that state. » State machine languages allow you to complement a structural description of the elements of the system with constraints on interactions and timed reactions to both internal and environmental stimuli.
- 37. 37 Documenting Quality Attributes Where do quality attributes show up in the documentation? There are five major ways: » Rationale that explains the choice of design approach should include a discussion about the quality attribute requirements and tradeoffs. » Architectural elements providing a service often have quality attribute bounds assigned to them, defined in the interface documentation for the elements, or recorded as properties that the elements exhibit. » Quality attributes often impart a “language” of things that you would look for. Someone fluent in the “language” of a quality attribute can search for the kinds of architectural elements put in place to satisfy that quality attribute requirement. » Architecture documentation often contains a mapping to requirements that shows how requirements (including quality attribute requirements) are satisfied. » Every quality attribute requirement will have a constituency of stakeholders who want to know that it is going to be satisfied. For these stakeholders, the roadmap tells the stakeholder where in the document to find it.
- 38. 38 Documenting Architectures ThatChange Faster Than You Can Document Them An architecture that changes at runtime, or as a result of a high-frequency release-and-deploy cycle, change much faster than the documentation cycle. Nobody will wait until a new architecture document is produced, reviewed, and released. In this case: » Document what is true about all versions of your system - Record those invariants as you would for any architecture. This may make your documented architecture more a description of constraints or guidelines that any compliant version of the system must follow. » Document the ways the architecture is allowed to change - This will usually mean adding new components and replacing components with new implementations. The place to do this is called the variability guide.
- 39. 39 Architecture Documentation Templates Rational Unified Process / Kruchten 4+1 Siemens Four Views C4ISR architecture framework IEEE Standard 1471 for architecture documentation Hewlett Packard’s architecture documentation framework Data flow and control flow views Enterprise architecture methods Zachman’s Information Architecture RM-ODP Etc.
- 40. 40 Seven Rules forSound Documentation (By P.Clements, et al.) Rule 1: Write Documentation from the Reader’s Point of View Rule 2: Avoid Unnecessary Repetition Rule 3: Avoid Ambiguity » Rule 3a: Explain Your Notation Rule 4: Use a Standard Organization (template) Rule 5: Record Rationale Rule 6: Keep Documentation Current but Not Too Current Rule 7: Review Documentation for Fitness of Purpose
- 41.
- 42. 42 Evaluating Software Architecture Software architecture evaluation is a method which determines the properties, strengths and weaknesses of software architecture. It provides assurance to developers that their chosen architecture will meet both functional and non-functional quality requirements. It is to verify if the architectural decisions are appropriate in early stages and predict if a system will have the required quality attributes. The goal is to determine the degree in which a software architecture or an architectural style satisfy the quality requirements.
- 43. 43 When Can anArchitecture Be Evaluated? As early as possible, even during actual development of SA. » The earlier problems are found, the earlier and cheaper they can be fixed. Certainly after the architecture is completed, you should validate it before development. Later to ensure consistency between design and implementation especially for legacy systems. Before acquiring a new system. The real answer is early and often!
- 44. 44 Who's Involved? Evaluationteam - these are the people who will conduct the evaluation and perform the analysis. Stakeholders - stakeholders are people who have a vested interest in the architecture and the system. Three Forms of Evaluation: » Evaluation by the designer within the design process. » Evaluation by peers within the design process. » Analysis by outsiders once the architecture has been designed.
- 45. 45 Contextual Factors forEvaluation What artifacts are available? » To perform an architectural evaluation, there must be an artifact that describes the architecture. Who sees the results? » Some evaluations are performed with the full knowledge and participation of all of the stakeholders. Others are performed more privately. Who performs the evaluation? » Evaluations can be carried out by an individual or a team. Which stakeholders will participate? » The evaluation process should provide a method to elicit the goals and concerns that the important stakeholders have regarding the system. Identifying the individuals who are needed and assuring their participation in the evaluation is critical. What are the business goals? » The evaluation should answer whether the system will satisfy the business goals.
- 46. 46 Why To Evaluatean Architecture? Forces Preparation for Review » Documentation/specifications must be provided, hence they must exist or be created. » Some evaluations use standard questions, and the architect can prepare ahead to ensure that the architecture scores well. » Evaluations make the criteria for evaluation explicit by prioritizing requirements or quality goals. Early Detection of Problems » The problems that can be found by an architecture evaluation include: – unreasonable requirements. – performance problems. – problems associated with potential future modifications. » The earlier in the life cycle that problems are found, the easier it is to fix them.
- 47. 47 Cont’d Validation ofRequirements » Evaluations put stakeholders in the same room with each other, often for the first time. – uncovers conflicts and tradeoffs. – provides a forum for negotiated resolution of problems. » It often results in the generation of new requirements or the clarification of existing requirements. Improved Architectures » Development organizations anticipate types of questions raised at evaluations and – design architectures with questions in mind. – prepare documentation of the type needed at evaluation. – give explicit consideration to qualities to be evaluated.
- 48. 48 Planned or UnplannedEvaluations Planned » Normal part of software life cycle. » Built into the project’s work plans, budget and schedule. » Scheduled right after completion of SA. » Planned evaluations are “proactive” and “team-building”. Unplanned » As a result of some serious problem. » “mid-course correction”. » Challenge to technical authority of Team. » Unplanned evaluations are “reactive” and “tension-filled”.
- 49. 49 Software Architecture EvaluationMethods In the last years, several methods for evaluating software architectures have been defined » Scenario-based Software Architecture Evaluation Methods – SAAM, ATAM, ALPSM and ALMA, SBAR, SALUTA, SAAMCS, ESAAMI, etc. » Mathematical Model-based Software Architecture Evaluation – Software Architecture-based Reliability Analysis (e.g., Cheung [30]) – Software Architecture-based Performance Analysis (e.g., BIM, CM, ABI) » Metrics-based Software Architecture Evaluation Methods – Early evaluation: QuADAI – Late evaluation: TLC, FA, MNS, SSC
- 50. 50 Cont’d After extensiveresearch, the Carnegie Mellon Software Engineering Institute (SEI) has come up with three architecture evaluation methods: » Architecture Tradeoff Analysis Method (ATAM) » Software Architecture Analysis Method (SAAM) » Architecture Review For Intermediate Design (ARID) In many expert opinions, ATAM is the best method of the three.
- 51. 51 The Architecture TradeoffAnalysis (ATAM) Method The Architecture Tradeoff Analysis Method (ATAM) has been used for over a decade to evaluate software architectures in domains ranging from automotive to financial to defense. The ATAM is designed so that evaluators need not be familiar with the architecture or its business goals, the system need not yet be constructed, and there may be a large number of stakeholders.
- 52. 52 Participants in theATAM The evaluation team. » External to the project whose architecture is being evaluated. » Three to five people; a single person may adopt several roles in an ATAM. » They need to be recognized as competent, unbiased outsiders. Project decision makers. » These people are empowered to speak for the development project or have the authority to mandate changes to it. » They usually include the project manager, and if there is an identifiable customer who is footing the bill for the development, he or she may be present (or represented) as well. » The architect is always included – the architect must willingly participate. Architecture stakeholders. » Stakeholders have a vested interest in the architecture performing as advertised. » Stakeholders include developers, testers, integrators, maintainers, performance engineers, users, builders of systems interacting with the one under consideration, and, possibly, others. » Their job is to articulate the specific quality attribute goals that the architecture should meet. » Expect to enlist 12 to 15 stakeholders for the evaluation of a large enterprise-critical architecture.
- 53. 53 ATAM Evaluation TeamRoles Role Responsibilities Team leader Sets up the evaluation; coordinates with client, making sure client’s needs are met; establishes evaluation contract; forms evaluation team; sees that final report is produced and delivered (although the writing may be delegated) Evaluation leader Runs evaluation; facilitates elicitation of scenarios; administers scenario selection/prioritization process; facilitates evaluation of scenarios against architecture; facilitates on-site analysis Scenario scribe Writes scenarios on flipchart or whiteboard during scenario elicitation; captures agreed-on wording of each scenario, halting discussion until exact wording is captured Proceeding s scribe Captures proceedings in electronic form on laptop or workstation: raw scenarios, issue(s) that motivate each scenario (often lost in the wording of the scenario itself), and resolution of each scenario when applied to architecture(s); also generates a printed list of adopted scenarios for handout to all participants Questioner Raises issues of architectural interest, usually related to the quality attributes in which he or she has expertise
- 54. 54 Outputs of theATAM 1. A concise presentation of the architecture. The architecture is presented in one hour 2. Articulation of the business goals. Frequently, the business goals presented in the ATAM are being seen by some of the assembled participants for the first time and these are captured in the outputs. 3. Prioritized quality attribute requirements expressed as quality attribute scenarios. These quality attribute scenarios take the form described in Chapter 4. 4. A set of risks and nonrisks. » A risk is defined as an architectural decision that may lead to undesirable consequences in light of quality attribute requirements. » A nonrisk is an architectural decision that, upon analysis, is deemed safe. » The identified risks form the basis for an architectural risk mitigation plan.
- 55. 55 Outputs of theATAM 5. A set of risk themes. When the analysis is complete, the evaluation team examines the full set of discovered risks to look for overarching themes that identify systemic weaknesses in the architecture or even in the architecture process and team. If left untreated, these risk themes will threaten the project’s business goals. 6. Mapping of architectural decisions to quality requirements. For each quality attribute scenario examined during an ATAM, those architectural decisions that help to achieve it are determined and captured. 7. A set of identified sensitivity and tradeoff points. These are architectural decisions that have a marked effect on one or more quality attributes.
- 56. 56 Intangible Outputs Thereare also intangible results of an ATAM-based evaluation. These include » a sense of community on the part of the stakeholders » open communication channels between the architect and the stakeholders » a better overall understanding on the part of all participants of the architecture and its strengths and weaknesses. While these results are hard to measure, they are no less important than the others and often are the longest-lasting.
- 57. 57 Phases of theATAM Phase Activity Participants Typical duration 0 Partnership and preparation: Logistics, planning, stakeholder recruitment, team formation Evaluation team leadership and key project decision- makers Proceeds informally as required, perhaps over a few weeks 1 Evaluation: Steps 1-6 Evaluation team and project decision- makers 1-2 days followed by a hiatus of 2-3 weeks 2 Evaluation: Steps 7-9 Evaluation team, project decision makers, stakeholders 2 days 3 Follow-up: Report generation and delivery, process improvement Evaluation team and evaluation client 1 week
- 58. 58 Step 1: Presentthe ATAM The evaluation leader presents the ATAM to the assembled project representatives. This time is used to explain the process that everyone will be following, to answer questions, and to set the context and expectations for the remainder of the activities. Using a standard presentation, the leader describes the ATAM steps in brief and the outputs of the evaluation.
- 59. 59 Step 2: PresentBusiness Drivers Everyone involved in the evaluation needs to understand the context for the system and the primary business drivers motivating its development. In this step, a project decision maker (ideally the project manager or the system’s customer) presents a system overview from a business perspective. The presentation should describe the following: » The system’s most important functions » Any relevant technical, managerial, economic, or political constraints » The business goals and context as they relate to the project » The major stakeholders » The architectural drivers (that is, the architecturally significant requirements)
- 60. 60 Step 3: Presentthe Architecture The lead architect (or architecture team) makes a presentation describing the architecture. The architect covers technical constraints such as operating system, hardware, or middleware prescribed for use, and other systems with which the system must interact. The architect describes the architectural approaches (or patterns, or tactics, if the architect is fluent in that vocabulary) used to meet the requirements. The architect’s presentation should convey the essence of the architecture and not stray into ancillary areas or delve too deeply into the details of just a few aspects. The architect should present the views that he or she found most important during the creation of the architecture and the views that help to reason about the most important quality attribute concerns of the system.
- 61. 61 Step 4: IdentifyArchitectural Approaches The ATAM focuses on analyzing an architecture by understanding its architectural approaches, especially patterns and tactics. By now, the evaluation team will have a good idea of what patterns and tactics the architect used in designing the system. » They will have studied the architecture documentation » They will have heard the architect’s presentation in step 3. » The team should also be adept at spotting approaches not mentioned explicitly The evaluation team simply catalogs the patterns and tactics that have been identified. The list is publicly captured and will serve as the basis for later analysis.
- 62. 62 Step 5: GenerateUtility Tree The quality attribute goals are articulated in detail via a quality attribute utility tree. Utility trees serve to make the requirements concrete by defining precisely the relevant quality attribute requirements that the architects were working to provide. The important quality attribute goals for the architecture under consideration were named in step 2. In this step, the evaluation team works with the project decision makers to identify, prioritize, and refine the system’s most important quality attribute goals. These are expressed as scenarios, which populate the leaves of the utility tree. The scenarios are assigned a rank of importance (High, Medium, Low).
- 63. 63 Step 6: AnalyzeArchitectural Approaches The evaluation team examines the highest-ranked scenarios one at a time; the architect is asked to explain how the architecture supports each one. Evaluation team members—especially the questioners—probe for the architectural approaches that the architect used to carry out the scenario. Along the way, the evaluation team documents the relevant architectural decisions and identifies and catalogs their risks, nonrisks, sensitivity points, and tradeoffs. Examples: » Risk: The frequency of heartbeats affects the time in which the system can detect a failed component. Some assignments will result in unacceptable values of this response. » Sensitivity point: The number of simultaneous database clients will affect the number of transactions that a database can process per second. » Tradeoff: The heartbeat frequency determines the time for detecting a fault. Higher frequency leads to better availability but consumes more processing time and communication bandwidth (potentially reducing performance). These, in turn, may catalyze a deeper analysis. The analysis is not meant to be comprehensive. The key is to elicit sufficient architectural information to establish a link between the architectural decisions made and the quality attribute requirements that need to be satisfied.
- 64.
- 65. 65 Step 7: Brainstormand Prioritize Scenarios The stakeholders brainstorm scenarios that are operationally meaningful with respect to the stakeholders’ individual roles. » A maintainer will likely propose a modifiability scenario » A user will probably come up with a scenario that expresses useful functionality or ease of operation » A quality assurance person will propose a scenario about testing the system or being able to replicate the state of the system leading up to a fault. The purpose of scenario brainstorming is to take the pulse of the larger stakeholder community: to understand what system success means for them. Once the scenarios have been collected, they are prioritized by voting. The list of prioritized scenarios is compared with those from the utility tree exercise. » If they agree, it indicates good alignment between what the architect had in mind and what the stakeholders actually wanted. » If additional driving scenarios are discovered—and they usually are—this may itself be a risk, if the discrepancy is large. This would indicate that there was some disagreement in the system’s important goals between the stakeholders and the architect.
- 66. 66 Step 8: AnalyzeArchitectural Approaches In this step the evaluation team performs the same activities as in step 6, using the highest-ranked, newly generated scenarios. The evaluation team guides the architect in the process of carrying out the highest ranked new scenarios. The architect explains how relevant architectural decisions contribute to realizing each one. This step might cover the top 5-10 scenarios, as time permits.
- 67. 67 Step 9: PresentResults The evaluation team confers privately to group risks into risk themes, based on some common underlying concern or systemic deficiency. » For example, a group of risks about inadequate or out-of-date documentation might be grouped into a risk theme stating that documentation is given insufficient consideration. » A group of risks about the system’s inability to function in the face of various hardware and/or software failures might lead to a risk theme about insufficient attention to backup capability or providing high availability. For each risk theme, the evaluation team identifies which of the business drivers listed in step 2 are affected. » This elevates the risks that were uncovered to the attention of management, who cares about the business drivers.
- 68. 68 Step 9: PresentResults The collected information from the evaluation is summarized and presented to stakeholders. The following outputs are presented: » The architectural approaches documented » The set of scenarios and their prioritization from the brainstorming » The utility tree » The risks discovered » The nonrisks documented » The sensitivity points and tradeoff points found » Risk themes and the business drivers threatened by each one
- 69. 69 Lightweight Architectural Evaluation An ATAM is a substantial undertaking. » It requires some 20 to 30 person-days of effort from an evaluation team, plus even more for the architect and stakeholders. » Investing this amount of time makes sense on a large and costly project, where the risks of making a major mistake in the architecture are unacceptable. We have developed a Lightweight Architecture Evaluation method, based on the ATAM, for smaller, less risky projects. » May take place in a single day, or even a half-day meeting. » May be carried out entirely by members internal to the organization. » Of course this lower level of scrutiny and objectivity may not probe the architecture as deeply. Because the participants are all internal to the organization and fewer in number than for the ATAM, giving everyone their say and achieving a shared understanding takes much less time. The steps and phases of a Lightweight Architecture Evaluation can be carried out more quickly.
- 70. 70 Typical Agenda: 4-6Hours Step Time Notes 1. Present the ATAM 0 hours Participants already familiar with process. 2. Present business drivers 0.25 hours The participants are expected to understand the system and its business goals and their priorities. A brief review ensures that these are fresh in everyone’s mind and that there are no surprises. 3. Present architecture 0.5 hours All participants are expected to be familiar with the system. A brief overview of the architecture, using at least module and C&C views, is presented. 1-2 scenarios are traced through these views. 4. Identify architectural approaches 0.25 hours The architecture approaches for specific quality attribute concerns are identified by the architect. This may be done as a portion of step 3. 5. Generate QA utility tree 0.5- 1.5 hours Scenarios might exist: part of previous evaluations, part of design, part of requirements elicitation. Put these in a tree. Or, a utility tree may already exist. 6. Analyze architectural approaches 2-3 hours This step—mapping the highly ranked scenarios onto the architecture—consumes the bulk of the time and can be expanded or contracted as needed. 7. Brainstorm scenarios 0 hours This step can be omitted as the assembled (internal) stakeholders are expected to contribute scenarios expressing their concerns in step 5. 8. Analyze architectural approaches 0 hours This step is also omitted, since all analysis is done in step 6. 9. Present results 0.5 hours At the end of an evaluation, the team reviews the existing and newly discovered risks, nonrisks, sensitivities, and tradeoffs and discusses whether any new risk themes have arisen.