Contextually-Driven System Architecture Reviews

408 views

Published on

When the World Trade Center collapsed, the telephone switching systems in the basement correctly diagnosed which lines were still working and continued to connect calls for several days using backup power. One factor contributing to this remarkable product reliability was the AT&T/Bell Labs practice of early systems architecture reviews. Michael Dedolph shares an architecture review method based on the Bell Labs Systems Architecture Review Board (SARB) process and discusses how that method was institutionalized and managed. The review method is a team process that uses a problem statement developed by the project as the basis for the review. The method is "low tech" and portable. SARB-style architecture reviews can be easily and flexibly tailored to your context. The flexibility of the method makes it suitable for many kinds of systems and problem domains. Take away an appreciation for the method and see if it might be useful in your organization.

Published in: Technology, News & Politics
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
408
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
10
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Contextually-Driven System Architecture Reviews

  1. 1.   BT2 Concurrent Session  11/14/2013 10:15 AM            "Contextually-Driven System Architecture Reviews "       Presented by: F. Michael Dedolph Levi Deal Consulting                   Brought to you by:        340 Corporate Way, Suite 300, Orange Park, FL 32073  888‐268‐8770 ∙ 904‐278‐0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
  2. 2. Michael Dedolph Levi Deal Consulting Michael Dedolph is self-employed as a consultant and innkeeper. For the past seven years, Michael led quality process improvement efforts at CSC and Lucent. Previously, he was a systems architecture review leader at Bell Labs (Lucent) and facilitated numerous risk identification and project retrospective sessions. Prior to 1997 Michael worked in the Risk and Process programs at the SEI where he was the technical lead for teams that developed the SCE and CBA-IPI appraisal methods. His IT career began with ten years as an Air Force computer officer, working in satellite systems, management information systems, and as an instructor of software engineering at the Air Force Institute of Technology.
  3. 3. Contextually-Driven Architecture Reviews For Better Software Conference East F. Michael Dedolph fmdedolph@netscape.net 14 November 2013 Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 1 Author’s Note Most of this presentation material was used previously in these venues. –  SEI’s SATURN 2009 conference –  Washington DC Area SPIN meeting in Feb 2009 –  PNSQC 2010 Architecture reviews were a good idea then, and they still are Better Software East Contextually Driven Architecture Reviews – Dedolph Contextually Driven Architecture Reviews Dedolph – 10/18/2010 11/14/2013 2 1
  4. 4. What is “Architecture”? Ø  Before we can review it, we should know what “it” is Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 3 “A System Architecture . . .” Provides a solution to a problem for a client –  Early on, it is a conceptual or potential solution Architecture has also been called “design with constraints” –  Alternatively, you could say architecture includes a design that works within the constraints, including cost and schedule! Ø Any given architecture is NOT the ONLY solution, but, some solutions are better than others Better Software East Contextually Driven Architecture Reviews – Dedolph Contextually Driven Architecture Reviews Dedolph – 10/18/2010 11/14/2013 4 2
  5. 5. Where Architecture Exists •  Architecture exists at the intersection of management, technology, and design Management Technology Design Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 5 Architecture Problem Statements •  The problem space has multiple aspects: “FFETO” –  Function – what it does, how well it does it. –  Form – what it “looks like”; includes major environmental interfaces –  Economy – cost, including development, maintenance, material, system retirement, etc. –  Time – relationship to past, present, and future –  Operational/ Developmental – Things that pertain to development constraints and system operating environment, rather than the system Better Software East Contextually Driven Architecture Reviews – Dedolph Contextually Driven Architecture Reviews Dedolph – 10/18/2010 11/14/2013 6 3
  6. 6. Problem Statements Ø are the basis for the review! Provide a succinct summary of critical success criteria –  Short: 2 pages is good –  Includes major constraints –  Client-centric –  Cover Function, Form, Economy, Time, and (optionally) Operational –  Be expressed in sufficient detail to make judgments about the proposed solution, but are not unnecessarily proscriptive of prescriptive Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 7 Problem Statements Ø Are NOT –  A requirements document, but may summarize the most critical high level requirements Problem statements also have unstated "always" criteria, e.g.: –  Possible to construct –  Can make money/ will provide value –  Solution won't result in harm –  Legal/ regulatory constraints Ø The team must understand these as well. Better Software East Contextually Driven Architecture Reviews – Dedolph Contextually Driven Architecture Reviews Dedolph – 10/18/2010 11/14/2013 8 4
  7. 7. Prob. Statements Provide Context •  Problem statements can be applied to almost any product or service. •  Advantage: representational independence •  Many architecture review techniques depend on the way the design is represented. •  But, design can be represented in many different ways —e.g., network diagrams, UML, physical prototypes, circuit diagrams, flow charts, use cases . . . Ø No representation captures all aspects of the system, and focusing too much on representation can actually obscure problem areas Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 9 Classical Architecture - Better Software East Contextually Driven Architecture Reviews – Dedolph Contextually Driven Architecture Reviews Dedolph – 10/18/2010 11/14/2013 10 5
  8. 8. Taj Problem Statement Included: Function: tomb, mosque, hotel, memorial Form: symmetry, untarnished finish, unprecedented weight, shearing force of the river, visual perspective maintained Economy: irrelevant, it will cost what it costs Time: it will take as long as it takes, but must last for all time Operational: expertise is needed from other kingdoms, supply chain issues Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 11 Architecture Review Method, in a Nutshell •  Define the problem the client wants solved •  Compare it to the architecture (proposed solution) •  Identify the gaps (or risks) Then: Ø Let the project resolve the gaps Better Software East Contextually Driven Architecture Reviews – Dedolph Contextually Driven Architecture Reviews Dedolph – 10/18/2010 11/14/2013 12 6
  9. 9. Architecture Review Questions Questions to answer before a review: •  Who decides? (Know the client) •  What problem are we trying to solve? •  Are key stakeholder interests represented? Questions to answer during a review: •  How good is the proposed solution? •  What are the issues/ risks/ gaps in the solution Questions to answer after a review: •  Which things will we (the project team) address? •  How will we (the project team) address them? Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 13 Ground Rules for Reviews Ground rules are covered in the pre-review, and again in the review meeting •  No attribution—review products, processes, and ideas, not people •  Team is there to identify, not solve problems •  Client requests (and pays) for the review, but, the project owns the findings and responsibility for correcting them Ø One exception, “Management Alert” (covered later) •  Reviewers can ask questions at any time Better Software East Contextually Driven Architecture Reviews – Dedolph Contextually Driven Architecture Reviews Dedolph – 10/18/2010 11/14/2013 14 7
  10. 10. Review Steps •  Preparation •  Respond to initial contact/request; develop problem statement; select team and develop agenda; arrange logistics – travel, space, connectivity; pre-review call •  Review Meeting •  Project presentations with Q&A; review team Caucus; conduct readout; client meeting (optional) •  Follow-up •  Write and distribute the Review Report; present findings to lessons learned/governing groups (Board Report); review the Project’s action plans; Close the review by meeting with project and/or sponsor Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 15 Preparation – Some Thoughts Toughest part: Problem Statement •  •  •  I have seen projects cancel a review because they “didn’t have time to develop a problem statement” How do you think the project fared? In many cases, the initial review schedule was delayed while the project and client worked out the problem statement The Problem Statement is reviewed in detail during the pre-review meeting, used to finalize agenda and assign pre-reading Team selection •  •  Based on the Problem Statement Key roles – Leader, “Angel”, SMEs Better Software East Contextually Driven Architecture Reviews – Dedolph Contextually Driven Architecture Reviews Dedolph – 10/18/2010 11/14/2013 16 8
  11. 11. Review Meeting - Overview •  Presentations – by the project team, any format, reusing existing materials •  Questions by the review team, at any time, moderated by the Leader or Angel •  Strengths, Issues, Observations recorded on index cards (Low Tech) •  Team Caucus determines findings •  Readout and (Optional) Sponsor/ Client meeting Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 17 Snow Card Example - 1 Save for Sequence Number Write large, One thought per card, with enough detail to be useful – but not too wordy. Save for Severity { Paper-based reviews can be hard to transcribe! FMD Note here if card is a Strength or Observation; default is an Issue Better Software East Contextually Driven Architecture Reviews – Dedolph Contextually Driven Architecture Reviews Dedolph – 10/18/2010 Don’t Forget Your Initials! 11/14/2013 18 9
  12. 12. Review Meeting - Caucus •  Led by review Leader and Angel. •  Categorization is done to provide a consistent “story” for the read-out –  A “typical” review produces 80–350 observations, arranged into 10-25 topics •  Issues are Prioritized:“Management Alert”, Critical, Major, Minor •  Summary of each topic area written by team subject matter experts •  Strengths should be preserved if changes are made to the architecture Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 19 Prioritization by the Review Team Management Alert: •  Goes to client/upper management •  “In the unanimous opinion of the Review Team, the project will fail unless these issues are immediately addressed: <List of issues>” Critical, Major, Minor, Strengths, & Suggestions go to the project •  Readout ensures no surprises in the report •  The review team will NOT withdraw a finding or lower the priority (project owns the action plan) •  Priority may be increased at project request Better Software East Contextually Driven Architecture Reviews – Dedolph Contextually Driven Architecture Reviews Dedolph – 10/18/2010 11/14/2013 20 10
  13. 13. Follow-up •  Readout and Sponsor/ Client meeting •  Management Alert and Critical Issue Letters within a week Ø Most management alerts are related to cost, schedule, and resources, NOT purely technical issues. Hence, upper management is responsible for the action plan for management alerts •  Critical Issue letters goes to the Project, Project management is responsible for the action plan •  Two Reports, one to the project, one to the Board. •  Review team Leader and Angel review the action plans. Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 21 What Happens When? • . . . a project ignores a Management Alert? • . . . a project ignores a Critical issue? • . . . a project fixes everything? • . . . the review team identifies an issue that turns out to not be a problem (“false positives”)? Better Software East Contextually Driven Architecture Reviews – Dedolph Contextually Driven Architecture Reviews Dedolph – 10/18/2010 11/14/2013 22 11
  14. 14. What's Different (or the Same) About this Kind of Review? Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 23 What's Different? •  Focus –  problem/solution congruence, Client ownership of the decision making, ownership by project team •  Context/ Domain independent –  Notation and representation independent, not model or standard based (although this can be included in the problem statement, if needed) •  Follow up –  Clear statement of potential failure points, Project sees and owns findings – can ignore or post on their web site Better Software East Contextually Driven Architecture Reviews – Dedolph Contextually Driven Architecture Reviews Dedolph – 10/18/2010 11/14/2013 24 12
  15. 15. What's the Same? Outside eyes—external to team Products, processes, ideas are reviewed, not people Not a problem solving session Ø  Balances a defined review process with extreme flexibility. Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 25 Sponsor Survey - Benefits Saved money/time •  Estimated at > $1M per large project, similar estimates were derived from Monte Carlo simulations based on analyzing the impact from correcting review findings •  Direct results from specific reviews, e.g., re-architecting after the review trimmed 9 months off the schedule, saved > $7 million. Intangible benefits include: •  Preparation (especially preparing the problem statement) forces people to think through the problem •  Cross-pollination of techniques across the organization •  Learning as a review team member •  Synching up project teams; “on the same page” Better Software East Contextually Driven Architecture Reviews – Dedolph Contextually Driven Architecture Reviews Dedolph – 10/18/2010 11/14/2013 26 13
  16. 16. Drawbacks to the Method •  Risks of a Client-centered method •  The Client can grow out of touch with the market, customer needs, and end user issues •  In contracting, many of the success criteria are negotiated into the contract, and may represent political compromises that can’t be changed easily •  Method dependencies •  Strongly oriented to face-to-face meetings, requires a deep pool of reviewers with expertise •  Incomplete findings/level of detail •  Any review’s findings are inherently incomplete. A high level review is not suited for all problem spaces; 2 to 3 days will not flush out all of the “devils in the details” Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 27 Institutionalizing the Method •  Requires building a “learning culture” Oversight Executive Champion Review Board “Angels” Push Review Organization Project Project Project Project Project Project Better Software East Review Leaders Reviewers, Reviews Contextually Driven Architecture Reviews – Dedolph Contextually Driven Architecture Reviews Dedolph – 10/18/2010 11/14/2013 28 14
  17. 17. Suggested Steps for Making Reviews Part of the Culture •  •  •  •  Start with a high-level champion/ executive sponsor Obtain central funding for an extended period – say 5 years Train a core team of leaders and angels to do reviews Charter a board; recruit advocates. Board members are expected to serve as review “Angels” •  Perform the reviews; capture data and lessons learned to document the method’s value •  Move to a sponsor-based funding model after the initial time period, when the method has proven itself •  Rigorously retain the focus on client-centered problem statements, project ownership of findings, limited but pertinent management alerts, and non-attribution Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 29 11/14/2013 30 Questions??? Better Software East Contextually Driven Architecture Reviews – Dedolph Contextually Driven Architecture Reviews Dedolph – 10/18/2010 15
  18. 18. Presenter Info F. Michael Dedolph 412-477-7163 fmdedolph@netscape.net BIO: F. Michael Dedolph is currently an innkeeper and owner of a bustling B&B, and part time consultant to the IT industry for reviews, risk management, quality methods, and process improvement. From 1997 to 2004, Michael was a systems architecture review leader at Bell Labs (Lucent); he also managed and taught Lucent’s Systems Architecture Introduction class. In addition to leading architecture Review Teams, he facilitated numerous risk identification, problem solving, and project retrospective sessions. Prior to 1997, Michael worked in the Risk and Process programs at the SEI. While at the SEI, he was the technical lead for the teams that developed the SCE and CBA-IPI appraisal methods, and was the team leader for several Risk Reviews. He started his IT career by spending 10 years as an Air Force computer officer, and also spent 7 years at CSC leading process improvement teams. Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 31 References: 1. CMU/SEI-2000-TR-004, "ATAM: Method for Architecture Evaluation"; Kazman, Klein, Clements, 2000 2. Problem Seeking: An Architectural Programming Primer, 4th Ed; Pena and Parshall, 2001, ISBN 0-913962-87-2 3. Handbook of Walkthroughs, Inspections, and Technical Reviews; Freedman and Weinberg, 1990, ISBN 0-932633-19-6 4. IEEE Software, March/April 2005, "Architecture Reviews: Practice and Experience"; Maranzano, Rozsypal, Zimmerman, Warnken, Wirth, and Weiss 5. STQE Jul/Aug 2002 (Vol. 4, Issue 4) Feature: Measurement & Analysis "A Blueprint for Success: Implementing an architectural review system“; Daniel Starr, Gus Zimmerman 6. CMU/SEI-2006-TR-012, "Risk Themes Discovered Through Architecture Evaluations"; Bass, Nord, Wood, Zubrow; 2006 7. CMU/SEI-2010-TN-018, “Relating Business Goals to Architecturally Significant Requirements for Software Systems”; Clements, Bass; 2010 8. CMU/SEI Webinar, “SoS Architecture Evaluation and Quality Attribute Specification (Webinar)”; Gagliardi; 2010 Better Software East Contextually Driven Architecture Reviews – Dedolph Contextually Driven Architecture Reviews Dedolph – 10/18/2010 11/14/2013 32 16

×