Contextually-Driven System Architecture Reviews

BT2
Concurrent Session
11/14/2013 10:15 AM

"Contextually-Driven System
Architecture Reviews "

Presented by:
F. Michael Dedolph
Levi Deal Consulting

Brought to you by:

340 Corporate Way, Suite 300, Orange Park, FL 32073
888‐268‐8770 ∙ 904‐278‐0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

Michael Dedolph
Levi Deal Consulting

Michael Dedolph is self-employed as a consultant and innkeeper. For the
past seven years, Michael led quality process improvement efforts at CSC
and Lucent. Previously, he was a systems architecture review leader at Bell
Labs (Lucent) and facilitated numerous risk identification and project
retrospective sessions. Prior to 1997 Michael worked in the Risk and
Process programs at the SEI where he was the technical lead for teams
that developed the SCE and CBA-IPI appraisal methods. His IT career
began with ten years as an Air Force computer officer, working in satellite
systems, management information systems, and as an instructor of
software engineering at the Air Force Institute of Technology.

Contextually-Driven
Architecture Reviews
For Better Software
Conference East
F. Michael Dedolph
fmdedolph@netscape.net
14 November 2013

Better Software East

Contextually Driven Architecture Reviews – Dedolph

11/14/2013

1

Author’s Note
Most of this presentation material was
used previously in these venues.
–  SEI’s SATURN 2009 conference
–  Washington DC Area SPIN meeting in Feb 2009
–  PNSQC 2010
Architecture reviews were a good idea then, and
they still are


Contextually Driven Architecture Reviews
Dedolph – 10/18/2010

11/14/2013

2

1

What is “Architecture”?
Ø  Before we can review it,
we should know what “it” is



11/14/2013

3

“A System Architecture . . .”
Provides a solution to a problem for a
client
–  Early on, it is a conceptual or potential solution

Architecture has also been called
“design with constraints”
–  Alternatively, you could say architecture includes
a design that works within the constraints,
including cost and schedule!
Ø Any given architecture is NOT the ONLY
solution, but, some solutions are better than
others


Dedolph – 10/18/2010

11/14/2013

4

2

Where Architecture Exists
•  Architecture exists at the intersection
of management, technology, and
design

Management

Technology

Design



11/14/2013

5

Architecture Problem Statements
•  The problem space has multiple
aspects: “FFETO”
–  Function – what it does, how well it does it.
–  Form – what it “looks like”; includes major
environmental interfaces
–  Economy – cost, including development,
maintenance, material, system retirement, etc.
–  Time – relationship to past, present, and future
–  Operational/ Developmental – Things that
pertain to development constraints and system
operating environment, rather than the system


Dedolph – 10/18/2010

11/14/2013

6

3

Problem Statements
Ø are the basis for the review!
Provide a succinct summary of critical
success criteria
–  Short: 2 pages is good
–  Includes major constraints
–  Client-centric
–  Cover Function, Form, Economy, Time, and
(optionally) Operational
–  Be expressed in sufficient detail to make
judgments about the proposed solution, but are
not unnecessarily proscriptive of prescriptive


11/14/2013

7

Problem Statements
Ø Are NOT
–  A requirements document, but may summarize
the most critical high level requirements

Problem statements also have unstated
"always" criteria, e.g.:
–  Possible to construct
–  Can make money/ will provide value
–  Solution won't result in harm
–  Legal/ regulatory constraints
Ø The team must understand these as well.


Dedolph – 10/18/2010

11/14/2013

8

4

Prob. Statements Provide Context
•  Problem statements can be applied to
almost any product or service.
•  Advantage: representational
independence
•  Many architecture review techniques depend on the way
the design is represented.
•  But, design can be represented in many different ways
—e.g., network diagrams, UML, physical prototypes,
circuit diagrams, flow charts, use cases . . .

Ø No representation captures all aspects of the
system, and focusing too much on representation
can actually obscure problem areas


11/14/2013

9

Classical Architecture -



Dedolph – 10/18/2010

11/14/2013

10

5

Taj Problem Statement Included:
Function: tomb, mosque, hotel, memorial
Form: symmetry, untarnished finish,
unprecedented weight, shearing force of
the river, visual perspective maintained
Economy: irrelevant, it will cost what it costs
Time: it will take as long as it takes, but must
last for all time
Operational: expertise is needed from other
kingdoms, supply chain issues


11/14/2013

11

Architecture Review Method, in a
Nutshell
•  Define the problem the client wants
solved
•  Compare it to the architecture
(proposed solution)
•  Identify the gaps (or risks)
Then:
Ø Let the project resolve the gaps


Dedolph – 10/18/2010

11/14/2013

12

6

Architecture Review Questions
Questions to answer before a review:
•  Who decides? (Know the client)
•  What problem are we trying to solve?
•  Are key stakeholder interests represented?

Questions to answer during a review:
•  How good is the proposed solution?
•  What are the issues/ risks/ gaps in the solution

Questions to answer after a review:
•  Which things will we (the project team) address?
•  How will we (the project team) address them?


11/14/2013

13

Ground Rules for Reviews
Ground rules are covered in the pre-review, and
again in the review meeting
•  No attribution—review products, processes, and
ideas, not people
•  Team is there to identify, not solve problems
•  Client requests (and pays) for the review, but,
the project owns the findings and responsibility
for correcting them
Ø One exception, “Management Alert” (covered later)

•  Reviewers can ask questions at any time



Dedolph – 10/18/2010

11/14/2013

14

7

Review Steps
•  Preparation
•  Respond to initial contact/request; develop problem
statement; select team and develop agenda; arrange
logistics – travel, space, connectivity; pre-review call

•  Review Meeting
•  Project presentations with Q&A; review team Caucus;
conduct readout; client meeting (optional)

•  Follow-up
•  Write and distribute the Review Report; present
findings to lessons learned/governing groups (Board
Report); review the Project’s action plans; Close the
review by meeting with project and/or sponsor


11/14/2013

15

Preparation – Some Thoughts
Toughest part: Problem Statement
• 
• 
• 

I have seen projects cancel a review because
they “didn’t have time to develop a problem
statement” How do you think the project fared?
In many cases, the initial review schedule was
delayed while the project and client worked out
the problem statement
The Problem Statement is reviewed in detail
during the pre-review meeting, used to finalize
agenda and assign pre-reading

Team selection
• 
• 

Based on the Problem Statement
Key roles – Leader, “Angel”, SMEs



Dedolph – 10/18/2010

11/14/2013

16

8

Review Meeting - Overview
•  Presentations – by the project team,
any format, reusing existing materials
•  Questions by the review team, at any
time, moderated by the Leader or
Angel
•  Strengths, Issues, Observations
recorded on index cards (Low Tech)
•  Team Caucus determines findings
•  Readout and (Optional) Sponsor/
Client meeting


11/14/2013

17

Snow Card Example - 1
Save for
Sequence
Number

Write large,
One thought
per card,
with enough
detail to be
useful – but
not too
wordy.

Save for Severity

{

Paper-based
reviews can be
hard to transcribe!
FMD

Note here if card is a Strength or
Observation; default is an Issue


Dedolph – 10/18/2010

Don’t Forget
Your Initials!
11/14/2013

18

9

Review Meeting - Caucus
•  Led by review Leader and Angel.
•  Categorization is done to provide a
consistent “story” for the read-out
–  A “typical” review produces 80–350 observations,
arranged into 10-25 topics

•  Issues are Prioritized:“Management
Alert”, Critical, Major, Minor
•  Summary of each topic area written by
team subject matter experts
•  Strengths should be preserved if changes
are made to the architecture


11/14/2013

19

Prioritization by the Review Team
Management Alert:
•  Goes to client/upper management
•  “In the unanimous opinion of the Review Team,
the project will fail unless these issues are
immediately addressed: <List of issues>”

Critical, Major, Minor, Strengths, &
Suggestions go to the project
•  Readout ensures no surprises in the report
•  The review team will NOT withdraw a finding or
lower the priority (project owns the action plan)
•  Priority may be increased at project request


Dedolph – 10/18/2010

11/14/2013

20

10

Follow-up
•  Readout and Sponsor/ Client meeting
•  Management Alert and Critical Issue
Letters within a week
Ø Most management alerts are related to cost,
schedule, and resources, NOT purely technical
issues. Hence, upper management is responsible
for the action plan for management alerts
•  Critical Issue letters goes to the Project, Project
management is responsible for the action plan

•  Two Reports, one to the project, one to
the Board.
•  Review team Leader and Angel review
the action plans.


11/14/2013

21

What Happens When?
• . . . a project ignores a Management
Alert?
• . . . a project ignores a Critical issue?
• . . . a project fixes everything?
• . . . the review team identifies an issue
that turns out to not be a problem
(“false positives”)?



Dedolph – 10/18/2010

11/14/2013

22

11

What's Different (or the Same)
About this Kind of Review?



11/14/2013

23

What's Different?
•  Focus
–  problem/solution congruence, Client ownership of
the decision making, ownership by project team

•  Context/ Domain independent
–  Notation and representation independent, not
model or standard based (although this can be
included in the problem statement, if needed)

•  Follow up
–  Clear statement of potential failure points, Project
sees and owns findings – can ignore or post on
their web site


Dedolph – 10/18/2010

11/14/2013

24

12

What's the Same?
Outside eyes—external to team
Products, processes, ideas are
reviewed, not people
Not a problem solving session
Ø  Balances a defined review
process with extreme flexibility.



11/14/2013

25

Sponsor Survey - Benefits
Saved money/time
•  Estimated at > $1M per large project, similar estimates
were derived from Monte Carlo simulations based on
analyzing the impact from correcting review findings
•  Direct results from specific reviews, e.g., re-architecting
after the review trimmed 9 months off the schedule,
saved > $7 million.

Intangible benefits include:
•  Preparation (especially preparing the problem
statement) forces people to think through the problem
•  Cross-pollination of techniques across the organization
•  Learning as a review team member
•  Synching up project teams; “on the same page”


Dedolph – 10/18/2010

11/14/2013

26

13

Drawbacks to the Method
•  Risks of a Client-centered method
•  The Client can grow out of touch with the market,
customer needs, and end user issues
•  In contracting, many of the success criteria are
negotiated into the contract, and may represent political
compromises that can’t be changed easily

•  Method dependencies
•  Strongly oriented to face-to-face meetings, requires a
deep pool of reviewers with expertise

•  Incomplete findings/level of detail
•  Any review’s findings are inherently incomplete. A high
level review is not suited for all problem spaces; 2 to 3
days will not flush out all of the “devils in the details”


11/14/2013

27

Institutionalizing the Method
•  Requires building a “learning culture”
Oversight

Executive
Champion

Review
Board
“Angels”

Push

Review
Organization

Project
Project
Project
Project
Project
Project

Review
Leaders

Reviewers,
Reviews

Dedolph – 10/18/2010

11/14/2013

28

14

Suggested Steps for Making
Reviews Part of the Culture
• 
• 
• 
• 

Start with a high-level champion/ executive sponsor
Obtain central funding for an extended period – say 5 years
Train a core team of leaders and angels to do reviews
Charter a board; recruit advocates. Board members are
expected to serve as review “Angels”
•  Perform the reviews; capture data and lessons learned to
document the method’s value
•  Move to a sponsor-based funding model after the initial
time period, when the method has proven itself
•  Rigorously retain the focus on client-centered problem
statements, project ownership of findings, limited but
pertinent management alerts, and non-attribution


11/14/2013

29

11/14/2013

30

Questions???



Dedolph – 10/18/2010

15

Presenter Info
F. Michael Dedolph
412-477-7163
fmdedolph@netscape.net

BIO:
F. Michael Dedolph is currently an innkeeper and owner of a bustling B&B, and part
time consultant to the IT industry for reviews, risk management, quality methods,
and process improvement.
From 1997 to 2004, Michael was a systems architecture review leader at Bell Labs
(Lucent); he also managed and taught Lucent’s Systems Architecture Introduction
class. In addition to leading architecture Review Teams, he facilitated numerous
risk identification, problem solving, and project retrospective sessions.
Prior to 1997, Michael worked in the Risk and Process programs at the SEI. While
at the SEI, he was the technical lead for the teams that developed the SCE and
CBA-IPI appraisal methods, and was the team leader for several Risk Reviews.
He started his IT career by spending 10 years as an Air Force computer officer, and
also spent 7 years at CSC leading process improvement teams.


11/14/2013

31

References:
1. CMU/SEI-2000-TR-004, "ATAM: Method for Architecture Evaluation";
Kazman, Klein, Clements, 2000
2. Problem Seeking: An Architectural Programming Primer, 4th Ed;
Pena and Parshall, 2001, ISBN 0-913962-87-2
3. Handbook of Walkthroughs, Inspections, and Technical Reviews;
Freedman and Weinberg, 1990, ISBN 0-932633-19-6
4. IEEE Software, March/April 2005, "Architecture Reviews: Practice and Experience";
Maranzano, Rozsypal, Zimmerman, Warnken, Wirth, and Weiss
5. STQE Jul/Aug 2002 (Vol. 4, Issue 4) Feature: Measurement & Analysis
"A Blueprint for Success: Implementing an architectural review system“;
Daniel Starr, Gus Zimmerman
6. CMU/SEI-2006-TR-012, "Risk Themes Discovered Through Architecture Evaluations";
Bass, Nord, Wood, Zubrow; 2006
7. CMU/SEI-2010-TN-018, “Relating Business Goals to Architecturally Significant
Requirements for Software Systems”; Clements, Bass; 2010
8. CMU/SEI Webinar, “SoS Architecture Evaluation and Quality Attribute Specification
(Webinar)”; Gagliardi; 2010



Dedolph – 10/18/2010

11/14/2013

32

16

Contextually-Driven System Architecture Reviews

More Related Content

What's hot

Viewers also liked

Similar to Contextually-Driven System Architecture Reviews

More from TechWell

Recently uploaded

Contextually-Driven System Architecture Reviews