Visual Architecting

No notes for slide

I can only say that in accepting this award, I am standing in for and on behalf of everyone who partnered and worked with me over the years I In particular, share this recognition with Dana Bredemeyer, and with everyone who recommended me for this award, for each has worked with me, and so contributed immensely to my experience and what I bring to others.

we’re going to talk about architecture from the inside out — defining concerns at center of how we think about architecture — and while 30 minutes is way too long to listen to me talk, it’s also not nearly long enough to back out to a fully encompassing view of architecture and what architects do, especially since I also promised to relate our discussion to visual architecting and how we give ourselves a visual boost to our embodied cognition and team cognition in the wild

wind back time to 1992

asks Ralph Johnson

we know what to do with decisions — or we will after Michael Keeling and Joe Runde’s talk at 1:00
very much like the patterns template that Chris Richardson described so well yesterday

<sumptuous emoluments>
and if you want to know how much the software community cares about architects and/or architecturally significant decisions, we see it quantified there — it’s 35 retweets

significant — cost of change — Kevlin drew our attention to this in the opening keynote on Tuesday

if you haven’t read it, go there, print it off — leave it conspicuously on

protect by not naming the source!

isolation zones — uncertainty and experiment
"The price of reliability is the pursuit of the utmost simplicity” – Hoare

“I define architecture as a word we use …when we want to puff it up to sound important” — Martin Fowler

Ralph Johnson: architecture is shared understanding among the developers which includes

Sometimes we just need to get a clue, and this talk has a bucket full. Ruth Malan explores the playful idea of a clue bucket (so that we can reach in and get a clue when we need one). Ruth considers sources of design inspiration—along with a collection of design guides, principles, tips, and heuristics—which she contextualizes by outlining the kinds of challenges the architect addresses.

Sometimes we just need to get a clue, and this talk has a bucket full. Ruth Malan explores the playful idea of a clue bucket (so that we can reach in and get a clue when we need one). Ruth considers sources of design inspiration—along with a collection of design guides, principles, tips, and heuristics—which she contextualizes by outlining the kinds of challenges the architect addresses.

That’s an accent, honey

more than we get by default; when we aren’t paying attention; don’t bring experience to bear

Boxes — how we render abstractions that are key building blocks / architectural elements or significant chunks or components that give shape to our system

one design will meet a set of goals better than another design

better design intuition

natural structures — like rivers and mountains, or organs like heart and lungs — don’t give us “obvious boundaries” — we have to invent them… but there are clues … where the landscape or terrain shifts …

Maps... navigation; location; planning
But we are designing; shaping; deciding
Not simply cartographers of a landscape; deciding where to create boundaries -- sure, trying to find the natural contours; but we're assessing alternatives and thinking them up!

Natural fit - to context (seams and cleavages in host systems; looking at distribution -- physical world is real af ,.. latency, intruders; failures….

Zhuang Zhou, often known as Zhuangzi, was an influential Chinese philosopher who lived around the 4th century BC during the Warring States period, a period corresponding to the summit of Chinese philosophy, the Hundred Schools of Thought.
leading philosopher representing the Taoist strain in Chinese thought.

Cook Ting was cutting up an ox for Lord Wen-hui. As every touch of his hand, every heave of his shoulder, every move of his feet, every thrust of his knee — zip! zoop! He slithered the knife along with a zing, and all was in perfect rhythm, as though he were performing the dance of the Mulberry Grove or keeping time to the Ching-shou music.
“Ah, this is marvelous!” said Lord Wen-hui. “Imagine skill reaching such heights!”
Cook Ting laid down his knife and replied, “What I care about is the Way, which goes beyond skill. When I first began cutting up oxen, all I could see was the ox itself. After three years I no longer saw the whole ox. And now — now I go at it by spirit and don’t look with my eyes. Perception and understanding have come to a stop and spirit moves where it wants. I go along with the natural makeup, strike in the big hollows, guide the knife through the big openings, and following things as they are. So I never touch the smallest ligament or tendon, much less a main joint.
“A good cook changes his knife once a year — because he cuts. A mediocre cook changes his knife once a month — because he hacks. I’ve had this knife of mine for nineteen years and I’ve cut up thousands of oxen with it, and yet the blade is as good as though it had just come from the grindstone. There are spaces between the joints, and the blade of the knife has really no thickness. If you insert what has no thickness into such spaces, then there’s plenty of room — more than enough for the blade to play about it. That’s why after nineteen years the blade of my knife is still as good as when it first came from the grindstone.
“However, whenever I come to a complicated place, I size up the difficulties, tell myself to watch out and be careful, keep my eyes on what I’m doing, work very slowly, and move the knife with the greatest subtlety, until — flop! the whole thing comes apart like a clod of earth crumbling to the ground. I stand there holding the knife and look all around me, completely satisfied and reluctant to move on, and then I wipe off the knife and put it away.”
“Excellent!” said Lord Wen-hui. “I have heard the words of Cook Ting and learned how to care for life!”

“Yeah -- best case you have 1 place to change instead of N, worst case you have N + 1 places ;-)” -- @schmonz on adapters

Fort design evolved as canon firing range and other weapons capabilities evolved… and with each advance in one, pressure was on to advance the other, to gain advantage…

why resilience is a watch word for us… ability to sense recover respond and out maneuver

Our Abstractions are well abstract… so we draw them as boxes and these boxes get their meaning from what we mean by them

“Programming is the breaking of one big impossible task into several very small possible tasks.” - Jazzwant

“Software is built on abstractions. Pick the right ones, and programming will flow naturally from design; modules will have small and simple interfaces; and new functionality will more likely fit in without extensive reorganization. Pick the wrong ones, and programming will be a series of nasty surprises”
— Daniel Jackson

ddd domain driven design
how do we do that? Look for the natural structure, the natural interstices — and “when I come to the tricky parts, I slow down”

As you try to model a larger domain, it gets progressively harder to build a single unified model. Different groups of people will use subtly different vocabularies in different parts of a large organization. The precision of modeling rapidly runs into this, often leading to a lot of confusion. Typically this confusion focuses on the central concepts of the domain. Early in my career I worked with a electricity utility - here the word "meter" meant subtly different things to different parts of the organization: was it the connection between the grid and a location, the grid and a customer, the physical meter itself (which could be replaced if faulty). These subtle polysemes could be smoothed over in conversation but not in the precise world of computers. Time and time again I see this confusion recur with polysemes like "Customer" and "Product".
DDD recognizes that we've learned that "total unification
of the domain model for a large system will not be
feasible or cost-effective" [1]. So instead DDD divides
up a large system into Bounded Contexts, each of
which can have a unified model - essentially a way of
structuring MultipleCanonicalModels.
-- Martin Fowler

Implementing Domain-Driven Design(Hardcover)
by Vaughn Vernon

http://martinfowler.com/bliki/BoundedContext.html

Sun Tsu Art of War

Our Abstractions are well abstract… so we draw them as boxes and these boxes get their meaning from what we mean by them

so we need to write down what we mean by them

amazed that we think we agree, when we use a named abstractions, but our assumptions and mental model can differ substantively…

CRCs 89 Beck and Cunningham

fundamentals remain fundamental mne·mon·ic — scars

SCARS — what we get from experience, “You have a point, you can take it out of my back now!”

and in particular, architecting impacts many people, who’ll all have a better idea

In 1982, Edsger Dijkstra wrote another classic paper entitled On the role of scientific thought. in which he introduced the term: The Separation of Concerns.
http://www.cs.utexas.edu/users/EWD/ewd04xx/EWD447.PDF

Jeremy Campbell, GRAMMATICAL MAN: Information, Entropy, Language and Life

Systems evolve – we’re constantly adapting them.
Complexity from accrual
Complexity from mess

Accidental complexity
Versus inherent complexity

paraphrase as “if the org arch and the system arch are at odds, the org architecture wins — overrides architect’s intent and determines actual structure of the system
let your team boundaries underscore the natural and determined/imposed boundaries of the system; if you want to separate and keep them separate, give them to different teams

Isolation is the most important trait. It is the foundation for many of the high-level benefits in microservices. But it is probably also the trait that has the biggest impact on your design and architecture. It will, and should, slice up the whole architecture, and therefore it needs to be considered from day one. It will even impact the way you break up and organize the teams and their responsibilities, as Melvyn Conway discovered and was later turned into Conway’s Law in 1967
https://www.oreilly.com/ideas/defining-a-reactive-microservice

“in technology we rarely start from zero. The choices we make persist in the software we create. As long as that software exists, it’s organizational memory. It makes some things easy and other things hard. Software isn’t a mechanical thing. In an odd way it’s alive. It grows. When we modify it, it reacts to us. Because of its presence, its value, and its constraints, we react to it.

I see our relationship to software as a symbiotic relations”
—Michael Feathers

Homomorphism?

“The odd thing about this split between business and development is that you can see it in the code. One of the key insights in software development is Conway’s Law. It states that the structure of what we produce ends up mirroring the structure of the organizations that produce it. Conway’s Law is a deep insight and there is much more to say about it but with regard to the business/development split it tells us that we should expect poor quality software when there’s a handoff or any gap in knowledge between groups.

Once you understand the Conway’s Law many things that seem surprising about software development make sense. When many teams work in the same code base, able to touch any part of it, there’s a tendency toward the Tragedy of the Commons. When individual teams concentrate on their own area of the code, the code reacts by modularizing over time. Process impacts code too. Frankly, everything does. This makes it important for us to have awareness of these effects. “
http://www.r7krecon.com/#!provocation/gfqa5

Code adjusts to us and we adjust to code. Code tends to mirror our team structure, and we create structures to deal with the effects of code - we create customer support, quality assurance and maintenance organizations. If code and organization interact at this deep level, the way we form teams and the way we work are vitally important.

When we ask what the code needs, we arrive at the insight that it needs different expertise at different times.
http://www.r7krecon.com/#!implications/t2tbw

“You always ship your organization” — feathers

you for example are a system, a biological organism, and you consist of parts, your heart, your lungs, stomach, pancreas, and so on, each of which can affect your behavior or your property

each part of the system, when it affects the system, is dependent for its effect on some other part, in other words, the parts are interdependent
no part of a system, or collection of parts of a system, has an independent effect on it. the way the heart affects you depends on what the lungs are doing and the brain and so forth. the parts are all interconnected. Therefore, the system is a whole that cannot be divided into independent parts

very important implications that are generally overlooked

the essential or defining properties of a system are properties of the whole which none of its parts have
for example, car — essential property is it can carry you from one place to another; not part of a car can do that — the wheel can’t, the axle can’t, the motor can’t even carry itself from one place to another, but the car can

you have life, none of your parts live; you can write, your hand can’t write — that’s easy to demonstrate — cut off your hand and see if it can write

when a system is taken apart, it loses its essential properties

if disassemble car in this room, even though have every part, its not a car, because system isn’t sum of behavior of its parts, it’s a product of their interaction

and properties emerge from parts and interactions

“Defining identity not in any one of parts”— Russell Ackoff

If achieve a nirvana of decoupling and independence, we dont have a system — or you have micro services that have grown into their ambitions to be systems

Laugh at microservices integrated at enterprise database but
Ddd 360 view of customer--because we had independently evolving views of "the customer" which flommuxes customers

trying to get more what we want, while doing as little as possible!!
Why? Enable; teams; cognitive traction; organizational traction; isolation zones - experiment; scale; fail;
Address challenges; desired outcomes - how are we going to do that? How do we convince ourselves it is worth trying? How do we make it better? What even is better? What do we have to give up to get that? Is it worth it? How will we convince ourselves? How will we know?
What forces do we need to take into account? How do they shape space? Resolve tensions; balance; tradeoffs
How does this change across contexts?
How do we make it simpler? Do we need to do this? What if we just not? If we do this, what could we remove?

stake in ground — revisit and refactor and revise and try out other alternatives

when assemble IKEA or potter Barn shelves or what have you, don’t tighten first screw put in; put a set in and move to fit before tighten, and tighten all somewhat rather than just one

design across the views; we’re just looking at a simplified pass through architecting here, but we’ll come back and update as we learn from other views, too

When to Model?
To support reasoning:
limitations of the human mind/complexity of the subject of inquiry
difficult to keep track of all dimensions/variables at the same time
make relationships among factors/variables explicit
reveal the structure
explore and illuminate dynamics
draw on analogies and models or theories from other domains
To support collaboration:
facilitate understanding
lead more structured discussions
discuss specifics in the context of the "big picture"
To support experimentation:
modeling effort is small, in comparison with the magnitude of the outcome
perform sensitivity analysis
test assumptions/beliefs
To support persuasion and influence (visual rhetoric):
improves confidence (team: "we can do it"; stakeholders: "they demonstrate they know what they're up against/need to do")
the power has influenced our idiom: "I see what you mean," "it draws people in"
interesting

To observe (more attentively)
To study, think, reason, to puzzle things out
To record
to think longer, harder
to show, to teach
To invent, to combine,to make (new) connections
To test ideas
thought experiments
To persuade

“Good judgment comes from experience, and experience comes from bad judgment.” -- Rita Mae Brown (The same or a closely similar quote is attributed to too many earlier writers to list, including Will Rogers.)

What do we do — selective attention and bubbles?

move to different PoV — value of the process! not to be a big framework thing but scaffolding; and reminder of where we can go (next) to take a different look, etc.
Also, a matter of discipline — self-discipline, and discipline of engineering — probe, criticism, come up with alternatives, model, reason, come up with counter arguments

pick another property as driving requirement,pick a trend and see what new requirements come up, change an assumption,
switch views
bring on other people — diversity; also stakeholders outside team

sequence to communication diagram — emphasizes/punches up topology

We see things not as they are, but as we are”
— Anais Nin

Commitment to making other people successful

Simon Brown’s components and containers in context
Simon: By "container" I mean something like a web server, application server, desktop application, mobile app, database, file system, etc. Essentially, what I call a container is anything that can host code or data.

Neely cartoon — used by Richard Cook at Velocity

LMAX is a retail financial trading platform.

Its business innovation is that it is a retail platform - allowing anyone to trade in a range of financial derivative products

Challenges: A trading platform like this needs very low latency - trades have to be processed quickly because the market is moving rapidly. A retail platform adds complexity because it has to do this for lots of people. So the result is more users, with lots of trades, all of which need to be processed quickly.

Given the shift to multi-core thinking, this kind of demanding performance would naturally suggest an explicitly concurrent programming model - and indeed this was their starting point. But the thing that got people's attention at QCon was that this wasn't where they ended up. In fact they ended up by doing all the business logic for their platform: all trades, from all customers, in all markets - on a single thread. A thread that will process 6 million orders per second using commodity hardware.

Although the business logic occurs in a single thread, there are a number tasks to be done before we can invoke a business object method. The original input for processing comes off the wire in the form of a message, this message needs to be unmarshaled into a form convenient for Business Logic Processor to use. Event Sourcing relies on keeping a durable journal of all the input events, so each input message needs to be journaled onto a durable store. Finally the architecture relies on a cluster of Business Logic Processors, so we have to replicate the input messages across this cluster. Similarly on the output side, the output events need to be marshaled for transmission over the network.

Figure 2: The activities done by the input disruptor (using UML activity diagram notation)
The replicator and journaler involve IO and therefore are relatively slow. After all the central idea of Business Logic Processor is that it avoids doing any IO.

At a crude level you can think of a Disruptor as a multicast graph of queues where producers put objects on it that are sent to all the consumers for parallel consumption through separate downstream queues.

http://martinfowler.com/articles/lmax.html

CQRS ( Command Query Responsibility Segregation)

Powerful Ideas need love too, Alan Kay

The Structure of the Government Must Furnish the Proper Checks and Balances Between the Different Departments, is an essay by James Madison, the fifty-first of The Federalist Papers.

In this Federalist Paper, James Madison explains and defends the checks and balances system in the Constitution. Each branch of government is framed so that its power checks the power of the other two branches; additionally, each branch of government is dependent on the people, who are the source of legitimate authority.
“It may be a reflection on human nature, that such devices [checks and balances] should be necessary to control the abuses of government. But what is government itself, but the greatest of all reflections on human nature? If men were angels, no government would be necessary.
— Billofs=RightsInstitute.org

Theory of Operation
How does it work (together) to create system capabilities (function and properties)
illuminates relationships

about building a theory of how things work — how forces impinge;

presenting the design in terms of forces and outcomes — our reasoning, connecting dots to goals and drivers, identifying constraints, etc

in so presenting the design (fragment), communicate — inform and coach— help others build up theory of this system — intent and realization; and bag theory for next system

notes — what we intend
and reflection of code

Da vinci with notebook -- always learning; share notes;
Draw -- use uml; umlish; ad hoc; annotate/accompanying notes; model to learn/see (better); to inform; to create shared thoughtspace; modelstorm; model out loud; in pairs and mobs... to explore... to test!! To try ideas out; to "run" changes across existing system and see what is impacted; to try alternatives and see which are promising enough to try in code. Can we afford to? Can we afford not to??! recall these are make or break! Can we afford to only try things in code??? Cheapest experiments to decide which experiments! Not just diverge and converge with "requirements"

"But the brain does much more than just recollectIt inter-compares, it synthesizes, it analyzes it generates abstractions...The brain has it's own languageFor testing the structure and consistency of the world”
— Carl Sagan
-- Carl Sagan 'A Glorious Dawn' ft Stephen Hawking (Cosmos Remixed) by John Boswell

A hotspot is complicated code that you have to work with often. Hotspots are calculated from two different data sources:
1 We use the lines of code as a simple proxy for complexity.
2 We calculate the change frequency of each file by mining their version-control history.

As you see in the picture above, the prioritized Hotspots only make up 2-3% of the total code size. Yet there’s a disproportional amount of development activity in that small part with 11-16% of all commits touching those Hotspots. This means any code improvement to a prioritized Hotspot is time well-invested.

Changing; evolving; reflection of design as built; check code against models; throw changes at evolving models; probes in code -- operational concerns; scale; attacks; also code health monitoring; code counting -- not nirvana but not going to lie to itself…

Not just what it looks like; how it works -- at every scope and scale... not just early. Throughtout... not just ci/cd ... lifecycle ...,

All design -- system in its context? That's design too.... what does it offer? What "job" does it do? How does it relate to other systems? Outcomes -- capabilities and properties; Trans contextual

Does architect do this? In building architecture but not generally in software....What do we gain when architect is involved? Feasibility/viable/desirable -- customer/technology/business(ecosysyems; value flows; differentiation)
Better decisions -- product/outward design and system/internal design; innovation

Evolutionary landscape -- capabilities evolve; landscapes are co-evolving; shifts in one can send ripples and waves of change; smart phones and uber; what does machine learning/ai unleash?

3x explore expand extract
Pioneer settler city planner
Ecosystems that coevolve;

Throw away good ideas – p28, p26 of 101 Things

Say No – steve jobs

"#Microservice architecture moves complexity to the area we are good at automating: operations“
https://www.oreilly.com/ideas/microservices-shift-complexity-to-where-it-belongs

Chad Fowler https://twitter.com/chadfowler/status/646624348028190720

what will break in production? break code? break user experience?
make viable, feasible, desirable

Make -- pressure to make it can warp integrity

Uber - gaming the ecosystem in some sinister ways
But can be strategic and steer away from doing active harm -- ecosystem and notions of commons
Win by any means -- breaking laws; intentionally misleading/betraying public trust; -- ecosystem health is not optional
Study ecosystem for what need to put in place to make system viable/healthy; some of that is business relationships; some is technology ... good to have architects as technical strategists and designers involved

So architect has a foot in where system is and how to do that better... business and tech... strategy and tactics; culture snd conversation/attention

And from time to time... heads up? What's changing? What does that mean for system?

So how about those emoluments?? :)

Fractals and scopes of influence and "authority" (feather)

Kodak -- booch -- future depends on people in this room
We build it -- the future, that is
We better get more strategic about it! The people we lead are responsible for all the good they build, but we, leaders, shape direction/enable and constrain... so we need to feel responsible, to feel responsible for, where that goes...

What is better? How do we get more of that?
What is integrity? Structural and design?
Sustainability ... in all its forms

Visual Architecting