LASTconf 2018 - System Mapping: Discover, Communicate and Explore the Real Complexity of Your Systems

System Mapping
Discover, Explore & Communicate the Real Complexity
of Your Systems
Colin Panisset @nonspecialist

@nonspecialist
Real-time Feedback
https://is.gd/last18realtime
Law of 2 feet

@nonspecialist
Runsheet (ish)
● What even are we on about here?
● Why on earth would you do this thing?
● Ok, I’m listening, how does it work?
● Show me some examples!
● I’m here for the shiny tools
● That’s lovely, tell me about some real times it helped
● Where are the tricky bits?
● I’m convinced; tell me more!

What even are we
on about here?
If this turns out to be a
presentation for a timeshare, I’ll
be quite put out
“The map is not the territory”
Alfred Korzybski

@nonspecialist
System Mapping: what is it?
A way to reveal hidden complexity
Communication and collaboration tool
Discovery
Sharing knowledge within and across teams
Documentation
Build a common understanding
Maps as Code (the why of this later)

@nonspecialist
Example 1 - microservice dependencies

@nonspecialist
Example 2 - software change flows

@nonspecialist
Example 3
Team workflow

Why on earth
would you do this
thing?
I’m still not convinced this isn’t
about timeshares
“Ask ‘why’ five times about every
matter”
Taiichi Ohno

@nonspecialist
Why to Map?
Technology can reveal some things; e.g. distributed tracing,
but:
Takes time to implement
Requires support in software
Doesn’t cross non-technical boundaries
Can miss infrequent/unlikely connections
Only maps data flows, not process flows

@nonspecialist
Why to Map 2?
As much about the journey as the destination
Build shared understanding of the system
Not just discover data dependencies -- map change flows, or
team workflows … process mapping as well

@nonspecialist
Why to Map 3?
Discover, describe and visualise the complexity:
● show points of coupling
● bottlenecks
Discover, describe and visualise the risks:
● Dependencies with specific names, like Emma or Nathan
● Flows of data/communication with fragile path
components

@nonspecialist
Why to Map 4?
Extract and capture the hidden oral tradition; see where the
bodies are buried.
I’ve never created a map that hasn’t had someone in a team go
“wait, I had no idea that was there” or “I thought we
decommissioned that?”
Brings clarity and common understanding to great complexity

@nonspecialist
Why to Map 5?
You can map both technical and non-technical things in a
manner which is accessible, comprehensible, relevant and
informative for all sorts of people in an organisation.
Maps are far more useful for quick “at a glance”
understanding
Maps work better for people who are visual learners
The IKEA principle: collaborative mapping builds collective
ownership of the system

OK, I’m listening,
how does it work?
You aren’t trying to sell me a tool
or a package or something, are
you?
“I always want to know how things
work. Had I been Aladdin, I am
certain that just after one wish or
two, I'd have taken that old lamp
apart to see if I could make another,
better lamp.”
Walter P. Chrysler

@nonspecialist
Doing the do
Address the elephant in the room: “it can’t work in my
organisation, because … “
Start anywhere
Ask people to tell you what they know
Capture it
Get feedback and corrections
Rinse and repeat

@nonspecialist
Practical tips
Small groups and teams work better to fill in specific domains
Print it out and hang it in a high-traffic area to generate buzz
Hang it next to a sign that says “Probably wrong, please fix”
with a pen on a string
Collect it after a week (or two, or a day), incorporate the
hand-drawn bits, print and re-hang
Finding the right level of granularity can be hard; experience
helps, but refactoring is OK as well

@nonspecialist
Practical tips 2
The “right level” of granularity depends on context; in
extreme cases, printing on A0 paper so the fonts are still
legible has been known
Use of colour and shape is critical: node fill colour is most
important, then line (edge) colour, then boundary colour (you
might use that to indicate ownership)

Show me some
examples!
The plural of “anecdote” isn’t
“data”
“The greatest value of a picture is
when it forces us to notice what we
never expected to see”
John Tukey

@nonspecialist
A worked example

I’m here for the
shiny tools
See! I knew you were trying to
sell me a tool!
“The alchemists in their search for
gold discovered many other things of
greater value.”
Schopenhauer

@nonspecialist
Handy Tools
I use all open source (see, not trying to sell anything):
Graphviz
Git
xdot
Graphviz “dot” language is easy to start with
Complexity can grow rapidly
The “random” layout is a feature
Other tools have similar capabilities but UI-based ones tend
to make it hard to share as code

@nonspecialist
Why Graphviz?
It’s how I started out (so there’s an inertia, for me)
Graph as code allows you to generate different
views/visualisations from the same set of relationships
Its history, the fact that it’s code, and its (moderate) barrier to
entry actually makes it more acceptable to highly-technical
individuals and teams (leaps over the technical elitism bar)
There are some good web-based versions too

@nonspecialist
Example 1a - dependencies (alt view)

@nonspecialist
Example 1a - dependencies (alt view)!!!

@nonspecialist
Toolchain suggestions
Your Favourite Editor (YFE) -> graphviz -> git -> build pipeline
-> PNG/SVG -> static website
YFE -> graphviz -> xdot (for team interaction)

@nonspecialist
Why not Graphviz?
Many other potential tools for capturing this kind of
information; so long as they can record arbitrary
relationships, and you can focus on the connections instead of
where the things are in the map. However, might not be able
to use different visualisations of the relationships
Some examples:
https://conceptmap.io
https://www.lucidchart.com/pages/examples/concept-maps
(likely many more, too … just none that are quite the same)

That’s lovely, tell
me about some
real times it
helped
This is the point of the talk where
I’d like to hear some horror
stories
“Those who do not remember the
past are condemned to repeat it.”
George Santayana

@nonspecialist
Real world stories
Finding the people in the system (billing system A, payroll
system B)
Discovery of a dependency: the system that nearly got away
Visualisation wins: why we can’t have Nice Things

@nonspecialist
Why we (couldn’t) have Nice Things

Where are the
tricky bits?
There are always tricky bits
“Stamping out fires is a lot of fun, but
it is only putting things back the way
they were.”
W. Edwards Deming

@nonspecialist
Caveats
Needs a committed curator
Takes time -- definitely not an overnight fix (but what is?)
Point-in-time snapshot; keeping updated often hard (though
arguably it’s served its purpose by then)
Tooling isn’t hugely user-friendly (it’s not bad but the DSL isn’t
terribly modern)

I’d like to learn
more …
Where does he get all those
wonderful toys?
“Your ideas are intriguing to me and I
wish to subscribe to your
newsletter.”
H. Simpson

@nonspecialist
References
Graphviz: https://www.graphviz.org/
xdot: https://github.com/jrfonseca/xdot.py
Mapping System Couplings and Dependencies for Fun and Profit:
https://cevo.com.au/tools/devops/methods/2016/11/27/mapping-system
-coupling.html
The exercise repo:
https://github.com/cevoaustralia/LASTconf-2018-system-mapping.git
https://dreampuf.github.io/GraphvizOnline/

@nonspecialist
Photo Credits
What is mapping:
https://www.publicdomainpictures.net/en/view-image.php?image=151698&picture=ancient-world-
map-from-1689
Why map 1: https://www.flickr.com/photos/usmcarchives/6049891451
Why map 2: Photo by Bobby Stevenson on Unsplash
Why map 3: Photo by Juan Cruz Mountford on Unsplash
Why map 4: Photo by Linus Sandvide on Unsplash
Why map 5: Photo by Ben White on Unsplash
Doing the do: Photo by Hans-Peter Gauster on Unsplash
Practical tips: Photo by hao wang on Unsplash
Practical tips 2: Photo by Shaouraav Shreshtha on Unsplash
Handy tools: Photo by Barn Images on Unsplash
Why (and why not) graphviz: Photo by Chanakya on Unsplash
Toolchain suggestions: Photo by Fré Sonneveld on Unsplash
Real world stories: Photo by lee dahye on Unsplash
Caveats: Photo by Ricardo Rocha on Unsplash
References: Photo by Giammarco Boscaro on Unsplash
Photo credits: Photo by Mahir Uysal on Unsplash

Thank you!
Colin Panisset @nonspecialist

LASTconf 2018 - System Mapping: Discover, Communicate and Explore the Real Complexity of Your Systems

Recommended

Recommended

More Related Content

Similar to LASTconf 2018 - System Mapping: Discover, Communicate and Explore the Real Complexity of Your Systems

Similar to LASTconf 2018 - System Mapping: Discover, Communicate and Explore the Real Complexity of Your Systems (20)

Recently uploaded

Recently uploaded (20)

LASTconf 2018 - System Mapping: Discover, Communicate and Explore the Real Complexity of Your Systems