Adapted from a presentation to the staff of ICPSR (the Inter-university Consortium for Political and Social Research), Ann Arbor, Michigan, October 28, 2009.Welcome!This talk is aboutexplanatory diagrams, and in particular about process maps.On the way we’ll talk about complexity and change; systems and processes; mapping and insight.
Processes and SystemsWhat’s a “process”? What’s a “system”?A process is how something interesting happens — how we decide, build, deliver, or fix something, or how we see something cycle in nature. And we call the target of the process, if it’s at all complicated, a “system”. That is:A system, according to the mathematicians, is a set of objects and the relationships between them. But more familiarly, we use the word “system” to mean:A computer; a computer’s operating system software; a set of computers operating together for a purpose, as to deliver a service.A set of people and machines operating together for a purpose. This can be very large; it can involve multiple organizations. As in: “The US Health Care System.”The ubiquitous, impersonal, entrenched juggernaut of modern systems-driven civilization. As in: “You can’t fight/change/beat The System.” The ultimate scapegoat.Processes and systems are intertwined. It takes a process to create a system, operate it, maintain it, change it. And we create systems to implement processes.
Mapping and InsightWhat is “mapping”? What is “insight”?As we strive to understand something complicated, we create visual representations (maps) in order to utilize the mind’s built-in facilities for spatial perception and reasoning. We become familiar with the landscape’s geography; we orient ourselves in the terrain and navigate over it; and as we explore the space, we discover patterns and connections that help us explain and predict and explore further. We call these discoveries insights – literally, inner visual experiences.Hence the activity of mapping and the experience of Insight are closely connected. Thus: driven by our craving for insight into the processes and systems that surround us, we resort to drawing pictures. Or we don’t, and we should!
Why We DrawPeople draw explanatory diagrams for many purposes, like: invention; problem-solving; training; persuasion; fulfillment of project management certification requirements. An important purpose that sometimes gets lost is: to give people who work together a clear and easily shared understanding of the complex, evolving systems they grapple with.If you’d like a pithy couple of words to summarize this, try: shareable insight. — §§§ —Many people want to draw pictures at work, yet shy away from it. Why?
Why We Don’t Draw, Even When We Want To“Drawing pictures at work” is reminiscent of “drawing pictures in class” which, we were once taught, is “something you shouldn’t be doing”. As in: “This isn’t art class; you’re supposed to be spending your time doing serious work here.”
Why We Don’t Draw, Even When We Want ToOther common reasons people don’t draw pictures at work:”I can’t draw.” A self-fulfilling prophecy. Translation: “I’m not Rembrandt, therefore I shouldn’t try my hand at a simple business diagram.” That’s like saying: “I’m not a professional race car driver or a jet fighter pilot, therefore I shouldn’t try driving to work.”“Easy-to-draw pictures are useless, but useful pictures are too hard to draw.” Sometimes the diagrams we need most are the hardest to draw and the hardest to maintain. We’ll deal with this shortly.“Drawing at work means drawing … flowcharts. Ugh! Ick! Boring!” Well, maybe, maybe not. We’ll deal with this shortly.“Isn’t the written documentation enough?” I’m all for good documentation, but no, often it’s not enough, when there’s a recurring need for people to grasp complex information quickly, like when systems break down and you have a role in fixing them or working around them.
Here’s a small but real-life example to illustrate the value of a diagram – and not just a gratuitous diagram for the sake of diagramming, but a helpful diagram with some clear thinking behind it.This example comes from the Gartner Group (www.gartner.com), a research and consulting firm that specializes in tracking and forecasting technology trends. A few years ago the Gartner people set forth a model to describe how the market’s awareness and perception of a new technology evolves as the technology proceeds through a life cycle. The market’s expectations of the technology goes through a process with 5 phases, listed here.You can imagine someone in the Gartner Group writing up their weighty white paper about this and saying “Hey, let’s add a diagram!” and coming up with something like this … [next slide]
Diagrams for the Sake of Diagramming: Not UsefulThe impulse to add a diagram is a good one – it’s a signal that you care about conveying your story to your audience, and that you anticipate your audience might get lost in page after page of unrelenting technical text.However, does a diagram like this add any value? No. Well, maybe there’s a little bit of value in visually breaking up the page. But we should be able to do better than that. And the Gartner folks did.
Plotting a Process as a Path Through Space Across TimeGartner came up with something a lot more interesting – a graph of how society’s expectations of the technology vary over time.This is a much clearer and quicker way to grasp how this process happens, and what the 5 phases actually mean. — §§§ —But here’s something that might strike you. Does this curve look at all familiar? The shape of the path itself, separate from the names of the axes and phases and the whole story behind it?
GraphicDéja VuHere’s another graph, one you might imagine should be totally unrelated to the curve of technology expectations: the change of voltage in a nerve cell as it transmits a signal. (This is what you see displayed on a hospital heart monitor.)Notice the resemblance? There is an initial triggering event; a loud rush in one direction; a turnaround and a loud rush in the reverse direction, to an excessive degree; a period of gradual quiet recovery; finally, stability.So here we have a process in mass social psychology mimicking a process in cellular biology. Is this deliciously spooky or what? Whatever this resemblance turns out to mean (which should be a story for another day), I offer it here as an example of a diagram-driven insight – a recognition of a pattern, a connection, a clue to a piece of strange logic in the universe.And now – please be honest here – if you had happened to read some text about technology market expectations and, at some other time, some text about nerve signal propagation, would you have seen the connection between these two phenomena?So this is a wonderful reason to spend time diagramming complex systems and processes: the quest for insight.
This leads us to a common problem: a predictable mismatch between the diagrams we do make and those we should make.We tend to diagram systems and processes that are simple and stable. When the systems evolve, the diagrams need updating, and we wonder whether it’s worth the effort.When the systems become complex, the diagramming effort becomes more challenging, and we wonder whether it’s worth the effort.When the systems evolve and become complex, we stop thinking about even trying to diagram them. But: “complex and evolving” is where we need good diagrams the most! This is where all the action is taking place; where we experience the greatest frustration and greatest opportunity; where we are most in need of shared clear understanding.I believe that the solution to this problem is not to shy away from diagramming what is complex and evolving, but to get really good at it. Then the time and effort to draw the diagrams we really need to draw shrinks from impractical to practical. — §§§ —Now if you are not yet convinced of the power and necessity of diagramming complex phenomena, allow me to appeal to several world-recognized authorities [next slides].
Sometimes it just takes a diagram to make things clear.If you find yourself explaining something over and over, waving your hands in the air, perhaps you could use a good diagram! The hand-waving suggests that you already see, at least in part, the diagram you want to draw. Then the trick is to draw what you see.
If you find yourself getting lost in a discussion that flits between past and present and future, or back and forth among multiple scenarios, or over a tangled web of real or potential cause and effect -- it’s time for a diagram to untangle it all.
Truly great technical writing can bring complex phenomena to the mind’s eye of a savvy reader. But truly great technical writing (like truly great anything) takes time and effort and ability; why not apply those assets to producing a diagram first, and let your readers see what you’re talking about instead of having to rely on their imagination?
A good diagram sets up an imaginary “space” that is populated by the concepts, instances of concepts, categories, relationships, terminology that you want to convey – which you can do by “walking through” the space with your audience.
Now we will focus on a particular class of explanatory diagram: theprocess map, a visual representation of how something interesting happens.
Traditional FlowchartsWhen I say “process map” here is what most people tend to think of: a typical traditional flowchart, with boxes and arrows depicting a sequence of actions and decisions.You see flowcharts depicting medical triage, differential diagnosis, computational algorithms, and call-prompter scripts.Here’s what’s great about traditional flowcharting:It gives you logical building blocks, and rules for putting them together; a visual vocabulary for structured, linear process thinking.It maps well to and from procedural programming languages, with control structures like branching and looping.
Standard Flowchart Symbols from the Information Technology WorldThere are ANSI and ISO standards for the visual vocabulary of traditional flowcharts.See:International Organisation for Standardisation (ISO), ISO 5807, “Information processing -- Documentation symbols and conventions; program and system flowcharts”.American National Standard, ANSI X3.6-1970, “Flowchart Symbols and their Usage in Information Processing”.
Handy Aids for Drawing Traditional FlowchartsCan’t draw? (Or don’t think you can draw?) No problem -- you can buy stencils with standard flowchart shapes.
Automated Aids for Drawing FlowchartsIf you prefer drawing with a computer to drawing by hand, you can find many software packages that can help you draw flowcharts. In fact, you might already have one: the drawing tools in Microsoft Office products (Word, Excel, Powerpoint) contain a palette of flowchart symbols.You can also draw flowcharts using other vector software packages with similar building-block tools, such as Visio. Some software packages offer additional helpful features, like “smart connectors” – lines that attach boxes and stay attached even when you move the boxes.There are also software packages such as those for computer-aided software engineering (“CASE)”) that can generate a flowchart based on a project management model, or a software design model, or a scan of existing software source code. In these situations, it’s the package (rather than you) that decides how to arrange the flowchart symbols on a page (for better or worse).
Flowcharts Have Their LimitsThere is a predictable problem with flowcharts that real people create in the real world: they don’t scale well. As the number of process steps increases, a flowchart becomes increasingly unwieldy. That is: as the flowchart page begins to fill up with steps, the flowchart developer often tries to cram them all in by twisting the path back and forth. The result: it takes the reader increasing effort to grasp where the path of logic begins, where it ends, and how it gets there.Here is an actual published example of a flowchart at the beginning stage of a descent into overloaded visual confusion. What might have originally been a clear straight line of logic is now twisted into a serpentine path.
Here’s an example of a flowchart at a later stage of cramming. Without the “you are here” sign, there is no way to discern where to begin; and the multiply looping and branching path has no overall shape for the eye to grasp.How to Cope?You can try spreading the flowchart across multiple pages. The downside is the hassle of trimming and taping an array of pages together.You can print the flowchart on bigger paper. This might mean spending money at a print shop, or on a large-format printer of your own.
However, eventually any diagram resulting from system / process detective work tends to grow to fill any alloted space, and then you’ll find yourself once again tempted to fold and twist the path to accommodate additional detail.Eventually your flowchart might look like a work of space-filling cleverness to you, but to the reader it will look like a maze.So ask yourself: is your goal to cram everything you learn into the page,or is it to provide clear understanding? Is your goal focused on you, or on the reader? (Hint: Don’t forget yourself, but focus your goal on the reader.)
Here’s the next typical attempt to cope with a flowchart effort that is growing unwieldy: “I know – we’ll buy some software that generates a flowchart automatically!”Alas, a typical large flowchart, especially one generated by a software package, comes to resemble … what? Come on, you can guess this … [next slide]
Yes, exactly – a tangled mass of spaghetti.Sometimes the scary spaghetti-ishness of a flowchart accurately reflects a process whose logic is truly that massive and that tangled. But often it is a clue that the diagram designer needs to impose some additional structure on the description of the process (which might be well to feed back into the process itself).For example … [next slide]
People who draw complex flowcharts try various methods to untangle the spaghetti, to impart some visual order to the chaos.For example: one variation on the traditional flowchart, popular with business process analysts, is the “Rummler-Brache diagram”, more commonly known as the “swimlane” diagram.With this method we give a “lane” to each actor in the process — as in, each department in the organization that has a role in the workflow— and then we lay out each process step in its proper lane according to who performs the step.The lanes can be horizontal (as in this example) or vertical.My observation: swimlane flowcharts (like any method) look great in textbooks, for small and well-chosen examples; but in real life they become huge and sparse and unwieldy and still prone to spaghetti-ishness.So: I contend that truly comprehensible diagrams of real-life complex processes require real alternatives to traditional flowcharts, and often require creative hand-crafting to exploit aspects of the particular process.
Here’s a wonderful example of a creative alternative to a standard flowchart: a process mapping style, pioneered by British designer Martin Kay, patterned after the London Underground subway maps.It is simple, clear, engaging, familiar to its (British) audience. Kay has defined an explicit visual process vocabulary, but to its audience it is self-explanatory. The process is like a train ride, with every step a station.This style addresses another problem with traditional flowcharts: now, decades after their inception, they are, for most of us, visually boring. It is a shame to see an inherently interesting project become visually reduced to plain boxes and arrows.So: Unless your audience loves or otherwise demands traditional flowcharts, don’t limit yourself to them.
So far what I’ve shown are variations on flowcharts, which model a person or machine that actively follows steps and makes decisions in fixed procedural sequences connected by control structures.Here’s an altogether different way to describe how something works:a “state diagram”, which shows a series of states and events that trigger transitions from one state to another.A state diagram models a person or machine that waits for events to happen and then responds to each event with a change of state.For example, you might model a media player as a machine with states like “Playing”, “Rewinding”, “Fast-Forwarding”, “Stopped at Beginning”, “Stopped in Middle”, “Stopped at End”, and events like “Play button pushed” and “End of media reached”.Diagramming the states let you see and fill in potential holes in the model; you can develop the diagram until it pins down what should happen for each possible combination of a state and an event.
Here’s an example from biology: a cell goes through various states in its life cycle. A sequence of cellular events (like the division of the nucleus) takes the cell from one state to the next.
Here at ICPSR, I used a state diagram to pin down the metadata workflow process that the Metadata Editor software and database should support.
Here’s another way to describe how something happens: a “Data Flow Diagram” that shows how material and/or information flows from one process to another.This deliberately does not show time, or decisions, or sequence, or any procedural logic. It is great for figuring out how to partition a big complicated process into smaller, more tractable processes.
Here’s an example of a Data Flow Diagram for a bookselling operation. You can see three processes (like “Receive order”) exchanging information and material with each other and with internal and external entitiies.Notice that although the process bubbles are numbered (1, 2, 3), this is not a strict sequence of operation as one would see in a flowchart. For instance, the store might collect payment [with bubble 3] before or after shipping a book [with bubble 2]. The purpose of the numbers is for easy reference by other diagrams and documents, and their order is likely the diagram developer’s choice based on the easiest way to tell the story.
Here is what the ICPSR pipeline process might look like as a “classic” Data Flow Diagram.
Now, Take a Walk on the Wild SideWhat if you’re feeling constrained or befuddled by all of these formal mapping methods, with their symbols and definitions and rules? What if you’re just trying to bring ideas to the surface and lay them out for all to see? And it’s not yet clear what form it should take?Here’s another way to draw how things happen: a “mind map”.This is a great for brainstorming – for putting to paper everything you (or a group) can bring to mind about a single topic.This style of mind map has been popularized by Tony Buzan, who touts the value of appealing to the human right brain’s spatial cognition abilities versus the standard flowchart’s appeal to the left brain’s logical abilities.
You can now even find graphic software packages just for mind mapping. Here is a Mind Map drawnwith the aid of a graphics program.
Here’s an example of what a mind map might turn into – a way to tell a complex story. What do you think of this diagram?Does this work for you?Although it looks daunting at first, if you wander through it close0up you can see that the artist has used color and typography to assemble many individually coherent stories and then link them together.
Here’s another example of a complex diagram that appears intended to tell a complex story: the “Organizational Chart of the House Democrats’ Health Plan”.What do you think of this diagram? Does this one work for you?Do you feel any different about this one compared to the one on the previous slide? Regardless of where you stand on the contentious issue of national health care overhaul, I think you can agree that this diagram does not appear to be designed to clarify. As it happens, it was drawn by House Republican staff.It appears built to terrify rather than inform; to deliver a single powerful message, something like “scream and run away as fast as you can”.
Is there a “best” style of process mapping?I believe that there is no single “best” style of process mapping that serves well in all situations – but for each particular situation, it is possible, permissible, and advisable to choose or devise a diagramming style that best tells The Story or answers The Question.How does one choose a diagramming style at the beginning of a project?I’ll confess that I usually don’t.Since my overall goal is to promote shared understanding, rather than to satisfy a rigid requirement of methodology:I begin a diagramming project with a very informal draft (on a whiteboard or napkin).I add formal structures gradually as I take the diagram through many iterations, and useful structures suggest themselves.I end up with a diagram that is not in a single pure formal style, but draws from more than one formal style and even retains some informality where that adds truth and/or visual appeal.
For example: with the ICPSR pipeline process, a style that made sense to me was a data flow diagram with additional labelling to show process ownership and sequence relationships.
How to construct a really fine process diagram?Regardless of the style of process map, I consider that it takes six activities to produce one. Only one of these activities is actually drawing.Focus on an articulated purpose for your diagramming project. It might sound like a boundary, or a motivating question, or a story you’d like to tell.Learn about the process from sources. Ask, listen, read, watch, discuss, until a picture comes to mind.Draw what you see.Test what you draw against your sources; against rules of logic; and against reality.Show what you draw to a larger audience. The drawing should support you in telling the story. This too is a kind of test.And finally, hopefully:Act: Let what you draw lead to some larger helpful action. Let a big question be answered, a troubling problem solved, a complex system improved or replaced.
Each of these activities does not necessarily require a separate person, but they do require a separate mindset. I describe these mindsets as though they were professional roles, like “detective” or “artist”.Each mindset has its own sense of what’s important, and (with time and practice) its own body of know-how.In which of these roles do you already most comfortable, most knowledgeable? And in which are you least comfortable, least knowledgeable? I’ll suggest that you might stretch yourself to give those latter roles a try, perhaps with some partnership and/or coaching assistance.
In any case: if you have wanted to draw more pictures at work, or make your pictures more clear and useful, I hope that our time together today has given you a bit of inspiration to give it a try!
And finally, I hope you get to draw all the pictures you want to draw to tell all the stories you want to tell. Please let me know if you have any questions, or if I can assist.At your service,— Cole —
Process Mapping, System Insight
Process Mapping<br />System Insight<br />Managing complexity and change at work<br />with explanatory diagrams<br />Cole Whiteman<br />email@example.com<br />
Why We Draw Pictures at Work<br />To give people who work together a clear, easily shared understanding of the complex, evolving systems and processes they grapple with.<br />To:<br /><ul><li>Invent
“Isn’t writing enough?”</li></li></ul><li>Sometimes Writing is Not Enough<br />Gartner’s Model of the Process of Technology Creation, Adoption, Application, and Maturity<br />Example: <br />Technology trigger<br />Inflated expectations<br />Disillusionment<br />Enlightenment<br />Productivity<br />
Words in Boxes are Not Enough<br />“Hey, let’s add a diagram!” <br />Technology Trigger<br />Inflated Expectations<br />Disillusionment<br />Enlightenment<br />Productivity<br />... A good impulse, but zero added value<br />
Sometimes a Process has a Shape<br />The Gartner Hype Curve<br />Peak of Inflated Expectations<br />Expectations<br />Plateau of Productivity<br />Technology Trigger<br />Slope of Enlightenment<br />Trough of Disillusionment<br />Time<br />
Sometimes a Process Shape is Recognizable<br />Nerve Signal Propagation<br />+40<br />Action Potential<br />Voltage (mV)<br />0<br />-55<br />-70<br />Resting State<br />Stimulus<br />Refractory Period<br />Time (msec)<br />
What We Diagram … and Don’t Diagram<br />Simple<br />Complex<br />Hard<br />Easy<br />Stable<br />But:<br />we need this the most!<br />Seems like wasted effort<br />Don’t even think about trying this<br />Evolving<br />