1.
I came to information architecture from concepts I encountered in grad school in
the late 90s: Vannevar Bush and Doug Engelbart instrumenting and visualizing
human associativity, hypertext theory, information visualization, information
seeking behavior. That was where I first realized that information can have
structure, and spatial and perceptual qualities, and behavior. After grad school, I
got a job as an information architect in an agency setting and went to the first IA
Summit. I’m wafting some of the concepts from Lou Rosenfeld’s talk that are still
so relevant today. I went to the second Summit, then contributed a paper to the
JASIST special topic issue on IA in 2002. I was very interested in the notion that IA
was a new field without its own internal body of theory. What did that mean? What
was that like? Eleven years flashed before my eyes. I finally went back to the IA
Summit in 2013 to find IA described as, “The structural integrity of meaning across
contexts,” by Jorge Arango. I was first introduced to the notion of embodied
cognition from Andrew Hinton’s talk. Last year, I brought a poster on data
visualization techniques. And here we are: IAS16. I recently joined Autodesk,
helping mechanical designers collaborate around 3D geometry for product design.

2.
If Information Architecture worries about the structural integrity of meaning
across contexts, then the spirit of this talk is to zoom in and really look at the
nature of information and the nature of meaning to inform our work.

3.
If Information Architecture worries about the structural integrity of
meaning across contexts, then the spirit of this talk is to zoom in and
really look at the nature of information and the nature of meaning to
inform our work.

4.
We will see that if we look at these things through the lens of embodied
cognition, we see structural properties of IA that we couldn’t see before
looking only from traditional cognition.

5.
Before we get to the new structural properties, we will first visualize the
nature of meaning. To visualize the nature of meaning, we’ll build a scene.
And this scene starts with the sun.
Image credit: NASA/SDO

6.
Let’s add to our scene a tree in nature, and a built chair.

7.
The Sun gives radiant light that shines on all the things.

8.
All the things reflect the light. That’s what we experience as ambient light.

9.
Or what James J. Gibson, the founder of Ecological Psychology back
in the 1960s, calls an ambient energy array.

10.
In this ambient energy array, we detect surfaces and edges and
textures. But the light shifts, we move around objects, objects move
around us: it’s not these surfaces, edges, textures themselves that we
pick up, but the relationships among them. These relationships are
invariant structure. And it’s the invariant structure that we detect,
regardless of our perspective.

11.
This invariant structure is information. That’s what information *is*.
And it’s already out there in the environment, we don’t need our
brains to do any special processing.

12.
If we think of a chair, we don’t need to have seen the chair from every
possible perspective to recognize it. We pick up the invariant
relationships among its edges and surface and textures that let us
understand the seat and the back and the legs and its the same chair
we’ve seen before.

13.
Let’s add to our scene an actor-observer (NOTE: this actor-observer
is based on the observer in JJ Gibson’s 1960’s diagrams from his
book, An Ecological Approach to Visual Perception).

14.
As we’ve seen, objects give information in the form of invariant
structure in relationships among surfaces, edges, and textures.

15.
Our actor-observer brings to this her goals...

16.
her actions...

17.
And it is the confluence of goals...

18.
Actions...

19.
And information...

20.
Where meaning emerges. Meaning emerges in this confluence.

21.
We’ve considered information about objects we see visually in the
form of invariant relationships among surfaces, edges, and
textures. There are other types of information in the ambient
energy array.

22.
The layout of a collection of objects gives information about way finding.

23.
We humans are wired for mechanical information: touch, cold, warm, pain.

24.
And chemical information: taste and scent.

25.
And these are all perceptual information types.

26.
We also have linguistic information.

27.
Words on a surface; physical or digital overlays.

28.
Words through the air; spoken or projected.

29.
Gestures evoking concepts.

30.
Even introspection in our heads. All of this makes up the ambient
energy array, the *information* in our environment.

31.
We’ve looked at where meaning emerges, now let’s zoom in to look
at the mechanism that let’s us interact with information in the
environment. We’ll start with perceptual information. For perceptual
information, the invariant structure, the information, is in the form of
affordances: what we can do to engage with objects. When we
engage with affordances for perceptual information, we form what is
called an action-perception coupling.

32.
As an example of a perception-action coupling, we’ll look at The Outfielder
Problem. How does a baseball outfielder know where to go to catch a fly ball?
Traditional cognition suggests the outfield sees the initial trajectory, figures
out where the ball is going to land, and runs to that spot. But that’s not what
happens. The outfielder actually forms a perception-action coupling with
angle relationships of the ball.
The outfielder problem described in, Wilson, A. & Golanka, S. (2013).
Embodied cognition is not what you think it is. Frontiers in Cognitive
Psychology.

33.
If the ball starts angling to one side, the outfielder shifts to remove
the angle. The perception-action coupling is simply: eliminate
horizontal angles.
The outfielder problem described in, Wilson, A. & Golanka, S. (2013).
Embodied cognition is not what you think it is. Frontiers in Cognitive
Psychology.

34.
In this way, outfielders make a series of course corrections to
maintain the angle relationship and appear to drift to end up in
the right spot to catch the ball.
(There is a perception-action coupling the outfielder forms with
vertical geometry of the ball too, but for simplicity, we are
looking at the horizontal coupling.)
The outfielder problem described in, Wilson, A. & Golanka, S.
(2013). Embodied cognition is not what you think it is. Frontiers
in Cognitive Psychology.

35.
Language is different. Light doesn’t reflect off of concepts. What is that
handle, that aspect of the semantics of a concept that allows us to
engage its meaning? We don’t have a word for that. It doesn’t fit the
strict definition of affordance. That’s something still under debate in
the embodied cognitive psychology community. Perhaps we can
contribute in that area.
Sabrina Golanka, an embodied cognitive psychologist, rolls both of
these up to say we form an information-behavior coupling to cover how
we engage both types of information.
See http://psychsciencenotes.blogspot.com/2014/06/a-gibsonian-
analysis-of-linguistic.html for Sabrina Golanka’s notion of an
information-behavior coupling (and forthcoming paper).

36.
An information-behavior coupling is how and where meaning
emerges. But, these things aren’t frozen in place.

37.
Our goals-directed actions and aspects of the environment are
changing, and if we’re going to maintain this information-behavior
coupling, these things must co-evolve.

38.
In fact, we can say that human cognition is the state space of
information-behavior couplings that form, break, co-evolve with
goal-directed action and environment dynamics.

39.
But, what does all this flux and co-evolution mean about meaning? The nature of meaning is flow.

40.
Flows have properties. Meaning has viscosity, or ease of flow. Meaning has
texture depending on what facets of information are participating in the
flow; meaning is subject to permeability in what it’s flowing through. We’ll
look at each of these in turn to see how we may use designed structures to
dial them up or down depending on our needs.

41.
First, we’ll look at viscosity, or the ease of the flow of meaning. I’m going to
introduce a new IA construct to account for viscosity in the flow of meaning.

42.
To do this, I want to ask the question: is information like water?
Image credit: Impact of a drop of water on a water surface, by Roger McLassus
https://commons.wikimedia.org/wiki/File:2006-01-28_Drop-impact.jpg

43.
If we run all the permutations on the different relative amounts of
temperature and pressure, we end up with what we know as the phase-
space of water. The nature of water is drastically different as a solid vs.
liquid or gas. We can phase-shift a solid by melting it to a liquid, or freeze
a liquid into a solid.

44.
I am asking the question, what if we do the same thing for the two types of
information: perceptual and linguistic? What is it like to interact with different
combinations of perceptual and linguistic information? Do we find
neighborhoods of combinations in which it is drastically different to engage
with the meaning of information?
Phase-space from Haverty, M (forthcoming). Meaning as flow: structural
properties of information architecture, Journal of Information Architecture.

45.
Perceptual information is tacit and reflexive. Once we’ve learned to detect it, we don’t have think about it
to engage with it. Perception flows easy. It has a low viscosity, like water.
Perceptual information is tacit and reflexive. Once we’ve learned
to detect it, we don’t have think about it to engage with it.
Perception flows easy. It has a low viscosity, like water.

46.
Language is laden with awareness and associativity, and requires
attention for us to use it. Language is much more viscous. It takes more
work to flow because we have to think about it, actively attend to it.

47.
Generally, the area dominated by perceptual information is reflexive. The area dominated by
linguistic information is more attentive.
Phase-space from Haverty, Marsha: Meaning as flow: structural properties of information
architecture (forthcoming), Journal of Information Architecture.

48.
But we can get more granular than that. If we have little or no language, all
perceptual information, we are engaging with meaning in a very visceral
manner: mechanical motions and eye movements. Similarly, if we look at the
area with little perceptual information, we are operating in the world of
concepts. We are thinking conceptually to engage meaning. If we have a lot of
language, especially if it’s abstract, which we’ll look at later, we need intense
concentration, in this highest viscosity state. If we have a lot of perceptual
information to deal with, we must engage with intense coordination. And when
we are faced with a lot of information of either type, an emotional response is
often triggered. This isn’t the only place where emotion can occur: we can have
an emotional component anywhere along the phase-space, but the presence of
large amounts of information does tend to evoke emotion. Further, if we have
too much information, we become overloaded and unable to engage well with
meaning.
Phase-space from Haverty, Marsha: Meaning as flow: structural properties of
information architecture (forthcoming), Journal of Information Architecture.

49.
For Twitter, before inline image preview, the information-behavior coupling we
form to engage with meaning in this place was dominated by language. The
nature of the information on Twitter put us in the mode of reading words.

50.
We bring a variety of goals to our engagement with Twitter: the intrigue of coming
across interesting articles or images or thoughts that we wouldn’t have encountered in
another way; humor; discourse sampling (getting a sample of latest news or
conversations among our friends or peers).
The nature of the information we engage on Twitter before image preview was
primarily language. We had some perceptual information in the avatars, but those were
constrained to a consistent spatial position along the periphery and used for occasional
glances for source information.
Engaging with meaning on Twitter is not like reading a novel where you have a thread
of and then this happened and then this happened… Twitter is semantic juxtaposition.
It requires concentration to do all that concept hopping from tweet to unrelated tweet,
even though we’re flowing the same activity: reading. And there’s a lot of it. We got
really good at scanning through the never-ending semantic mashup.
Because of the high-concentration concept hopping, the flow of meaning had a highly
viscous nature.

51.
When Twitter introduced inline images, suddenly our mode of engaging meaning by
scanning words with eye movements was interrupted.
We now have these perceptual swaths interrupting our high-concentration scanning.
We either have to skip over them, or we have to phase-switch from scanning words
to glancing images. We probably don’t even think about the images anymore, we’re
so used to them, but what it’s like to meaningfully engage with Twitter, to form an
information-behavior coupling with Twitter, has phase-shifted. We’re acting-
engaging with meaning in a different phase-space neighborhood. It’s not the same.

52.
This is a visual archive of 5 years of issues of a design blog called
Infosthetics. Each issue is color coded by key categories, and they are
ordered by time, most recent at the top, oldest at the bottom.

53.
When you select a category, in this case Architecture, we get an
immediate visual understanding of how this category spanned the
years of issues. We see some clusters early on at the bottom, and
most recently. We see other categories that co-occurred in the same
issue. We could interact to get details about the concept.

54.
We would locate this visual archive well down in the perceptual
dominated region. Category labels are used as semantic anchors for a
primarily reflexive gleaning of the distribution of topics across time.

55.
In the movie HER, Theodore has a relationship with an AI that manifests solely as
words projected in his ear. He builds a complete relationship made of language.

56.
We would locate the movie HER way up in the intense concentration
neighborhood of the phase-space with virtually no perceptual
information. In a relationship, actively attending is the point. And
Theodore is a writer; he loves words. This mode, this extremely high
viscosity suits him, and is fulfilling for him.

57.
Shortly after this movie came out, Ben Schneiderman said, “The future
of computing will be more visual than verbal. Voice is important for
human relationships, but can’t keep up with the human mind’s desire
for information abundance and swift decisions.”

58.
Our design projects likely fall in this general area. More dominant on
language, but also use perceptual information to help our users make
sense of text; we worry about white space and layout, especially
planning for adapting to different viewport sizes. We infuse our
navigation and function triggers with edges and surfaces and textures.
We craft node-link structures for associative wayfinding.

59.
But because our digital environment can be anywhere, with more language, more
to perceive, our designs are actually phase-shifted to a different neighborhood.

60.
To mitigate this attention overload, our designs are starting to make use
of information at the extremes of the phase space; especially, for
wearable sensor visualizations and Internet of Things displays.

61.
But often, the system status is fully perceptual, and then, as soon as we go to
engage with it, it phase-shifts us to language. That may be appropriate: the
situation may need that level of attention required by forming information-
behavior couplings in the more viscous linguistic mode. But sometimes
language is just too viscous for the situation.

62.
I want to consider an example of fitting the nature of the information-
behavior coupling to the situation. This is a concept for a new way of
designing a car dashboard control panel. Instead of a dashboard of buttons
and labels, it’s just a blank screen. The driver touches the screen anywhere
and the interface is summoned to that spot. Depending on the number of
fingers, the driver is controlling a different aspect of the car: radio, heater,
whatever. The driver simply drags fingers up or down to adjust.

63.
And it’s a forgiving information-behavior coupling. The driver drags the
control up or down, but doesn’t have to be exact: as long as the motion
is generally up or generally down, that’s close enough to maintain the
engagement. The driver is worried about wayfinding, not hitting a cyclist,
talking to a passenger… This is a design and information modality that
respects the high viscosity of the surrounding situation and the need to
phase-shift dashboard interaction to more perceptual and forgiving
coupling. This information structure starts perceptual and stays
perceptual.

64.
Given the pervasive information ecosystem in which we find ourselves,
we need to consider the entire phase-space when we make decisions
about the information in our designs. We need to note 2 phase-space
locations: the design itself, and the design in the greater information
ecosystem. We need to decide when to use the higher viscosity of
language and when to offload some meaning to perception.

65.
The next aspect of the flow of meaning we’ll discuss is texture. The
texture of meaning is to ask what facets of perceptual and linguistic
information are participating in the information in our designs.

66.
We’ll start with linguistic texture facets. Let’s consider some usual IA facets:
controlled vocabulary, faceted classification, taxonomy, ontology, content strategy.

67.
Let’s consider these along the phase-space. Controlled vocabulary is
a high-viscosity state of information as we carefully select among
related labels. Taxonomy infuses concepts with some perceptual
qualities in the form of semantic groupings.

68.
Faceted classification is a berrypicking journey (in the Marcia Bates sense) all
across the language-dominated phase-space. Content strategy (taken as
information that has bound with it instructions for how to shape-shift
(semantically and physically) across viewports and other context factors), is a
mapping walking a harmonious line between language and perception.
NOTE: For more on berrypicking, see: Bates, Marcia (1989). THE DESIGN OF
BROWSING AND BERRYPICKING TECHNIQUES FOR THE ONLINE SEARCH
INTERFACE accessed online: http://pages.gseis.ucla.edu/faculty/bates/
berrypicking.html

69.
Ontology is an information-behavior coupling. Actually, it’s a set of
information-behavior couplings in which the relationships among the
conceptual entities serve as the invariant structure.

70.
Let’s consider another facet of language: where the concept falls on the
concept spectrum, concrete to abstract. A concept is concrete if it has a
physical referent, is spatially constrained, and for which it is easy to
visualize context. The concept “spoon” is very concrete: we may easily
picture a spoon as an object located in a place; we can easily visualize
multiple contexts (a spoon in a bowl, in a drawer, stirring a pot). We’d
locate the concept “spoon” in the still language-dominated region of the
phase-space, but in a region of lower viscosity, and more perceptual
influence.
The concept “calculus” though is highly abstract. We can’t picture calculus
as an object located in a place; we’d have trouble thinking of multiple
contexts for calculus (unless we were mathematicians). It requires highly
conceptual, highly viscous concentration to engage with the meaning of
calculus.

71.
Vannavar Bush, in the 1940s built a machine called the Differential
Analyzer. This machine mechanically performed calculus through
physical movements of levers and rods and gears.
It was said that, “Those who used the Analyzer acquired what
[Bush] called a ‘mechanical calculus,’ an internalized knowledge of
the machine…like a combination of motor memory and
mathematical skill, learned directly from the machine. Bush
described how one user, “did not understand [calculus] in any
formal sense, he understood the fundamentals; he had it under his
skin.” From Belinda Barnet’s history, Memory Machines: The
Evolution of Hypertext.

72.
Bush’s machine phase-shifts the abstraction of “knowing the meaning of
calculus” from a high-viscosity act of conceptual concentration to a purely
visceral understanding of visual and physical relationships. One may later
try to describe or summarize this new understanding using words, but the
meaning itself was already gleaned in a purely perceptual manner. In that
perceptual coupling is where the meaning emerged.

73.
Two other methods for phase-shifting abstract concepts include using
metaphors and context priming. There is a lot written already about
metaphors. Part of what makes abstract concepts abstract is that it’s hard to
think of contexts for them. If we provide context ahead of or along with the
concept, it essentially phase-shifts that concept to a lower region on the
phase-space, requiring less active concentration and lower viscosity to
engage. This type of shift is likely much less extreme than the Differential
Analyzer example, but it can work to dial down the viscosity.

74.
Let’s switch to perceptual texture facets. We don’t all need to become
information visualization specialists, but we should recognize that it is edges,
surfaces, and texture relationships that we detect as information, and we
should consider these aspects and tune them to suit the information objects
we design. Do we need to adjust surface/edge/texture qualities to show our
information objects are… fixed vs. movable? overlapping vs. fused?
NOTE: here are a few excellent references for surface/edge/texture design
qualities:
Semiology of Graphics, Jacques Bertin
Readings in Information Visualization, Stuart Card, Jock Mackinlay, Ben
Schneiderman
Information Visualization: Perception for Design, Colin Ware
Visualization Analysis & Design, Tamara Munzner

75.
With respect to the wayfinding and place element of perception, this territory
is huge and well-covered in past IAS talks and all the IA books out there.

76.
The other aspect of visual information from the ecological psychology point of view is
events. Objects have locomotion and physical transformation, and occlusion (objects
overlap, are hidden, are revealed).
As designers, we have revealed additional meaning to types of events we can instrument in
our information structures. David Kirsh considers the meaningful events we can bring to
external representations. Things like rearrangement, and if we have many instances of the
same representation, we can enact different events on each to explore alternatives.
Karl Fast has a framework of epistemic interactions, or meaningful interactions we can do
with visual representations. Things like: chunking, cloning, collecting, composing, cutting,
fragmenting, probing, rearranging, repicturing. All of these facets are design dials for
meaningful events.
Kirsh, D. (2010). Thinking with external representations. AI & Society.
Fast, K. & Sedig, K. (2010). Interaction and the epistemic potential of digital libraries.
International Journal of Digital Libraries.

77.
Essentially, we can take advantage of the materiality of diagrams (visualizations,
images, even physical models), to enact (and maintain) meaningful events. It’s that
maintain part that’s really important when we think about meaning as flow.

78.
Next I want to talk about some factors that impact the permeability of the flow of
meaning. In our case, permeability relates to how well the actor-observer can
engage with and control the information-behavior coupling.

79.
Some of these permeability factors are familiar things to us and are things we already worry
about, but we need to frame them in terms of embodied cognition and meaning-as-flow.

80.
We can have high permeability, where the flow of meaning is completely unobstructed
all the way down to completely obstructed with no flow, meaning unable to emerge.
Sabrina Golanka has three continuums of information that I’m framing here as
permeability factors. The ability to detect structure: if the actor-observer has not yet
learned to recognize invariant structure, there’s no chance to engage that structure.
Coordinating behavior: if the actor-observer detects structure, but is unable to
coordinate actions to engage appropriately with the structures, the flow can’t happen
either. And it’s a continuum because we can get better by degrees with experience over
time. Structure persistence: if the information is needed to maintain a coupling over
time, but is intermittent or vanishes, the coupling breaks and the flow of meaning
stops.
Permeability factors adapted from Sabrina Golanka’s Information Taxonomy: http://
psychsciencenotes.blogspot.co.uk/2013/03/a-taxonomy-of-information.html

81.
I would like to add another permeability factor: tolerance. How precise must our
behavior be to maintain the coupling? If the coupling has a wide tolerance, it is
forgiving if our actions veer a bit, or the information persistence waivers. If the
coupling has a narrow tolerance, our actions and the information which we are
engaging must be much more precise, or the flow of meaning is interrupted or
stopped.
Tolerance is particularly helpful to consider when we think about the case of the
distracted driver using a dashboard control display or a tiny display on our arms or
in our palms while we’re walking.

82.
When we look through the lens of embodied cognition, we recognize that
our designed information structures participate directly in the information-
behavior couplings that give rise to the flow of meaning. With our design
choices, we can modify what it’s like to engage meaning with our
structures. Because of pervasive digital overlays, we must to consider the
entire phase-space of information in our designs.

83.
At core, we are tribal hunter-gatherer-poets that move in the world and understand with things.
inForm image credit: MIT Media Lab http://tangible.media.mit.edu/project/inform/
meadow image credit: Nicholas Tonelli via flickr