This document discusses synesthesia, a neurological phenomenon where senses are joined so that stimuli from one sense simultaneously evoke sensations in another sense. It provides examples of synesthetes who see letters and numbers as colored or experience tastes as physical touches. While once dismissed, research is now accumulating evidence that synesthesia is a genuine brain phenomenon. This challenges some theories of how the brain works and perception is organized. The document examines what defines synesthesia and explores what it can teach us about the relationship between subjective experience and objective reality in brain science.

1. Sometimes the bizarre is just the bizarre, but when
the anomaly is a product of fundamental brain processes
yet seems to confound them, the theory makers have
by Richard E. Cytowic, M.D.
a problem. For several decades, most neurologists who
heard about synesthetes—people who see colored
letters, feel colored pain, and taste shapes—just
shrugged their shoulders or rolled their eyes.
Touching Tastes,
Seeing Smells—and Shaking Up
Brain Science
Now, as evidence accumulates that synesthesia cherished conceptions of contemporary brain
is a bonafide neurological phenomenon, some are science. The good news is that when old theoret-
asking how this squares with some of the most ical structures fall, new light may flood in.
7

2. Cerebrum
S
ome 20 years ago, my dinner host, person, object, or concept evokes synes-
Michael Watson, delayed our seating thetic sensations.
by announcing, “There aren’t As children, synesthetes are surprised
enough points on the chicken.” He meant to discover that others do not share these
that the taste still failed to evoke the prickly experiences. Often ridiculed and disbelieved,
sensation he sought. To Michael, tastes and they learn to keep their atypical perceptions
smells were also felt as a physical touch in to themselves. Nonetheless, the phenomenon
his face and hands. “With an intense flavor,” remains involuntary and consistent throughout
he tried to explain, “a feeling sweeps down their lives. The trait runs strongly in families,
into my hand, and I feel weight, texture, and the genetics of its inheritance are reason-
shape, and whether it’s hot or cold as if I’m ably well understood. Some type of synesthetic
actually grasping something.” experience occurs in perhaps 1 in 200 individ-
“Ah,” I exclaimed, “You have uals, and more than 75 percent are women.
synesthesia.” Like most anomalies that lie outside the
Michael looked stunned. “You mean explanation of conventional theories, synes-
there’s a name for this thing?” thesia was long dismissed as a mere curiosity
Sharing a Greek root with anesthesia, or, worse, as just subjective imagination. I
which means “no sensation,” synesthesia wondered how the brains of people like
means “joined sensation,” wherein two or Michael Watson might differ from the majori-
more senses are coupled in such a way that ty, but my colleagues refused to accept that
a voice, for example, is not only heard but his experience might be a bona fide neuro-
also felt, seen, or tasted. We call individuals logical phenomenon. Pointing to 200 years of
with this coupling synesthetes.
The process of synesthesia usually As children, synesthetes are
travels in only one direction. For example,
surprised to discover that others do
a sweet taste made Michael feel a cool,
polished, curved surface in his hands, but not share these experiences.
handling a billiard ball would not elicit Often ridiculed and disbelieved,
any flavor in him. About 40 percent of
they learn to keep their atypical
synesthetes have multiple types of synes-
thesia; the couplings that have been perceptions to themselves.
observed by scientists do not include all
possible pairings. Sensing letters, numbers, synesthesia history in the annals of medicine
or words as colored accounts for two- and psychology did not sway them. Today,
thirds of the instances of synesthesia. For however, researchers in some 15 countries
those with “colored hearing,” sounds are studying synesthesia, and many doctoral
evoke colored shapes that arise, move, candidates have chosen it for their theses.
alter, and fade—somewhat like fireworks. If others have gradually come to
For others, merely thinking about a certain accept the reality of synesthesia, they must
8

3. The Dana Forum on Brain Science
now relinquish some received wisdom about regarding sensation and perception based
how the brain works. Our concepts of how on observers’ reports, taking as a given that
things work are but models, after all, reduc- mental states exist. Scientists in the 20th
tions of reality that arise from human minds; century, however, consistently strove to
history has shown repeatedly that reality eliminate the subjective role of a human
has a way of making a mess of neat and tidy observer in gathering empirical data. Within
concepts. Like most exceptions of nature, psychology, the triumph of behaviorist
synesthesia is forcing a paradigm shift. One
cannot admit a wrecking ball and expect the
Like most exceptions of nature,
house to remain standing. Paradoxically, the
very thing that destroys simultaneously illu- synesthesia is forcing a paradigm
minates, and what emerges may surprise us. shift. One cannot admit a
wrecking ball and expect the house
S U B J E C T I V E R E A L I T Y O R P O P P YC O C K ?
In 1989, I reported my initial studies on to remain standing.
several dozen synesthetes in Synesthesia:
A Union of the Senses. Here I proposed that theory further ensured that inquiry into
the phenomenon pointed to deep cogni- mental life would remain taboo for decades.
tion, meaning fundamental processes that Because a technological focus domi-
underlie how we perceive and think. It was nated science in general and medicine in
“a voice in the wilderness,” to quote one particular, my neurology colleagues unsur-
of today’s researchers, V. S. Ramachandran. prisingly asked what Michael’s CAT scan
In fact, 11 years before my own effort, showed. In questioning synesthesia’s reality,
Lawrence Marks, a psychophysicist at Yale, they sought a third-person technological
suggested in The Unity of the Senses that verification of a first-person experience.
understanding synesthesia might shed light Technical corroboration is one thing; but
on the perceptual basis of metaphor and the sweeping assumption that anyone’s
perhaps even the acquisition of language personal experience is invalid is quite another.
itself. He, too, was mostly ignored. Even current functional brain imaging,
Often throughout its history, synes- which is supposed to be anatomically objec-
thesia had been dismissed simply because tive, starts with what one wants to verify
the condition is revealed only through an objectively: the subject’s state of mind.
individual’s self-reported mental state. Synesthesia refuses to be ignored,
There is no test for it in the usual sense of affirming loudly that subjective mental
that word. The complaint that introspection worlds do exist. Among other things, there-
is inherently unreliable and therefore imper- fore, synesthesia grants us an opportunity
missible as scientific data has a long history. to examine the dichotomy of objective-
In the 19th century, psychophysicists such subjective experience. But its importance
as Gustav Fechner tried to formulate laws goes deeper than that.
9

4. Cerebrum
A L I N K W I T H T H E PA S T never satisfied, for instance, with saying
Compared with the hostility of modern ‘blue,’ but take a great deal of trouble
objectivists, a fair number of earlier scientists to express or match the particular blue
accepted synesthesia as a genuine phenom- they mean.”
enon after Sir Francis Galton’s 1880 report Indeed, one can take the details
in Nature on “visualized numerals”—if of a given synesthete today and find
only because the individual stories sounded matching examples in the classical scientific
so similar, giving it the clinician’s feel of a literature, linking the efforts of scientists
genuine phenomenon. The earliest medical a century ago with those of contemporary
reference, a case of sound-induced color, ones. Ironically, it is precisely synesthetes’
dates to 1710, but the style and details subjective claims that now form the
of Galton’s report make his the first recog- basis of today’s experiments that address
nizably modern one. For example, Galton’s predictions regarding the trait’s perceptual
synesthetes express astonishment at discov- reality.
ering that they are unusual. Most claim
to have had the ability as far back as they W H AT D E F I N E S S Y N E S T H E S I A ?
can remember and, far from trying to Synesthesia can be acquired via epilepsy or
appear special or call attention to themselves, the ingestion of hallucinogens such as
genuine synesthetes prefer to hide their mescaline or LSD, but idiopathic (or devel-
trait because of the ridicule they suffer opmental) synesthesia arises naturally with-
upon disclosure. out an external agent or brain abnormality.
The experience of synesthesia is There is nothing in need of medical treat-
difficult to express, as witnessed by the ment. The subjective, ineffable, and idio-
syncratic nature of this kind of synesthesia
does make it an easy target for dismissal.
Compared with the hostility of
Even the term “synesthesia” has been used
modern objectivists, a fair number of imprecisely over time, referring to everything
earlier scientists accepted synesthesia from metaphor (loud tie, sharp cheese,
sweet voice) to deliberate contrivances such
as a genuine phenomenon.
as son et lumière theatrical performances
and “smellavision.”
collective struggle to convey exactly what is A clear definition avoids a muddle.
sensed. Even computer animations are said Idiopathic synesthesia is defined by five
to be only about 60 percent representative clinical findings: It is (1) involuntary
of “what it is really like.” As Galton noted and automatic, (2) spatially extended,
in his 1883 Inquiries into Human Faculty, (3) consistent and generic, (4) memorable,
those with visual synesthesia are “invariably and (5) affect-laden. These refer to specific
most minute in their description of the characteristics of the synesthetic person’s
precise tint and hue of the color. They are experience.
10

5. The Dana Forum on Brain Science
I N VO LU N TA RY A N D AU TO M AT I C by printing graphemes (a language’s
Synesthetes claim to hear a certain sound written elements) in ink colors that are
or to look at a letter, for example, and then either congruent or incongruent with
to see a color. “It just happens,” some say. a given synesthete’s perceptions—and then
How can we demonstrate that they have no measuring reaction times to them has
control over their experiences? Phenomena become a popular approach in current research.
called “perceptual grouping” and “pop-out” In another setup, surrounding a target
demonstrate that the response is indeed grapheme in the visual periphery with other
automatic. For example, imagine that a letters renders it “invisible,” meaning that it
group of 2’s arranged to form a triangle is is not consciously perceived. Remarkably, it
embedded in an array of 5’s drawn in a way still evokes the synesthetic color. “It must be
‘A’ because I see red,” a subject will say. This
Two individuals with the same implies that synesthesia is evoked at an early
sensory level—a preconscious one, in fact.
kind of synesthesia will rarely
As these examples show, many of the
agree as to the particulars of what probes designed to reveal whether synesthesia
they perceive. The numeral 2 is automatic also turn out to prove that
synesthesia is perceptual. What are called
may be green or red or turquoise for
random dot stereograms do even more,
different people. helping us identify the lowest brain level at
which synesthesia can occur. When the left
that the figures resemble mirror images. eye looks at one pattern of black dots and
When told to look for a hidden shape, most the right eye at another, the two images
of us would take time to hunt down the fuse in the brain, causing a three-dimen-
target triangle buried within the distracting sional object to pop out from the viewing
5’s. But a synesthete who sees every numeral plane. Synesthetes see the object, as every-
as differently colored would immediately see one else does, but they see it in color. This
the target pop out of an alternatively colored result says two things: that synesthetic color
background. If the perception is involuntary, arises after binocular fusion (setting the low-
synesthetes should perform much faster than er brain limit above the first synaptic level of
nonsynesthetes—and they do. visual neurons in the cerebral cortex, called
Because synesthetic associations are V1), and that color appears to be bound
idiosyncratic, such tests must be tailored to to a form as the form is being recognized.
the individual. That is, two individuals
with the same kind of synesthesia will rarely S PAT I A L LY E X T E N D E D
agree as to the particulars of what they Some synesthetes describe Technicolor
perceive. The numeral 2 may be green or reading “on the page,” even as they simul-
red or turquoise for different people. taneously see the black ink of the printing.
Deliberately inducing mismatches—say, Others with colored hearing speak, for
11

6. Cerebrum
Tests of visual perception demonstrate the reality and consistency of synesthetes’ responses. These are
examples from synesthetes who see colored numbers—for example 5 is green and 2 is orange. In the
top illustration, the black numeral 5 composed of smaller 2’s is seen as green when this synesthete
focuses on the large figure but as orange when focusing on its components. In the middle series, a ran-
dom dot stereogram presents one pattern of dots to the right eye and a different pattern to the left eye.
The two images are fused in the brain (“binocular vision”) and, once again, the numeral 2 stands out as
orange for the synesthete. At the bottom, a triangle of 2’s is imbedded in a pattern of 5’s; most of us
would have to hunt for the triangle, but it instantly pops out in red for this synesthete.
12

7. The Dana Forum on Brain Science
example, of watching “a screen about six days of the week, and the like are not senses
inches from my nose.” Michael Watson often at all; they are categories of knowledge.
reached out in front of him to feel shapes at Because they reckon among the most fre-
arm’s length. Even those who say the synes- quent manifestations of synesthesia, we
thesia is in their “mind’s eye” remark that it need to enlarge our definition beyond pure
differs from ordinary vision and imagination sensory-sensory pairings to include the
by its quality of Euclidean locus, meaning binding of sensory fragments (qualia) to
that it has a sense of physical place. That is, categories of mental concepts. I will return
synesthetes speak of “going to” or “looking to this later.
at” a certain place to examine a sensation.
This quality of spatial extension is CONSISTENT AND GENERIC
particularly dramatic in the perception of Once established in childhood, synesthetic
what are called “number forms.” (The term associations remain stable throughout life, as
is somewhat of a misnomer, given that num- demonstrated by tests and retests spanning
ber forms concern not just integers but any many years. For example, synesthetes may
concept involving serial order.) The percep- be asked to indicate their color responses to
tual qualities of spatial location, shape, and, a list of words. When tested without warning
often, color become synesthetically joined a year later, they report almost identical
to semantically ordered concepts such as responses, whereas controls without synes-
integers, months, the alphabet, shoe sizes, thesia, even if forewarned of retesting a
temperature, and so forth. For example, month before, perform near chance level.
each day of the week or month of the year Synesthetes often remark that some
may be associated with a different colored colors they see are “weird”—ones that they
would never deliberately choose. They
may see colors that they do not like or wish
Synesthetes often remark that that they saw their favorite ones more
some colors they see are “weird”— often. This should not be surprising, given
ones that they would never that their visual systems are being stimulated
via nonoptical means over which they have
deliberately choose. no control. In one interesting example, a
color-blind synesthete with S-cone deficien-
shape, which is perceived in a location cy—which makes it hard to discriminate
specific to the individual. Number forms are blues and purples—speaks of seeing num-
usually colored and create circles, zigzags, bers in “Martian colors,” meaning colors he
loops, and various tortured configurations. is unable to see in the real world. Curiously,
Note that we may speak of synesthesia synesthesia happens to be more common in
as “joined senses”—a sound being associat- blind individuals than the general population.
ed with a visual perception, for example— Saying that synesthesia is generic, as well
but spatial configuration, letters, words, as consistent, means that what is experienced
13

8. Cerebrum
is not complex and pictorial, but elementary— AFFECT–LADEN
blobs, lattices, cold, rough, sour, zigzags, Synesthesia carries a sense of certitude, some-
simple geometric shapes, and so forth. times a “Eureka!” feeling. Most find it highly
pleasurable. Trivial tasks are laden with emo-
MEMORABLE tional affect, so that mental calculations are
When asked what good the trait does, “very pleasurable” and recalling a phone num-
synesthetes immediately answer, “It helps ber is “delightful.” Mismatched perceptions
you remember.” They do have measurably can be “like fingernails on a blackboard.”
high memories, sometimes photographic In a minority of cases, what is
ones, “eidetic” in psychological parlance. perceived is so wretched—for example
The extra bits of information help synesthetes vile-tasting words, or nausea when playing
remember things like telephone numbers a musical instrument—that the condition
and names. As one synesthetic neuropathol- interferes with daily life. Nevertheless,
ogist puts it, “I use it…to help me remem- synesthetes say that they would never part
ber correct sequences of numbers, words, with their perceptions. It is hard to
phrases, letters, to help me remember names overstate the intensity and pervasiveness of
and locations of anatomical structures (espe- affect in synesthesia.
cially neuroanatomical structures—you
should see the beautiful array of colors in PI C T U R E S , P L E A S E
the brain!) and neuropathological classifica- Synesthesia’s reality is demonstrated by its
tions. I could go on and on.” automaticity, consistency, and durability;
The memory expert that renowned by its induction of perceptual grouping and
Russian neuropsychologist Alexander Luria pop-out; by the evocation of colors by
described in The Mind of a Mnemonist “invisible” graphemes at an unconscious
level; by its strong heritability as an X-
linked dominant trait; by the fact that having
Synesthesia carries a sense of
one type of synesthesia makes one more
certitude, sometimes a “Eureka!” likely to have a second or third type; and by
feeling. Most find it highly the ability of color-blind and blind persons
to see colors.
pleasurable.
Despite these kinds of proofs, some
skeptics can be satisfied only by machine
possessed a flawless memory because every- verifications that produce pictures of the
thing he recalled was accompanied by brain. What is remarkable is how profoundly
synesthesiae in each of his senses: “I heard the emphasis of those pictures has switched
the bell ringing... A small round object from structure to function. When, around
rolled right before my eyes... My fingers 20 years ago, my colleagues asked about
sensed something rough like a rope... then Michael Watson’s CAT scan, they expected
a taste of saltwater... and something white.” that a gross brain abnormality must underlie
14

9. The Dana Forum on Brain Science
Artists and composers who are synesthetes
often seek to express their unique perceptions in
their works. This sculpture by synesthete artist
Carol Steen is called Cyto, because it represents
for her the shapes and colors of the name
[Richard] Cytowic, from whom she first gained
knowledge, beyond her own experience, of the
widespread phenomenon of synesthesia. “The
forms are constructed one on top of the other in
a vertical arrangement,” she says, “because I
often see flying colored forms appear that way in
my synesthetic visions.”
synesthesia if it were real. In other words, flow.” This showed that Michael’s brain
where was “the hole in his head”? But given behaved much differently from nonsynesthetic
that synesthetes such as Michael are normal, ones, being strongly perturbed by ordinary
manifesting no evident neurological impair- stimuli such as smell. This study also con-
ment, a structural lesion such as a stroke, firmed that synesthesia was a phenomenon
tumor, or a bit of missing brain would be of the brain’s left hemisphere. This left-
unlikely. As expected, his CAT and MRI brain locus disappoints some people, who
scans, which assess structure, were normal. want it to be a right-brain function because
What was wanted was a test of function. they consider synesthesia artistic and creative.
In 1980, I performed the first such In 1995, Eraldo Paulesu and col-
functional test on a synesthete, using a leagues performed PET scans on six women
technique called “regional cerebral blood who saw colors in response to spoken
15

10. Cerebrum
V3A Front
V3
V1/V2
V4 (color)
Face and object V5 (motion)
recognition areas
Areas of the brain’s visual cortex are labeled according to their primary functions, V1 having to do with
sorting the signals for various visual tasks, V2 and V3 relating to the perception of form, V4 relating
to color, and V5 relating to motion and direction. Imaging studies show that, surprisingly, synesthetes
can generate conscious visual experiences without activating V1 or V2.
words. PET offers superior spatial resolution kinds of signals to different destinations
and other advantages to assessing function where different types of transformations are
compared with my earlier technique. In this carried out, and so it is expected to activate
study, spoken words activated auditory in all visual tasks. At the second synaptic
and language areas in both synesthetes and level, V5 pertains to motion and direction,
controls, but only in the synesthetes did V4 to color, and V2 and V3 to form
they also activate some visual areas. perception. At the fourth synaptic level,
Scientists have labeled only a few of neither the areas pertaining to facial recog-
the numerous cortical areas involved in nition nor spatial-location encoding has
vision using a numbering scheme. V1, for- yet received a “V” label. Whereas Paulesu’s
merly called the primary visual cortex, is study did not show the hoped-for activation
the first level at which retinal projections of the unique human color area, V4
synapse in the cortex. V1 acts like a post (probably due to a limitation of the PET
office, sorting and forwarding different technique), it did provide a result that was
16

11. The Dana Forum on Brain Science
startling: a failure to activate V1 or V2 in B E C A R E F U L W H AT YO U W I S H F O R
synesthetes. These two early visual areas do In 2002, a functional MRI (fMRI) study
activate when control subjects view colors. by Julia Nunn and her colleagues at last
This result is inconsistent with a confirmed what was long expected: V4 acti-
major premise of what is called “blind- vation (without V1 or V2 activity) in synes-
sight.” Some brain-damaged patients retain thetes who see color in response to spoken
capacities of which they are not conscious. words. Whereas both synesthetes and controls
Oxymoronic terms such as “blindsight” or activated auditory and language areas as
“numbsense” convey how someone unable expected, the synesthetes also activated the
to see or feel can nonetheless discriminate color area (V4), but only on the left—in
visual or tactile test targets with high accu-
racy, despite insisting on not being able to
Synesthetes in the PET study
“see” or “feel” anything. Because stricken
individuals are oblivious to their unconscious proved that the brain can generate
know-how that allows correct discrimination, conscious visual experiences
researchers have postulated that the primary
without contribution from the
sensory cortex (such as S1, V1, A1), which is
damaged in these individuals, is indispensable primary visual cortex.
for any conscious awareness. In the words of
Lawrence Weiskrantz, the acknowledged agreement with earlier results. Such lateral-
authority in the field, “striate cortex [V1] is ization is tantalizing, given that their color
essential…for any ‘seen’ [consciously expe- experiences were not confined to the right
rienced] perception whatsoever.” visual field. The fMRI technique, which is
Not any longer. Synesthetes in the the most refined one we have to date, also
PET study proved that the brain can gener- disclosed activation in areas concerned with
ate conscious visual experiences without memory and emotion, again supporting
contribution from the primary visual cortex both the subjective statements and clinical
(V1). Blindsight’s implications for conscious- observations of synesthetes.
ness studies therefore need to be rethought. An unexpected result of this study was
In the meantime, synesthesia supports the that when actually viewing colored surfaces,
claim by vision researcher Semir Zeki that synesthetes do not activate their left V4,
activity in any given module sustaining a the area for color. Right V4 did function
given visual function (V4 for color, V5 for similarly for both synesthetes and controls.
motion, V3 for form) is sufficient, as well Ordinarily, viewing colors activates both
as necessary, for one to be conscious of that right and left V4, as well as the early visual
color, motion, or form. That is, activation of areas V1 and V2. The implication, there-
V4 alone is sufficient to “see” color, without fore, is that the participation of left V4 in
the necessity of recruiting other visual synesthetic color experience renders it
modules, either upstream or downstream. unavailable for ordinary color perception—
17

12. Cerebrum
in other words, synesthesia appears to have projection from auditory speech areas to the
hijacked an existing brain function. This visual color area known as V4.
surprise is consistent with the observation Those of us who study synesthesia
that nonsynesthetes merely imagining colors mostly concur now that inheriting a genetic
(compared to performing a visual control mutation results in a failure in synesthetes’
task not involving color) do not activate brains to prune the projections between
V4. Thus, the brain basis of synesthetic brain structures that normally exist tem-
color experience is consistent with real color porarily during the development of all
perception rather than color imagery. This brains. This is what we call the “neonatal
refutes earlier criticisms that synesthetes hypothesis” for synesthesia: Everyone is
are just “making it up” or have “overactive
imaginations.” The objectivists have finally gotten a
Lastly, this study has largely over-
machine proof of synesthesia, but it
thrown the only strong alternative explana-
tion of synesthesia, namely, that it results has disappointed their expectations.
from childhood learning through associa-
tion. This claim said that playing with born synesthetic, only to lose the capacity
refrigerator magnets or coloring books, for as the brain matures. Because it is not
example, makes some children form enduring possible to directly map hardwiring in living
associations such as “ ‘A’ is red.” Rigorous humans, we are at present debating precisely
efforts to train controls to imagine colors where these projections might lie, and
in response to words demonstrate that this is dreaming up ways to confirm or disprove
not so. Despite training until controls our conjectures.
achieved 100 percent accuracy, they showed So, the objectivists have finally gotten
no activity whatsoever in V4 on either a machine proof of synesthesia, but it has
side. To further show that synesthetes did disappointed their expectations.
not possess extraordinary associative skills,
synesthetes who had claimed no spontaneous C O N V E N T I O N U N D E R T H R E AT
color response to music were trained to asso- The existence of any physical projection
ciate colors with a melody, as were controls; as a basis for synesthesia threatens one of
neither group had activity in the V4 region contemporary neuroscience’s widely held
that had activated when synesthetes heard concepts, modularity. As initially proposed
spoken words. Thus, not only was learning by Rutgers University philosopher Jerry
ruled out as an explanation, but also the pat- Fodor, the mind is constructed of indepen-
terns of brain activity could easily distinguish dent subsystems that receive inputs only
the subjective states that synesthetes claimed from a specific category of stimulus and
to experience (word-color) or denied having that operate uninfluenced by activity in
(music-color). Taken together, these results other modules or systems. The concept of
support the existence of a direct neural modularity originally referred to cognitive
18

13. The Dana Forum on Brain Science
domains, but over time has extended into opposite sides of the brain, one optical and
the physical organization of the brain, such the other synesthetic, are both subjectively
that relatively self-contained entities such experienced as the color red.
as V1, V4, and the grapheme area are also Another argument put forth for func-
referred to as modules. The mental and tionalism is that functions giving rise to
physical concepts are not wholly comparable, qualia must benefit the organism, because
but this is not central to my point. Synes- evolution selects for traits favoring survival.
thesia obviously raises the question of If this is correct, one should not encounter
whether the concept of modularity per se qualia that interfere with the functions of
remains entirely valid. which they are part. I have already men-
Another endangered favorite of tioned the situations where a perceptual
philosophers and cognitive scientists is mismatch slows performance, however, and
functionalism. This concept relates to what I give many examples of sensory interference
is called the “hard problem of conscious- in my textbook, to say nothing of the
ness,” namely, the subjective aspect of unpleasant and sometimes disruptive affect
perception. Functionalism describes the accompanying some synesthesiae. Nor is
relations among sensory inputs and their there any positive evidence that the quale of
neural transformations, the resulting behavior, color helps aural or visual word perception.
and our conscious experience. The concept These observations are incompatible with
has engendered many varieties of philo- the evolutionary claim of functionalism.
sophical argument. One popular formulation In 1997, Jeffrey Gray was the first to
states that each subjective experience notice the danger that synesthesia posed to
(“quale,” plural “qualia”) is identical to the the hard question of consciousness, and he
function with which it is associated. That is,
functionalism replaces any supposition that
Two different neural processes
red “feels like” a certain state with, instead,
an observable behavior, such as a person on opposite sides of the brain, one
saying “red” or pointing to it. Functionalism optical and the other synesthetic,
says that qualia are the functions (input-
are both subjectively experienced as
processing-behavioral output) by which
they are supported and nothing more. the color red.
If so, then two conditions incompatible
with functionalism would be two qualia has studied this problem in depth. Because
produced by a single function, or two func- functionalism purports to be a general
tions producing the same quale. In synes- account of consciousness, a single negative
thetes, the quale of “red,” for example, can instance that it cannot explain is sufficient
arise either by optical or nonoptical routes. to render it invalid, just as the axiom
This is an example of the second condition, “All swans are white” can be invalidated by
since two different neural processes on observing a single black swan. If functionalism
19

14. Cerebrum
“I watched the black background become pierced by a bright red color that began to form in the middle
of the rich velvet blackness. The red began as a small dot of color and grew quite large rather quickly, chasing
much of the blackness away. I saw green shapes appear in the midst of the red color and move around the
red and black fields.” Carol Steen
20

15. The Dana Forum on Brain Science
does not work in synesthesia, it does not
C R E AT I V I T Y A N D S Y N E S T H E S I A work anywhere and thus cannot be a general
account of consciousness.
Painter and sculptor Carol Steen, whose The ready objections that synesthetes
work appears on pages 7, 15, 20, and 25, are not really seeing red—that they are
is one of many artists with synesthesia. merely being artistic or metaphorical, or
Touch, sound, smell, taste, and pain, as saying what they do only because of a vivid
well as letters and numbers, all give her memory of some past association such as
perceptions of colors and shapes, most of refrigerator magnets—have already been
which she experiences as internal. Loud addressed. Because it is unlikely that
or unexpected sounds or sensations may philosophers will now succeed in eliminating
produce visions that she sees externally synesthesia, they must either eliminate func-
or feels as compression waves through tionalism or refine it. I feel confident they
her body. will choose the latter, because philosophers
Steen says, “The intensely brilliant, never tire of arguing.
luminous colors and simple, soft-edged
Lastly, synesthesia deals a blow to the
staunchest objectivists by showing clearly
three-dimensional shapes are also textured
how perception is not passive, how it is not
and kinetic, but cast no shadow. In these
an impression in the brain transferred by
rich visions, lustrous, vividly colored
objective physics in the world “out there”
shapes move in layers on equally saturated
(philosophers call this direct realism). When
colored fields in arbitrary spatial arrange-
a synesthete responds to the word “butter”
ments almost faster than my vision can see
by saying “blue circles moving off to the
them and my memory can record them.
right,” she demonstrates a lack of correspon-
The shapes move, and the backgrounds
dence, let alone an identity, between the
they appear against move as well.”
physical world “out there” that produces the
Many of Steen’s colored touch experi- percept and the percept itself. Many other
ences have arisen during acupuncture approaches have supported this notion that
treatments. Vision (1996), on the facing perception is active and constructive; synes-
page, was the first painting in which she thesia happily provides a clear example.
recorded such a vision. Aurora, (2002), So much for the wrecking ball. What
on page 7 was also inspired by Steen’s issues might synesthesia illuminate? Two
perceptions during an acupuncture session. big ones are the so-called binding problem
She says, “What I paint matches my and metaphor.
experience only as closely as the medium
of paint will permit...The colors I see T H E B I N D I N G P RO B L E M
synesthetically are the colors of light, not Diverse perceptual attributes (such as color
of pigment.” or shape) are processed in different areas of
my brain, yet I perceive an apple as a unitary
21

16. Cerebrum
entity, not something red + round + edible. knowledge and skill, a stricken bird watcher
What is more, attributes are processed not says that all the birds look alike, a farmer
only in different locations but also at differ- can no longer distinguish his cows, and a
ent times in my brain. For example, color is gardener cannot tell one plant from another.
perceived before motion, which is perceived Might synesthesia relate to the brain’s
before form. How all of these sundry, search for constancy and the assignment of
asynchronous attributes get bound into a essential features that constitute a category?
seamless perception—red apple—endlessly An enduring puzzle of neuroscience is how,
baffles neuroscientists. Inasmuch as synes- out of a constantly changing and infinite
thesia binds perceptual qualia together in energy flux, the brain—whose resources are
anomalous combinations, might it not finite—assigns objects their constant features.
say something useful about the process of Color and form, so prominent in
binding in general? synesthesia, are properties constructed by
There is a further twist. I mentioned the brain through what are called constancy
earlier that synesthesia’s most common operations. For example, most of us accept
manifestation is a coupling of sensory qualia the explanation that something looks red
to categories of knowledge: for example, because it reflects red wavelengths more
color, flavor, texture, locus, or configura- than others, but color is actually a property
tion may be bound to letters and integers, of brains and not of the physical world. For
members of a serially ordered set (such as surface colors to be perceived as constant
despite ever-changing illumination, it is
precisely the wavelength composition of
An enduring puzzle of neuroscience
reflected light that the brain must ignore.
is how, out of a constantly changing Grass looks green, whether it is in bright
and infinite energy flux, the brain— sunlight or shade, despite large differences
in wavelength composition of the light.
whose resources are finite—assigns
Similarly, all constructed properties require
objects their constant features. that the brain discount certain things.
With color, it is wavelength composition of
days of the week), words, or even symbols reflected light that the brain must ignore;
such as braille. Consider how many neuro- with form, it is the viewing angle; and with
logical syndromes (the agnosias) as well as size, it is viewing distance.
imaging studies demonstrate that we think Synesthesia has led me over time to
in categories. In prosopagnosia, for example, favor a model of brain organization called
stricken individuals can no longer recognize the distributed system. The prime features
faces. They recognize a face as a face, but of this model are a distribution of function
cannot say whose face it is. Their larger fail- (hence the name) across structures—as in
ure is in comprehending examples within neural networks—and simultaneity of activity
a category. Thus, despite all their previous on several levels, compared to the older and
22

17. The Dana Forum on Brain Science
more familiar hierarchical and sequential a given cerebral module participates in
cascade, in which a module is assumed to more than one cognitive function and con-
complete its transformation of neural inputs nects with several-to-many other nodes. A
before passing the result on to the next given function is not so much localized in
module in the sequence. This older idea the sense of classical neurology, but exists as
may be likened to stations in a factory the dominant process within its distributed
connected by a conveyor belt by means of system at any given time. Multiple synaptic
which one thing after another is added, levels are active simultaneously, each node
influencing the state of adjacent levels (as in
the example of our simultaneous authors).
The answer to synesthesia will not
Such organization reminds us that localiza-
be a “where” but a “what.” tion is a function of probability—and
not just in this model but in any scheme of
whereas the distributed system is like neural organization. (Try drawing the
different authors simultaneously writing boundaries of Wernicke’s area on a standard
separate chapters of a book without fully brain atlas—you can’t.) Scans mislead us
knowing how the other chapters end. The by emphasizing peak probabilities, which
distributed system also departs from the we misconstrue as fixedly anatomical. The
older idea of a strict one-to-one mapping answer to synesthesia will not be a “where”
of function to anatomy, depending instead but a “what.”
on topological relations and convergent- It would thus be wrong for me to
divergent connections among brain modules. leave the impression that V4 is the seat
These two features result in the multiple of synesthesia: Any module found active by
mapping of a given function, as seen in the a scan (or other means) is really just one
numerous modules pertaining to vision, node in the distributed system underlying
some of which we understand better than expression. The totality of synesthetic expe-
others (such as V4 for color, or V5 for rience involves more than the conscious
motion and direction). Relevant to synes- perception of a single quale, as I hope I
thesia, what are called transmodal modules have conveyed throughout this article. My
(meaning “not pertaining to any single comments regarding the participation of
sense”) do three things: They construct transmodal modules in synesthesia are not
multisensory representations of the world, incompatible with the idea, mentioned
they provide memory and affect to experi- earlier, that an inherited genetic mutation
ence, and they critically participate in causes extraordinary, one-way projections
establishing categories via groups of coarsely between cerebral modules that underlie
tuned neurons. very specific functions. A connection
This model organizes brain tissue between, say, the grapheme area that allows
into five major networks and many lesser one to understand written numbers and the
distributed systems. In any one such system, V4 color area does not fully “explain”
23

18. Cerebrum
synesthetically colored numbers, however, heterogeneity in the depth of subjective
because it leaves out the affect of the experience, from purely sensory-sensory, to
experience, its memorability, whether the categorical-sensory, to verbal-sensory. In
synesthetic color moves, has a given spatial this last, even a concept—just thinking of
location, and so forth. As V. S. Ramachan- the number 5, say, or a person named
dran points out, what are called transcription Marion—is sufficient to trigger synesthesia.
factors can partly solve this shortcoming Some time ago, both Lawrence Marks and
by causing the gene’s effects to be I proposed a cognitive continuum extend-
expressed either discretely or diffusely—or ing from perception to synesthesia to
anywhere between—in the brain. Such metaphor to language. With time, others
variability goes a long way toward explain- have come to concur.
ing the observed variety of synesthetic Systematic correspondences exist
experience, and why some people have among dimensions of a given sense for
only one kind whereas others have three synesthetes and nonsynesthetes alike. For
example, both say that louder tones are
brighter than soft tones, that higher ones
I proposed a cognitive continuum
are smaller than lower ones, and that low
extending from perception to tones are both larger and darker than high
synesthesia to metaphor to language. ones. The perceptual similarities that yield
such orderly relationships among pitch,
With time, others have come
loudness, brightness, and size, for exam-
to concur. ple, turn out to be rooted in fundamental
similarities of physical experience itself.
or four different kinds of synesthesia. Thus, Perceptual similarities, synesthetic equiva-
transcription factors expressed in different lences, and metaphoric identities in turn
places through-out the brain could account, become available to the more abstract
theoretically at least, for subsidiary features knowledge that is embodied in language.
of synesthesia such as memorability and In other words, the acquisition of
affective charge. But it is precisely this metaphor relies not on a capacity for ver-
necessity of widespread expression that makes bal abstraction, as many mistakenly
me point out why synesthetic experience believe, but on our physical interaction
per se cannot be localized to any one physical with the world. The subjective-objective
spot in the brain and why scans mislead us dichotomy of experience should be turned
in this regard. into a unity, because we need both points
of view.
M E TA PH O R A N D L A N G UAG E Objectivity fails to see how the human
The heterogeneity of the synesthetic expe- system of concepts is metaphoric, involving
rience connotes more than wide variety an imaginative understanding of one
of perceptual combinations. There is also thing in terms of another. We elaborate the
24

19. The Dana Forum on Brain Science
Synesthetes who see colored letters have their own individual
alphabets. Carol Steen, who painted this representation of her
alphabet, says she saw many of the more brightly colored letters
as a young child, but the iridescent and metallic colored letters did
not appear until she was in her 30s. Many synesthetes say that
their perceptions become richer and more complex as they age.
metaphor “The mind is an entity” into sense of fragmentary information, and
another metaphor, “The mind is a machine,” the unexpected suddenness of insight. By
when we say, “He ran out of steam.” switching metaphors, we alter how we
Metaphors emphasize some aspects of an comprehend a thing.
object but hide others. The machine Subjectivity fails to see that even the
metaphor paints the mind as having a most imaginative flights occur in a context
source of power, an on-off state, and an of objective experience gained by living in
expected level of efficiency, but it hides a physical and cultural world. Increasingly,
the vagaries of thought, its ability to make science is viewing metaphor as an emergent
25

20. Cerebrum
property of mind that is rooted in the tionally bound to discern form, movement,
body. As semiotics have long known, direction, spatial location, and other qualia
meaning inheres in affect, which the body that we conventionally ascribe to vision.
feels as physical and the mind apprehends The capacity for anomalous binding, which
as mental. Because metaphor perceives is the essence of synesthesia, is therefore
the similar in the dissimilar, it also points latent in all brains.
to constancy and categorization, features Nature reveals herself through excep-
germane to synesthesia. Perhaps a tendency tions. Those objectivists who tried to dis-
to map one concept to another unconven- miss synesthesia throughout its history
tionally even underlies what appears seem to have forgotten this maxim. Far
to be synesthetes’ distinctive approach to from being a mere curiosity irrelevant to
creativity. real questions, synesthesia turns out to illu-
One implication of a continuum from minate a wide swath of mental life and
perception to synesthesia to metaphor to forces us to rethink some fundamental
language is that synesthesia resides univer- issues regarding mind and brain. At pre-
sally in each of us but, for reasons yet sent, I can think of nothing more relevant
unknown, rises to consciousness in only a to our quest for self-understanding.
few. Heinz Werner suggested as much in
References
the 1930s but technology takes time to
catch up with ideas. Two bits of recent Bogen, JE, Bogen, GM. “Wernicke’s region–Where is it?”
Annals of the New York Academy of Science 1975; 280:
work support this conjecture. One study 834-843.
Cytowic, RE. The Man Who Tasted Shapes. Cambridge.
MIT Press, 1998.
Nature reveals herself through Cytowic, RE. Synesthesia: A Union of The Senses, 2d ed.
exceptions. Those objectivists who Cambridge. MIT Press, 2002.
Gray, JA, et al. “Possible questions of synesthesia for
tried to dismiss synesthesia the hard question of consciousness.” In S Baron-Cohen
& JE Harrison, eds., Synaesthesia: Classic and Contem-
throughout its history seem to have porary Readings. Oxford. Blackwell, 1997: 173-181.
forgotten this maxim. Luria, AR. The Mind of a Mnemonist. New York. Basic
Books, 1968.
Marks, LE. The Unity of The Senses: Interrelations
found that synesthesia is 100 times more Among the Modalities. New York. Academic Press, 1978.
frequent during Zen meditation; the other Nunn, JA, et al. “Functional magnetic resonance imag-
ing of synesthesia: activation of V4/V8 by spoken
confirmed the ability of both blind and words.” Nature Neuroscience 2002; 5(4): 571-575.
sighted persons to “see” video impulses Paulesu, E, et al. “The physiology of coloured-hearing:
fed into an electrode array placed on a PET activation study of colour-word synaesthesia.”
Brain 1995; 118: 661-676.
the tongue. We do not see with our eyes,
Ramachandran, VS, Hubbard EM. “Synesthesia:
anyway, but with our brains. What this a window into perception, thought, and language.”
latter demonstration shows is that tactile Journal of Consciousness Studies 2001; 8(12): 3-34.
sensations on the tongue can be unconven- Zeki, S. A Vision of the Brain. Oxford. Blackwell, 1993.
26