Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.



Published on

  • Be the first to comment

  • Be the first to like this


  1. 1. Sometimes the bizarre is just the bizarre, but when the anomaly is a product of fundamental brain processes yet seems to confound them, the theory makers have by Richard E. Cytowic, M.D. a problem. For several decades, most neurologists who heard about synesthetes—people who see colored letters, feel colored pain, and taste shapes—just shrugged their shoulders or rolled their eyes. Touching Tastes,Seeing Smells—and Shaking Up Brain ScienceNow, as evidence accumulates that synesthesia cherished conceptions of contemporary brainis a bonafide neurological phenomenon, some are science. The good news is that when old theoret-asking how this squares with some of the most ical structures fall, new light may flood in. 7
  2. 2. CerebrumS ome 20 years ago, my dinner host, person, object, or concept evokes synes- Michael Watson, delayed our seating thetic sensations. by announcing, “There aren’t As children, synesthetes are surprisedenough points on the chicken.” He meant to discover that others do not share thesethat the taste still failed to evoke the prickly experiences. Often ridiculed and disbelieved,sensation he sought. To Michael, tastes and they learn to keep their atypical perceptionssmells were also felt as a physical touch in to themselves. Nonetheless, the phenomenonhis face and hands. “With an intense flavor,” remains involuntary and consistent throughouthe tried to explain, “a feeling sweeps down their lives. The trait runs strongly in families,into my hand, and I feel weight, texture, and the genetics of its inheritance are reason-shape, and whether it’s hot or cold as if I’m ably well understood. Some type of synestheticactually grasping something.” experience occurs in perhaps 1 in 200 individ- “Ah,” I exclaimed, “You have uals, and more than 75 percent are women.synesthesia.” Like most anomalies that lie outside the Michael looked stunned. “You mean explanation of conventional theories, synes-there’s a name for this thing?” thesia was long dismissed as a mere curiosity Sharing a Greek root with anesthesia, or, worse, as just subjective imagination. Iwhich means “no sensation,” synesthesia wondered how the brains of people likemeans “joined sensation,” wherein two or Michael Watson might differ from the majori-more senses are coupled in such a way that ty, but my colleagues refused to accept thata voice, for example, is not only heard but his experience might be a bona fide neuro-also felt, seen, or tasted. We call individuals logical phenomenon. Pointing to 200 years ofwith this coupling synesthetes. The process of synesthesia usually As children, synesthetes aretravels in only one direction. For example, surprised to discover that others doa sweet taste made Michael feel a cool,polished, curved surface in his hands, but not share these experiences.handling a billiard ball would not elicit Often ridiculed and disbelieved,any flavor in him. About 40 percent of they learn to keep their atypicalsynesthetes have multiple types of synes-thesia; the couplings that have been perceptions to themselves.observed by scientists do not include allpossible pairings. Sensing letters, numbers, synesthesia history in the annals of medicineor words as colored accounts for two- and psychology did not sway them. Today,thirds of the instances of synesthesia. For however, researchers in some 15 countriesthose with “colored hearing,” sounds are studying synesthesia, and many doctoralevoke colored shapes that arise, move, candidates have chosen it for their theses.alter, and fade—somewhat like fireworks. If others have gradually come toFor others, merely thinking about a certain accept the reality of synesthesia, they must 8
  3. 3. The Dana Forum on Brain Sciencenow relinquish some received wisdom about regarding sensation and perception basedhow the brain works. Our concepts of how on observers’ reports, taking as a given thatthings work are but models, after all, reduc- mental states exist. Scientists in the 20thtions of reality that arise from human minds; century, however, consistently strove tohistory has shown repeatedly that reality eliminate the subjective role of a humanhas a way of making a mess of neat and tidy observer in gathering empirical data. Withinconcepts. Like most exceptions of nature, psychology, the triumph of behavioristsynesthesia is forcing a paradigm shift. Onecannot admit a wrecking ball and expect the Like most exceptions of nature,house to remain standing. Paradoxically, thevery thing that destroys simultaneously illu- synesthesia is forcing a paradigmminates, and what emerges may surprise us. shift. One cannot admit a wrecking ball and expect the houseS U B J E C T I V E R E A L I T Y O R P O P P YC O C K ?In 1989, I reported my initial studies on to remain standing.several dozen synesthetes in Synesthesia:A Union of the Senses. Here I proposed that theory further ensured that inquiry intothe phenomenon pointed to deep cogni- mental life would remain taboo for decades.tion, meaning fundamental processes that Because a technological focus domi-underlie how we perceive and think. It was nated science in general and medicine in“a voice in the wilderness,” to quote one particular, my neurology colleagues unsur-of today’s researchers, V. S. Ramachandran. prisingly asked what Michael’s CAT scanIn fact, 11 years before my own effort, showed. In questioning synesthesia’s reality,Lawrence Marks, a psychophysicist at Yale, they sought a third-person technologicalsuggested in The Unity of the Senses that verification of a first-person experience.understanding synesthesia might shed light Technical corroboration is one thing; buton the perceptual basis of metaphor and the sweeping assumption that anyone’sperhaps even the acquisition of language personal experience is invalid is quite another.itself. He, too, was mostly ignored. Even current functional brain imaging, Often throughout its history, synes- which is supposed to be anatomically objec-thesia had been dismissed simply because tive, starts with what one wants to verifythe condition is revealed only through an objectively: the subject’s state of mind.individual’s self-reported mental state. Synesthesia refuses to be ignored,There is no test for it in the usual sense of affirming loudly that subjective mentalthat word. The complaint that introspection worlds do exist. Among other things, there-is inherently unreliable and therefore imper- fore, synesthesia grants us an opportunitymissible as scientific data has a long history. to examine the dichotomy of objective-In the 19th century, psychophysicists such subjective experience. But its importanceas Gustav Fechner tried to formulate laws goes deeper than that. 9
  4. 4. CerebrumA L I N K W I T H T H E PA S T never satisfied, for instance, with sayingCompared with the hostility of modern ‘blue,’ but take a great deal of troubleobjectivists, a fair number of earlier scientists to express or match the particular blueaccepted synesthesia as a genuine phenom- they mean.”enon after Sir Francis Galton’s 1880 report Indeed, one can take the detailsin Nature on “visualized numerals”—if of a given synesthete today and findonly because the individual stories sounded matching examples in the classical scientificso similar, giving it the clinician’s feel of a literature, linking the efforts of scientistsgenuine phenomenon. The earliest medical a century ago with those of contemporaryreference, a case of sound-induced color, ones. Ironically, it is precisely synesthetes’dates to 1710, but the style and details subjective claims that now form theof Galton’s report make his the first recog- basis of today’s experiments that addressnizably modern one. For example, Galton’s predictions regarding the trait’s perceptualsynesthetes express astonishment at discov- reality.ering that they are unusual. Most claimto have had the ability as far back as they W H AT D E F I N E S S Y N E S T H E S I A ?can remember and, far from trying to Synesthesia can be acquired via epilepsy orappear special or call attention to themselves, the ingestion of hallucinogens such asgenuine synesthetes prefer to hide their mescaline or LSD, but idiopathic (or devel-trait because of the ridicule they suffer opmental) synesthesia arises naturally with-upon disclosure. out an external agent or brain abnormality. The experience of synesthesia is There is nothing in need of medical treat-difficult to express, as witnessed by the ment. The subjective, ineffable, and idio- syncratic nature of this kind of synesthesia does make it an easy target for dismissal.Compared with the hostility of Even the term “synesthesia” has been usedmodern objectivists, a fair number of imprecisely over time, referring to everythingearlier scientists accepted synesthesia from metaphor (loud tie, sharp cheese, sweet voice) to deliberate contrivances suchas a genuine phenomenon. as son et lumière theatrical performances and “smellavision.”collective struggle to convey exactly what is A clear definition avoids a muddle.sensed. Even computer animations are said Idiopathic synesthesia is defined by fiveto be only about 60 percent representative clinical findings: It is (1) involuntaryof “what it is really like.” As Galton noted and automatic, (2) spatially extended,in his 1883 Inquiries into Human Faculty, (3) consistent and generic, (4) memorable,those with visual synesthesia are “invariably and (5) affect-laden. These refer to specificmost minute in their description of the characteristics of the synesthetic person’sprecise tint and hue of the color. They are experience. 10
  5. 5. The Dana Forum on Brain ScienceI N VO LU N TA RY A N D AU TO M AT I C by printing graphemes (a language’sSynesthetes claim to hear a certain sound written elements) in ink colors that areor to look at a letter, for example, and then either congruent or incongruent withto see a color. “It just happens,” some say. a given synesthete’s perceptions—and thenHow can we demonstrate that they have no measuring reaction times to them hascontrol over their experiences? Phenomena become a popular approach in current research.called “perceptual grouping” and “pop-out” In another setup, surrounding a targetdemonstrate that the response is indeed grapheme in the visual periphery with otherautomatic. For example, imagine that a letters renders it “invisible,” meaning that itgroup of 2’s arranged to form a triangle is is not consciously perceived. Remarkably, itembedded in an array of 5’s drawn in a way still evokes the synesthetic color. “It must be ‘A’ because I see red,” a subject will say. ThisTwo individuals with the same implies that synesthesia is evoked at an early sensory level—a preconscious one, in fact.kind of synesthesia will rarely As these examples show, many of theagree as to the particulars of what probes designed to reveal whether synesthesiathey perceive. The numeral 2 is automatic also turn out to prove that synesthesia is perceptual. What are calledmay be green or red or turquoise for random dot stereograms do even more,different people. helping us identify the lowest brain level at which synesthesia can occur. When the leftthat the figures resemble mirror images. eye looks at one pattern of black dots andWhen told to look for a hidden shape, most the right eye at another, the two imagesof us would take time to hunt down the fuse in the brain, causing a three-dimen-target triangle buried within the distracting sional object to pop out from the viewing5’s. But a synesthete who sees every numeral plane. Synesthetes see the object, as every-as differently colored would immediately see one else does, but they see it in color. Thisthe target pop out of an alternatively colored result says two things: that synesthetic colorbackground. If the perception is involuntary, arises after binocular fusion (setting the low-synesthetes should perform much faster than er brain limit above the first synaptic level ofnonsynesthetes—and they do. visual neurons in the cerebral cortex, called Because synesthetic associations are V1), and that color appears to be boundidiosyncratic, such tests must be tailored to to a form as the form is being recognized.the individual. That is, two individualswith the same kind of synesthesia will rarely S PAT I A L LY E X T E N D E Dagree as to the particulars of what they Some synesthetes describe Technicolorperceive. The numeral 2 may be green or reading “on the page,” even as they simul-red or turquoise for different people. taneously see the black ink of the printing.Deliberately inducing mismatches—say, Others with colored hearing speak, for 11
  6. 6. CerebrumTests of visual perception demonstrate the reality and consistency of synesthetes’ responses. These areexamples from synesthetes who see colored numbers—for example 5 is green and 2 is orange. In thetop illustration, the black numeral 5 composed of smaller 2’s is seen as green when this synesthetefocuses on the large figure but as orange when focusing on its components. In the middle series, a ran-dom dot stereogram presents one pattern of dots to the right eye and a different pattern to the left eye.The two images are fused in the brain (“binocular vision”) and, once again, the numeral 2 stands out asorange for the synesthete. At the bottom, a triangle of 2’s is imbedded in a pattern of 5’s; most of uswould have to hunt for the triangle, but it instantly pops out in red for this synesthete. 12
  7. 7. The Dana Forum on Brain Scienceexample, of watching “a screen about six days of the week, and the like are not sensesinches from my nose.” Michael Watson often at all; they are categories of knowledge.reached out in front of him to feel shapes at Because they reckon among the most fre-arm’s length. Even those who say the synes- quent manifestations of synesthesia, wethesia is in their “mind’s eye” remark that it need to enlarge our definition beyond purediffers from ordinary vision and imagination sensory-sensory pairings to include theby its quality of Euclidean locus, meaning binding of sensory fragments (qualia) tothat it has a sense of physical place. That is, categories of mental concepts. I will returnsynesthetes speak of “going to” or “looking to this” a certain place to examine a sensation. This quality of spatial extension is CONSISTENT AND GENERICparticularly dramatic in the perception of Once established in childhood, synestheticwhat are called “number forms.” (The term associations remain stable throughout life, asis somewhat of a misnomer, given that num- demonstrated by tests and retests spanningber forms concern not just integers but any many years. For example, synesthetes mayconcept involving serial order.) The percep- be asked to indicate their color responses totual qualities of spatial location, shape, and, a list of words. When tested without warningoften, color become synesthetically joined a year later, they report almost identicalto semantically ordered concepts such as responses, whereas controls without synes-integers, months, the alphabet, shoe sizes, thesia, even if forewarned of retesting atemperature, and so forth. For example, month before, perform near chance level.each day of the week or month of the year Synesthetes often remark that somemay be associated with a different colored colors they see are “weird”—ones that they would never deliberately choose. They may see colors that they do not like or wishSynesthetes often remark that that they saw their favorite ones moresome colors they see are “weird”— often. This should not be surprising, givenones that they would never that their visual systems are being stimulated via nonoptical means over which they havedeliberately choose. no control. In one interesting example, a color-blind synesthete with S-cone deficien-shape, which is perceived in a location cy—which makes it hard to discriminatespecific to the individual. Number forms are blues and purples—speaks of seeing num-usually colored and create circles, zigzags, bers in “Martian colors,” meaning colors heloops, and various tortured configurations. is unable to see in the real world. Curiously, Note that we may speak of synesthesia synesthesia happens to be more common inas “joined senses”—a sound being associat- blind individuals than the general population.ed with a visual perception, for example— Saying that synesthesia is generic, as wellbut spatial configuration, letters, words, as consistent, means that what is experienced 13
  8. 8. Cerebrumis not complex and pictorial, but elementary— AFFECT–LADENblobs, lattices, cold, rough, sour, zigzags, Synesthesia carries a sense of certitude, some-simple geometric shapes, and so forth. times a “Eureka!” feeling. Most find it highly pleasurable. Trivial tasks are laden with emo-MEMORABLE tional affect, so that mental calculations areWhen asked what good the trait does, “very pleasurable” and recalling a phone num-synesthetes immediately answer, “It helps ber is “delightful.” Mismatched perceptionsyou remember.” They do have measurably can be “like fingernails on a blackboard.”high memories, sometimes photographic In a minority of cases, what isones, “eidetic” in psychological parlance. perceived is so wretched—for exampleThe extra bits of information help synesthetes vile-tasting words, or nausea when playingremember things like telephone numbers a musical instrument—that the conditionand names. As one synesthetic neuropathol- interferes with daily life. Nevertheless,ogist puts it, “I use it…to help me remem- synesthetes say that they would never partber correct sequences of numbers, words, with their perceptions. It is hard tophrases, letters, to help me remember names overstate the intensity and pervasiveness ofand locations of anatomical structures (espe- affect in synesthesia.cially neuroanatomical structures—youshould see the beautiful array of colors in PI C T U R E S , P L E A S Ethe brain!) and neuropathological classifica- Synesthesia’s reality is demonstrated by itstions. I could go on and on.” automaticity, consistency, and durability; The memory expert that renowned by its induction of perceptual grouping andRussian neuropsychologist Alexander Luria pop-out; by the evocation of colors bydescribed in The Mind of a Mnemonist “invisible” graphemes at an unconscious level; by its strong heritability as an X- linked dominant trait; by the fact that havingSynesthesia carries a sense of one type of synesthesia makes one morecertitude, sometimes a “Eureka!” likely to have a second or third type; and byfeeling. Most find it highly the ability of color-blind and blind persons to see colors.pleasurable. Despite these kinds of proofs, some skeptics can be satisfied only by machinepossessed a flawless memory because every- verifications that produce pictures of thething he recalled was accompanied by brain. What is remarkable is how profoundlysynesthesiae in each of his senses: “I heard the emphasis of those pictures has switchedthe bell ringing... A small round object from structure to function. When, aroundrolled right before my eyes... My fingers 20 years ago, my colleagues asked aboutsensed something rough like a rope... then Michael Watson’s CAT scan, they expecteda taste of saltwater... and something white.” that a gross brain abnormality must underlie 14
  9. 9. The Dana Forum on Brain Science Artists and composers who are synesthetes often seek to express their unique perceptions in their works. This sculpture by synesthete artist Carol Steen is called Cyto, because it represents for her the shapes and colors of the name [Richard] Cytowic, from whom she first gained knowledge, beyond her own experience, of the widespread phenomenon of synesthesia. “The forms are constructed one on top of the other in a vertical arrangement,” she says, “because I often see flying colored forms appear that way in my synesthetic visions.”synesthesia if it were real. In other words, flow.” This showed that Michael’s brainwhere was “the hole in his head”? But given behaved much differently from nonsynestheticthat synesthetes such as Michael are normal, ones, being strongly perturbed by ordinarymanifesting no evident neurological impair- stimuli such as smell. This study also con-ment, a structural lesion such as a stroke, firmed that synesthesia was a phenomenontumor, or a bit of missing brain would be of the brain’s left hemisphere. This left-unlikely. As expected, his CAT and MRI brain locus disappoints some people, whoscans, which assess structure, were normal. want it to be a right-brain function becauseWhat was wanted was a test of function. they consider synesthesia artistic and creative. In 1980, I performed the first such In 1995, Eraldo Paulesu and col-functional test on a synesthete, using a leagues performed PET scans on six womentechnique called “regional cerebral blood who saw colors in response to spoken 15
  10. 10. Cerebrum V3A Front V3 V1/V2 V4 (color) Face and object V5 (motion) recognition areas Areas of the brain’s visual cortex are labeled according to their primary functions, V1 having to do with sorting the signals for various visual tasks, V2 and V3 relating to the perception of form, V4 relating to color, and V5 relating to motion and direction. Imaging studies show that, surprisingly, synesthetes can generate conscious visual experiences without activating V1 or V2.words. PET offers superior spatial resolution kinds of signals to different destinationsand other advantages to assessing function where different types of transformations arecompared with my earlier technique. In this carried out, and so it is expected to activatestudy, spoken words activated auditory in all visual tasks. At the second synapticand language areas in both synesthetes and level, V5 pertains to motion and direction,controls, but only in the synesthetes did V4 to color, and V2 and V3 to formthey also activate some visual areas. perception. At the fourth synaptic level, Scientists have labeled only a few of neither the areas pertaining to facial recog-the numerous cortical areas involved in nition nor spatial-location encoding hasvision using a numbering scheme. V1, for- yet received a “V” label. Whereas Paulesu’smerly called the primary visual cortex, is study did not show the hoped-for activationthe first level at which retinal projections of the unique human color area, V4synapse in the cortex. V1 acts like a post (probably due to a limitation of the PEToffice, sorting and forwarding different technique), it did provide a result that was 16
  11. 11. The Dana Forum on Brain Sciencestartling: a failure to activate V1 or V2 in B E C A R E F U L W H AT YO U W I S H F O Rsynesthetes. These two early visual areas do In 2002, a functional MRI (fMRI) studyactivate when control subjects view colors. by Julia Nunn and her colleagues at last This result is inconsistent with a confirmed what was long expected: V4 acti-major premise of what is called “blind- vation (without V1 or V2 activity) in synes-sight.” Some brain-damaged patients retain thetes who see color in response to spokencapacities of which they are not conscious. words. Whereas both synesthetes and controlsOxymoronic terms such as “blindsight” or activated auditory and language areas as“numbsense” convey how someone unable expected, the synesthetes also activated theto see or feel can nonetheless discriminate color area (V4), but only on the left—invisual or tactile test targets with high accu-racy, despite insisting on not being able to Synesthetes in the PET study“see” or “feel” anything. Because strickenindividuals are oblivious to their unconscious proved that the brain can generateknow-how that allows correct discrimination, conscious visual experiencesresearchers have postulated that the primary without contribution from thesensory cortex (such as S1, V1, A1), which isdamaged in these individuals, is indispensable primary visual cortex.for any conscious awareness. In the words ofLawrence Weiskrantz, the acknowledged agreement with earlier results. Such lateral-authority in the field, “striate cortex [V1] is ization is tantalizing, given that their coloressential…for any ‘seen’ [consciously expe- experiences were not confined to the rightrienced] perception whatsoever.” visual field. The fMRI technique, which is Not any longer. Synesthetes in the the most refined one we have to date, alsoPET study proved that the brain can gener- disclosed activation in areas concerned withate conscious visual experiences without memory and emotion, again supportingcontribution from the primary visual cortex both the subjective statements and clinical(V1). Blindsight’s implications for conscious- observations of synesthetes.ness studies therefore need to be rethought. An unexpected result of this study wasIn the meantime, synesthesia supports the that when actually viewing colored surfaces,claim by vision researcher Semir Zeki that synesthetes do not activate their left V4,activity in any given module sustaining a the area for color. Right V4 did functiongiven visual function (V4 for color, V5 for similarly for both synesthetes and controls.motion, V3 for form) is sufficient, as well Ordinarily, viewing colors activates bothas necessary, for one to be conscious of that right and left V4, as well as the early visualcolor, motion, or form. That is, activation of areas V1 and V2. The implication, there-V4 alone is sufficient to “see” color, without fore, is that the participation of left V4 inthe necessity of recruiting other visual synesthetic color experience renders itmodules, either upstream or downstream. unavailable for ordinary color perception— 17
  12. 12. Cerebrumin other words, synesthesia appears to have projection from auditory speech areas to thehijacked an existing brain function. This visual color area known as V4.surprise is consistent with the observation Those of us who study synesthesiathat nonsynesthetes merely imagining colors mostly concur now that inheriting a genetic(compared to performing a visual control mutation results in a failure in synesthetes’task not involving color) do not activate brains to prune the projections betweenV4. Thus, the brain basis of synesthetic brain structures that normally exist tem-color experience is consistent with real color porarily during the development of allperception rather than color imagery. This brains. This is what we call the “neonatalrefutes earlier criticisms that synesthetes hypothesis” for synesthesia: Everyone isare just “making it up” or have “overactiveimaginations.” The objectivists have finally gotten a Lastly, this study has largely over- machine proof of synesthesia, but itthrown the only strong alternative explana-tion of synesthesia, namely, that it results has disappointed their expectations.from childhood learning through associa-tion. This claim said that playing with born synesthetic, only to lose the capacityrefrigerator magnets or coloring books, for as the brain matures. Because it is notexample, makes some children form enduring possible to directly map hardwiring in livingassociations such as “ ‘A’ is red.” Rigorous humans, we are at present debating preciselyefforts to train controls to imagine colors where these projections might lie, andin response to words demonstrate that this is dreaming up ways to confirm or disprovenot so. Despite training until controls our conjectures.achieved 100 percent accuracy, they showed So, the objectivists have finally gottenno activity whatsoever in V4 on either a machine proof of synesthesia, but it hasside. To further show that synesthetes did disappointed their expectations.not possess extraordinary associative skills,synesthetes who had claimed no spontaneous C O N V E N T I O N U N D E R T H R E ATcolor response to music were trained to asso- The existence of any physical projectionciate colors with a melody, as were controls; as a basis for synesthesia threatens one ofneither group had activity in the V4 region contemporary neuroscience’s widely heldthat had activated when synesthetes heard concepts, modularity. As initially proposedspoken words. Thus, not only was learning by Rutgers University philosopher Jerryruled out as an explanation, but also the pat- Fodor, the mind is constructed of indepen-terns of brain activity could easily distinguish dent subsystems that receive inputs onlythe subjective states that synesthetes claimed from a specific category of stimulus andto experience (word-color) or denied having that operate uninfluenced by activity in(music-color). Taken together, these results other modules or systems. The concept ofsupport the existence of a direct neural modularity originally referred to cognitive 18
  13. 13. The Dana Forum on Brain Sciencedomains, but over time has extended into opposite sides of the brain, one optical andthe physical organization of the brain, such the other synesthetic, are both subjectivelythat relatively self-contained entities such experienced as the color V1, V4, and the grapheme area are also Another argument put forth for func-referred to as modules. The mental and tionalism is that functions giving rise tophysical concepts are not wholly comparable, qualia must benefit the organism, becausebut this is not central to my point. Synes- evolution selects for traits favoring survival.thesia obviously raises the question of If this is correct, one should not encounterwhether the concept of modularity per se qualia that interfere with the functions ofremains entirely valid. which they are part. I have already men- Another endangered favorite of tioned the situations where a perceptualphilosophers and cognitive scientists is mismatch slows performance, however, andfunctionalism. This concept relates to what I give many examples of sensory interferenceis called the “hard problem of conscious- in my textbook, to say nothing of theness,” namely, the subjective aspect of unpleasant and sometimes disruptive affectperception. Functionalism describes the accompanying some synesthesiae. Nor isrelations among sensory inputs and their there any positive evidence that the quale ofneural transformations, the resulting behavior, color helps aural or visual word perception.and our conscious experience. The concept These observations are incompatible withhas engendered many varieties of philo- the evolutionary claim of functionalism.sophical argument. One popular formulation In 1997, Jeffrey Gray was the first tostates that each subjective experience notice the danger that synesthesia posed to(“quale,” plural “qualia”) is identical to the the hard question of consciousness, and hefunction with which it is associated. That is,functionalism replaces any supposition that Two different neural processesred “feels like” a certain state with, instead,an observable behavior, such as a person on opposite sides of the brain, onesaying “red” or pointing to it. Functionalism optical and the other synesthetic,says that qualia are the functions (input- are both subjectively experienced asprocessing-behavioral output) by whichthey are supported and nothing more. the color red. If so, then two conditions incompatiblewith functionalism would be two qualia has studied this problem in depth. Becauseproduced by a single function, or two func- functionalism purports to be a generaltions producing the same quale. In synes- account of consciousness, a single negativethetes, the quale of “red,” for example, can instance that it cannot explain is sufficientarise either by optical or nonoptical routes. to render it invalid, just as the axiomThis is an example of the second condition, “All swans are white” can be invalidated bysince two different neural processes on observing a single black swan. If functionalism 19
  14. 14. Cerebrum“I watched the black background become pierced by a bright red color that began to form in the middleof the rich velvet blackness. The red began as a small dot of color and grew quite large rather quickly, chasingmuch of the blackness away. I saw green shapes appear in the midst of the red color and move around thered and black fields.” Carol Steen 20
  15. 15. The Dana Forum on Brain Science does not work in synesthesia, it does notC R E AT I V I T Y A N D S Y N E S T H E S I A work anywhere and thus cannot be a general account of consciousness.Painter and sculptor Carol Steen, whose The ready objections that synestheteswork appears on pages 7, 15, 20, and 25, are not really seeing red—that they areis one of many artists with synesthesia. merely being artistic or metaphorical, orTouch, sound, smell, taste, and pain, as saying what they do only because of a vividwell as letters and numbers, all give her memory of some past association such asperceptions of colors and shapes, most of refrigerator magnets—have already beenwhich she experiences as internal. Loud addressed. Because it is unlikely thator unexpected sounds or sensations may philosophers will now succeed in eliminatingproduce visions that she sees externally synesthesia, they must either eliminate func-or feels as compression waves through tionalism or refine it. I feel confident theyher body. will choose the latter, because philosophers Steen says, “The intensely brilliant, never tire of arguing.luminous colors and simple, soft-edged Lastly, synesthesia deals a blow to the staunchest objectivists by showing clearlythree-dimensional shapes are also textured how perception is not passive, how it is notand kinetic, but cast no shadow. In these an impression in the brain transferred byrich visions, lustrous, vividly colored objective physics in the world “out there”shapes move in layers on equally saturated (philosophers call this direct realism). Whencolored fields in arbitrary spatial arrange- a synesthete responds to the word “butter”ments almost faster than my vision can see by saying “blue circles moving off to thethem and my memory can record them. right,” she demonstrates a lack of correspon-The shapes move, and the backgrounds dence, let alone an identity, between thethey appear against move as well.” physical world “out there” that produces the Many of Steen’s colored touch experi- percept and the percept itself. Many otherences have arisen during acupuncture approaches have supported this notion thattreatments. Vision (1996), on the facing perception is active and constructive; synes-page, was the first painting in which she thesia happily provides a clear example.recorded such a vision. Aurora, (2002), So much for the wrecking ball. Whaton page 7 was also inspired by Steen’s issues might synesthesia illuminate? Twoperceptions during an acupuncture session. big ones are the so-called binding problemShe says, “What I paint matches my and metaphor.experience only as closely as the mediumof paint will permit...The colors I see T H E B I N D I N G P RO B L E Msynesthetically are the colors of light, not Diverse perceptual attributes (such as colorof pigment.” or shape) are processed in different areas of my brain, yet I perceive an apple as a unitary 21
  16. 16. Cerebrumentity, not something red + round + edible. knowledge and skill, a stricken bird watcherWhat is more, attributes are processed not says that all the birds look alike, a farmeronly in different locations but also at differ- can no longer distinguish his cows, and aent times in my brain. For example, color is gardener cannot tell one plant from another.perceived before motion, which is perceived Might synesthesia relate to the brain’sbefore form. How all of these sundry, search for constancy and the assignment ofasynchronous attributes get bound into a essential features that constitute a category?seamless perception—red apple—endlessly An enduring puzzle of neuroscience is how,baffles neuroscientists. Inasmuch as synes- out of a constantly changing and infinitethesia binds perceptual qualia together in energy flux, the brain—whose resources areanomalous combinations, might it not finite—assigns objects their constant features.say something useful about the process of Color and form, so prominent inbinding in general? synesthesia, are properties constructed by There is a further twist. I mentioned the brain through what are called constancyearlier that synesthesia’s most common operations. For example, most of us acceptmanifestation is a coupling of sensory qualia the explanation that something looks redto categories of knowledge: for example, because it reflects red wavelengths morecolor, flavor, texture, locus, or configura- than others, but color is actually a propertytion may be bound to letters and integers, of brains and not of the physical world. Formembers of a serially ordered set (such as surface colors to be perceived as constant despite ever-changing illumination, it is precisely the wavelength composition ofAn enduring puzzle of neuroscience reflected light that the brain must how, out of a constantly changing Grass looks green, whether it is in brightand infinite energy flux, the brain— sunlight or shade, despite large differences in wavelength composition of the light.whose resources are finite—assigns Similarly, all constructed properties requireobjects their constant features. that the brain discount certain things. With color, it is wavelength composition ofdays of the week), words, or even symbols reflected light that the brain must ignore;such as braille. Consider how many neuro- with form, it is the viewing angle; and withlogical syndromes (the agnosias) as well as size, it is viewing distance.imaging studies demonstrate that we think Synesthesia has led me over time toin categories. In prosopagnosia, for example, favor a model of brain organization calledstricken individuals can no longer recognize the distributed system. The prime featuresfaces. They recognize a face as a face, but of this model are a distribution of functioncannot say whose face it is. Their larger fail- (hence the name) across structures—as inure is in comprehending examples within neural networks—and simultaneity of activitya category. Thus, despite all their previous on several levels, compared to the older and 22
  17. 17. The Dana Forum on Brain Sciencemore familiar hierarchical and sequential a given cerebral module participates incascade, in which a module is assumed to more than one cognitive function and con-complete its transformation of neural inputs nects with several-to-many other nodes. Abefore passing the result on to the next given function is not so much localized inmodule in the sequence. This older idea the sense of classical neurology, but exists asmay be likened to stations in a factory the dominant process within its distributedconnected by a conveyor belt by means of system at any given time. Multiple synapticwhich one thing after another is added, levels are active simultaneously, each node influencing the state of adjacent levels (as in the example of our simultaneous authors).The answer to synesthesia will not Such organization reminds us that localiza-be a “where” but a “what.” tion is a function of probability—and not just in this model but in any scheme ofwhereas the distributed system is like neural organization. (Try drawing thedifferent authors simultaneously writing boundaries of Wernicke’s area on a standardseparate chapters of a book without fully brain atlas—you can’t.) Scans mislead usknowing how the other chapters end. The by emphasizing peak probabilities, whichdistributed system also departs from the we misconstrue as fixedly anatomical. Theolder idea of a strict one-to-one mapping answer to synesthesia will not be a “where”of function to anatomy, depending instead but a “what.”on topological relations and convergent- It would thus be wrong for me todivergent connections among brain modules. leave the impression that V4 is the seatThese two features result in the multiple of synesthesia: Any module found active bymapping of a given function, as seen in the a scan (or other means) is really just onenumerous modules pertaining to vision, node in the distributed system underlyingsome of which we understand better than expression. The totality of synesthetic expe-others (such as V4 for color, or V5 for rience involves more than the consciousmotion and direction). Relevant to synes- perception of a single quale, as I hope Ithesia, what are called transmodal modules have conveyed throughout this article. My(meaning “not pertaining to any single comments regarding the participation ofsense”) do three things: They construct transmodal modules in synesthesia are notmultisensory representations of the world, incompatible with the idea, mentionedthey provide memory and affect to experi- earlier, that an inherited genetic mutationence, and they critically participate in causes extraordinary, one-way projectionsestablishing categories via groups of coarsely between cerebral modules that underlietuned neurons. very specific functions. A connection This model organizes brain tissue between, say, the grapheme area that allowsinto five major networks and many lesser one to understand written numbers and thedistributed systems. In any one such system, V4 color area does not fully “explain” 23
  18. 18. Cerebrumsynesthetically colored numbers, however, heterogeneity in the depth of subjectivebecause it leaves out the affect of the experience, from purely sensory-sensory, toexperience, its memorability, whether the categorical-sensory, to verbal-sensory. Insynesthetic color moves, has a given spatial this last, even a concept—just thinking oflocation, and so forth. As V. S. Ramachan- the number 5, say, or a person nameddran points out, what are called transcription Marion—is sufficient to trigger synesthesia.factors can partly solve this shortcoming Some time ago, both Lawrence Marks andby causing the gene’s effects to be I proposed a cognitive continuum extend-expressed either discretely or diffusely—or ing from perception to synesthesia toanywhere between—in the brain. Such metaphor to language. With time, othersvariability goes a long way toward explain- have come to the observed variety of synesthetic Systematic correspondences existexperience, and why some people have among dimensions of a given sense foronly one kind whereas others have three synesthetes and nonsynesthetes alike. For example, both say that louder tones are brighter than soft tones, that higher onesI proposed a cognitive continuum are smaller than lower ones, and that lowextending from perception to tones are both larger and darker than highsynesthesia to metaphor to language. ones. The perceptual similarities that yield such orderly relationships among pitch,With time, others have come loudness, brightness, and size, for exam-to concur. ple, turn out to be rooted in fundamental similarities of physical experience itself.or four different kinds of synesthesia. Thus, Perceptual similarities, synesthetic equiva-transcription factors expressed in different lences, and metaphoric identities in turnplaces through-out the brain could account, become available to the more abstracttheoretically at least, for subsidiary features knowledge that is embodied in language.of synesthesia such as memorability and In other words, the acquisition ofaffective charge. But it is precisely this metaphor relies not on a capacity for ver-necessity of widespread expression that makes bal abstraction, as many mistakenlyme point out why synesthetic experience believe, but on our physical interactionper se cannot be localized to any one physical with the world. The subjective-objectivespot in the brain and why scans mislead us dichotomy of experience should be turnedin this regard. into a unity, because we need both points of view.M E TA PH O R A N D L A N G UAG E Objectivity fails to see how the humanThe heterogeneity of the synesthetic expe- system of concepts is metaphoric, involvingrience connotes more than wide variety an imaginative understanding of oneof perceptual combinations. There is also thing in terms of another. We elaborate the 24
  19. 19. The Dana Forum on Brain Science Synesthetes who see colored letters have their own individual alphabets. Carol Steen, who painted this representation of her alphabet, says she saw many of the more brightly colored letters as a young child, but the iridescent and metallic colored letters did not appear until she was in her 30s. Many synesthetes say that their perceptions become richer and more complex as they age.metaphor “The mind is an entity” into sense of fragmentary information, andanother metaphor, “The mind is a machine,” the unexpected suddenness of insight. Bywhen we say, “He ran out of steam.” switching metaphors, we alter how weMetaphors emphasize some aspects of an comprehend a thing.object but hide others. The machine Subjectivity fails to see that even themetaphor paints the mind as having a most imaginative flights occur in a contextsource of power, an on-off state, and an of objective experience gained by living inexpected level of efficiency, but it hides a physical and cultural world. Increasingly,the vagaries of thought, its ability to make science is viewing metaphor as an emergent 25
  20. 20. Cerebrumproperty of mind that is rooted in the tionally bound to discern form, movement,body. As semiotics have long known, direction, spatial location, and other qualiameaning inheres in affect, which the body that we conventionally ascribe to vision.feels as physical and the mind apprehends The capacity for anomalous binding, whichas mental. Because metaphor perceives is the essence of synesthesia, is thereforethe similar in the dissimilar, it also points latent in all constancy and categorization, features Nature reveals herself through excep-germane to synesthesia. Perhaps a tendency tions. Those objectivists who tried to dis-to map one concept to another unconven- miss synesthesia throughout its historytionally even underlies what appears seem to have forgotten this maxim. Farto be synesthetes’ distinctive approach to from being a mere curiosity irrelevant tocreativity. real questions, synesthesia turns out to illu- One implication of a continuum from minate a wide swath of mental life andperception to synesthesia to metaphor to forces us to rethink some fundamentallanguage is that synesthesia resides univer- issues regarding mind and brain. At pre-sally in each of us but, for reasons yet sent, I can think of nothing more relevantunknown, rises to consciousness in only a to our quest for self-understanding.few. Heinz Werner suggested as much in Referencesthe 1930s but technology takes time tocatch up with ideas. Two bits of recent Bogen, JE, Bogen, GM. “Wernicke’s region–Where is it?” Annals of the New York Academy of Science 1975; 280:work support this conjecture. One study 834-843. Cytowic, RE. The Man Who Tasted Shapes. Cambridge. MIT Press, 1998.Nature reveals herself through Cytowic, RE. Synesthesia: A Union of The Senses, 2d ed.exceptions. Those objectivists who Cambridge. MIT Press, 2002. Gray, JA, et al. “Possible questions of synesthesia fortried to dismiss synesthesia the hard question of consciousness.” In S Baron-Cohen & JE Harrison, eds., Synaesthesia: Classic and Contem-throughout its history seem to have porary Readings. Oxford. Blackwell, 1997: 173-181.forgotten this maxim. Luria, AR. The Mind of a Mnemonist. New York. Basic Books, 1968. Marks, LE. The Unity of The Senses: Interrelationsfound that synesthesia is 100 times more Among the Modalities. New York. Academic Press, 1978.frequent during Zen meditation; the other Nunn, JA, et al. “Functional magnetic resonance imag- ing of synesthesia: activation of V4/V8 by spokenconfirmed the ability of both blind and words.” Nature Neuroscience 2002; 5(4): 571-575.sighted persons to “see” video impulses Paulesu, E, et al. “The physiology of coloured-hearing:fed into an electrode array placed on a PET activation study of colour-word synaesthesia.” Brain 1995; 118: 661-676.the tongue. We do not see with our eyes, Ramachandran, VS, Hubbard EM. “Synesthesia:anyway, but with our brains. What this a window into perception, thought, and language.”latter demonstration shows is that tactile Journal of Consciousness Studies 2001; 8(12): 3-34.sensations on the tongue can be unconven- Zeki, S. A Vision of the Brain. Oxford. Blackwell, 1993. 26