Primate neuroethology

Primate Neuroethology

This page intentionally left blank

Primate NeuroethologyEdited byMichael L. PlattAsif A. Ghazanfar12010

1Oxford University Press, Inc., publishes works that furtherOxford University’s objective of excellencein research, scholarship, and education.Oxford New YorkAuckland Cape Town Dar es Salaam Hong Kong KarachiKuala Lumpur Madrid Melbourne Mexico City NairobiNew Delhi Shanghai Taipei TorontoWith offices inArgentina Austria Brazil Chile Czech Republic France GreeceGuatemala Hungary Italy Japan Poland Portugal SingaporeSouth Korea Switzerland Thailand Turkey Ukraine VietnamCopyright Ó 2010 by Michael L. Platt and Asif A. GhazanfarPublished by Oxford University Press, Inc.198 Madison Avenue, New York, New York 10016www. oup. comOxford is a registered trademark of Oxford University PressAll rights reserved. No part of this publication may be reproduced,stored in a retrieval system, or transmitted, in any form or by any means,electronic, mechanical, photocopying, recording, or otherwise,without the prior permission of Oxford University Press.Library of Congress Cataloging-in-Publication DataPrimate neuroethology / edited by Michael L. Platt and Asif A. Ghazanfar. p. cm. Includes bibliographical references and index. ISBN 978-0-19-532659-8 1. Primates—Nervous system. 2. Primates—Psychology. 3. Primates—Behavior. I. Platt, Michael L. II. Ghazanfar, Asif A. QL737. P9P67265 2010 599. 8—dc22 20090241159 8 7 6 5 4 3 2 1Printed in the United States of America on acid-free paper

ABOUT THE EDITORSMichael L. Platt is an Associate Professor of Neurobiology and Evolutionary Anthropology atDuke University, and Director of the Center for Cognitive Neuroscience. His research focuses onthe neuroethology and neuroeconomics of human and nonhuman primate behavior andcognition. Michael received his B.A from Yale and his Ph.D. from the University of Pennsylvania,both in biological anthropology, and was a post-doctoral fellow in neuroscience at New YorkUniversity.Asif A. Ghazanfar is an Assistant Professor in the Neuroscience Institute and Departments ofPsychology and Ecology & Evolutionary Biology at Princeton University. His research focuses onthe neurobiology and evolution of primate vocal communication and how both aspects are influencedby body morphology and socioecological context. Asif received his B.Sci. in Philosophy from theUniversity of Idaho and his Ph.D. in Neurobiology from Duke University. He was a postdoctoralfellow at Harvard University and a research scientist at the Max Planck Institute for BiologicalCybernetics in Tuebingen, Germany, before moving to Princeton.

CONTRIBUTORSLouise Barrett Asif A. GhazanfarDepartment of Psychology Neuroscience InstituteUniversity of Lethbridge Departments of Psychology and Ecology &Lethbridge, Alberta, Canada Evolutionary Biology Princeton UniversityElizabeth M. Brannon Princeton, NJCenter for Cognitive NeuroscienceDepartment of Psychology Herbert Gintis and Neuroscience Central European UniversityDuke University Budapest, HungaryDurham, NC Katalin M. Gothard Department of PhysiologyCharles Cadieu University of ArizonaRedwood Center for Theoretical Tucson, AZ NeuroscienceHelen Wills Neuroscience Institute Michael S. A. GrazianoUniversity of California Neuroscience InstituteBerkeley, CA Department of Psychology Princeton UniversityMatt Cartmill Princeton, NJDepartment of Evolutionary AnthropologyDuke University Medical Center Jennifer M. Groh Center for Cognitive NeuroscienceDurham, NC Department of Psychology and Neuroscience &Dorothy L. Cheney NeurobiologyDepartment of Biology Duke UniversityUniversity of Pennsylvania Durham, NCPhiladelphia, PA Brian HareYale E. Cohen Center for CognitiveDepartment of Otorhinolaryngology: Head and Neuroscience Neck Surgery Department of EvolutionaryUniversity of Pennsylvania School of Anthropology Medicine Duke UniversityPhiladelphia, PA Durham, NC vii

viii CONTRIBUTORSNicholas G. Hatsopoulos Daeyeol LeeDepartment of Organismal Biology & Anatomy Department of NeurobiologyCommittee on Computational Neuroscience Yale University School ofUniversity of Chicago MedicineChicago, IL New Haven, CTBenjamin Y. Hayden Margaret S. LivingstoneDepartment of Neurobiology Department of NeurobiologyDuke University Medical School Harvard Medical SchoolDurham, NC Boston, MAKari L. Hoffman Dario MaestripieriDepartment of Psychology Department of Comparative HumanYork University DevelopmentToronto, Ontario, Canada University of ChicagoWilliam D. Hopkins Chicago, ILDepartment of Psychology Julian A. MattielloAgnes Scott College Department of Surgery, Section ofDecatur, GA NeurosurgeryDivision of Psychobiology University of ChicagoYerkes National Primate Research Center Chicago, ILAtlanta, GeorgiaAtsushi Iriki Cory T. MillerLaboratory for Symbolic Cognitive Department of Psychology Development University of California, San DiegoRIKEN Brain Science Institute La Jolla, CASaitama, Japan Tirin MooreKarline Janmaat Department of NeurobiologyInstitute for Biodiversity and Ecosystem Stanford University School Dynamics of MedicineUniversity of Amsterdam Stanford, CAAmsterdam, the Netherlands Yuji NayaSarah M. Jones Center for Neural ScienceCenter for Cognitive New York University Neuroscience New York, NYDepartment of Psychology and NeuroscienceDuke University Andreas NiederDurham, NC Department of AnimalKerry E. Jordan PhysiologyDepartment of Psychology ¨ University of TubingenUtah State University ¨ Tubingen, GermanyLogan, UT Behrad NoudoostJon H. Kaas Department of NeurobiologyDepartment of Psychology Stanford University School ofVanderbilt University MedicineNashville, TN Stanford, CA

CONTRIBUTORS ixDinesh K. Pai Daniel SchmittDepartment of Computer Science Department of Evolutionary AnthropologyUniversity of British Columbia Duke UniversityVancouver, British Columbia, Canada Durham, NCMichael L. Platt Wolfram SchultzCenter for Cognitive Neuroscience Department of Physiology, Development andDepartment of Neurobiology NeuroscienceDuke University Medical Center University of CambridgeDurham, NC Cambridge, UKTodd M. Preuss Robert M. SeyfarthDivision of Neuroscience Department of PsychologyYerkes National Primate Research Center University of PennsylvaniaEmory University Philadelphia, PAAtlanta, GA Stephen V. ShepherdDrew Rendall Neuroscience InstituteDepartment of Psychology Princeton UniversityUniversity of Lethbridge Princeton, NJLethbridge, Alberta, Canada Jeffrey R. StevensLizabeth M. Romanski Center for Adaptive BehaviorDepartment of Neurobiology & Anatomy and CognitionUniversity of Rochester Max Planck Institute for HumanRochester, NY Development Berlin, GermanyAlexandra G. RosatiCenter for Cognitive Neuroscience Wendy A. SuzukiDepartment of Evolutionary Anthropology Center for Neural ScienceDuke University New York UniversityDurham, NC New York, NYOsamu Sakura Doris Y. TsaoInterfaculty Initiative in Information Studies Division of BiologyThe University of Tokyo California Institute of TechnologyTokyo, Japan Pasadena, CAMaryam Saleh Jonathan D. WallisDepartment of Organismal Biology & Anatomy Department of PsychologyCommittee on Computational Neuroscience University of California at BerkeleyUniversity of Chicago Berkeley, CAChicago, IL Yumiko YamazakiLaurie R. Santos Laboratory for Symbolic CognitiveDepartment of Psychology DevelopmentYale University RIKEN Brain Science InstituteNew Haven, CT Saitama, JapanRobert J. Schafer ¨ Klaus ZuberbuhlerDepartment of Neurobiology School of PsychologyStanford University School of Medicine University of St. AndrewsStanford, CA St. Andrews, Scotland

CONTENTS1. Introduction 3 Michael L. Platt and Asif A. Ghazanfar2. Primate Classification and Diversity 10 Matt Cartmill3. Primate Locomotor Evolution: Biomechanical Studies of Primate Locomotion and Their Implications for Understanding Primate Neuroethology 31 Daniel Schmitt4. Foraging Cognition in Nonhuman Primates 64 ¨ Klaus Zuberbuhler and Karline Janmaat5. Primate Vocal Communication 84 Robert M. Seyfarth and Dorothy L. Cheney6. Rational Decision Making in Primates: The Bounded 98 and the Ecological Jeffrey R. Stevens7. Primate Social Cognition: Thirty Years After Premack and Woodruff 117 Alexandra G. Rosati, Laurie R. Santos and Brian Hare8. Behavioral Signatures of Numerical Cognition 144 Elizabeth M. Brannon, Kerry E. Jordan and Sarah M. Jones9. The Foundations of Transdisciplinary Behavioral Science 160 Herbert Gintis10. Sensory and Motor Systems in Primates 177 Jon H. Kaas11. Vision: A Neuroethological Perspective 201 Benjamin Y. Hayden12. Circuits of Visual Attention 223 Tirin Moore, Robert J. Schafer and Behrad Noudoost13. Vocalizations as Auditory Objects: Behavior and Neurophysiology 237 Cory T. Miller and Yale E. Cohen xi

xii CONTENTS14. Encoding and Beyond in the Motor Cortex 256 Nicholas G. Hatsopoulos, Maryam Saleh, and Julian A. Mattiello15. Looking at Sounds: Neural Mechanisms in the Primate Brain 273 Jennifer M. Groh and Dinesh K. Pai16. Circuits of Emotion in the Primate Brain 292 Katalin M. Gothard and Kari L. Hoffman17. Neurophysiological Correlates of Reward Learning 316 Wolfram Schultz18. Associative Memory in the Medial Temporal Lobe 337 Yuji Naya and Wendy A. Suzuki19. Neurobiology of Social Behavior 359 Dario Maestripieri20. Neural Bases of Numerical Cognition 385 Andreas Nieder21. Executive Control Circuits 405 Jonathan D. Wallis22. Reinventing Primate Neuroscience for the Twenty-First Century 422 Todd M. Preuss23. Ethologically Relevant Movements Mapped on the Motor Cortex 454 Michael S. A. Graziano24. Object Recognition: Physiological and Computational Insights 471 Doris Y. Tsao, Charles F. Cadieu, and Margaret S. Livingstone25. The Primate Frontal and Temporal Lobes and Their Role in Multisensory Vocal Communication 500 Lizabeth M. Romanski and Asif A. Ghazanfar26. Neuroethology of Attention in Primates 525 Stephen V. Shepherd and Michael L. Platt27. Neuroethology of Decision Making 550 Daeyeol Lee28. Out of Our Minds: The Neuroethology of Primate Strategic Behavior 570 Louise Barrett and Drew Rendall29. The Comparative Neuropsychology of Tool Use in Primates with Specific Reference to Chimpanzees and Capuchin Monkeys 587 William D. Hopkins30. Evolution of an Intellectual Mind in the Primate Brain 615 Atsushi Iriki, Yumiko Yamazaki, and Osamu SakuraAuthor Index 633Subject Index 655

Primate Neuroethology

CHAPTER 1IntroductionMichael L. Platt and Asif A. GhazanfarMOTIVATION FOR THE BOOK Fernando Nottebohm, Nobuo Suga, and others has provided rich insights into the minds of aWhy do people find monkeys and apes so com- number of different nonhuman animals.pelling to watch? One clear answer is that they Research into the natural behavior of bats, forseem so similar to us, and thus perhaps provide a example, led scientists to discover that thesewindow into our own minds and how they have animals use the acoustic and timing differencesevolved over millennia. As Charles Darwin wrote between the sound of an emitted vocalizationin his Notebook M, ‘‘He who understands and its subsequent echo to identify and localizebaboon would do more toward metaphysicsthan Locke.’’ Such similarities notwithstanding, a target prey (Simmons, 1989). With this beha-Darwin recognized that behavior and cognition, vioral foundation, neuroscientists used the tem-and the neural architecture that support them, poral and spectral attributes of echolocationevolved to solve specific social and ecological signals to reveal the specialized functional orga-problems (Darwin, 1872). nization of the bat’s auditory cortex (Suga, Darwin, and later the pioneering ethologists 1990). A similar story holds for the electric fishKonrad Lorenz, Niko Tinbergen, and Karl Von (Heiligenberg, 1991). In the early 1960s, etholo-Frisch (who shared the Nobel Prize in 1973), gists discovered that certain species of fish emitargued that behavior must be understood in electrical discharges for locating salient objectsterms of its proximate causes, evolutionary ori- and can adaptively shift the frequency of thesegins, developmental sequence, and physiological discharges so that they do not interfere with theand anatomical mechanisms (Hinde, 1982). discharges of other fish. Once this ‘‘jammingDefining and operationalizing species-typical avoidance’’ behavior was characterized in morebehaviors for neurobiological study and con- detail, neuroscientists were able to anatomicallyveying neurobiological results to ethologists and physiologically map out the sensorimotorand psychologists are therefore fundamental to neural circuitry underlying it. Bats and electrican evolutionary understanding of brain and fish continue to be popular model systems pri-behavior. Neurobiological, psychophysical, and marily because of this strong link between nat-ethological perspectives must be integrated. ural behaviors and brain function.Unfortunately, behavioral scientists and neuro- In stark contrast, neuroscientists who investi-biologists rarely interact, and most practitioners gate the function and structure of primate brainsremain experts in their own fields but maintain often focus on more general cognitive processeslittle knowledge of the others. and neglect their species-typical behaviors. The ‘‘neuroethological’’ approach envisioned Psychological and neurobiological studies of pri-by Darwin, pioneered by the European etholo- mates typically require them to discriminategists, and finally refined by modern neurobiolo- simple stimuli whose salience or behavioral sig-gists and biologists like Walter Heiligenberg, nificance is arbitrarily assigned. Observational 3

4 PRIMATE NEUROETHOLOGYstudies of primates in the wild, however, demon- biology in chapters written by the foremoststrate that primates (like bats, electric fish, and experts in the field. The book is roughly orga-numerous other species) are not ‘‘generalized’’ nized into three sections. The first sectionprocessors of information. Instead, specific sti- reviews our current understanding of key issuesmuli, such as the facial gestures or vocalizations in primate taxonomy, behavior, and cognition.of others in their groups, are intrinsically salient, The second section reviews recent advances inattract attention, and evoke species-typical our knowledge of the neural mechanisms under-responses. Many of these behaviors are shared lying perception, motor control, and cognition.by both human and nonhuman primates, The final portion of the book covers work thatalthough they may vary according to social struc- explicitly attempts to bring together species-ture, habitat, mating systems, and developmental typical behavior and neurobiology—work thatprocesses. Ignoring the species-typical behavior represents a new wave of neuroethologicalof primates leads to the potentially erroneous research on primate behavior and cognition.idea that all primate brains are essentially Our hope is that this synthesis will set the stagedifferent-sized versions of the same basic plan. for an interdisciplinary dialogue between inves-A more promising, and biologically realistic, tigators on either side of the behavior–biologyway to examine the neural bases of primate beha- divide.vior would be to move beyond measures of brain The first section begins with a discussion ofsize or neocortex size and investigate the anatomy current understanding of primate phylogeny byand physiology of particular brain structures as Cartmill. His thesis is that a cladistic approachthey relate to species-typical behaviors. That is, based on genetics, supplemented by morpholo-we must develop a neuroethology of primate gical and behavioral data, offers unique promisebehavior and cognition. for organizing relationships among living pri- The goal of this book is to do just that. Our mates as well as their pattern of descent from aaim is to bridge the epistemological gap between common ancestor. Understanding evolutionaryethologists and neurobiologists who study pri- relationships within the Order Primates is a keymates by collecting, for the first time in a single starting point for the comparative study of pri-book, both basic and cutting-edge information mate behavior and neurobiology. This chapter ison primate behavior and cognition, neuro- followed by an in-depth review of primate loco-biology, and the emerging discipline of neu- motion by Schmitt, who argues that primatesroethology. In this volume, leading scientists in (including humans) show patterns of locomo-several fields review work ranging from primate tion and locomotor control that are differentforaging behavior to the neurophysiology of from all other mammals. Schmitt argues thatmotor control, from vocal communication to changes in limb function associated with thethe functions of the auditory cortex. Written by adaptive diversification of locomotor patternssome of the foremost experts in these fields, we in the primate clade probably required the evo-hope this book will serve as an important lution of profound specializations in the neuralresource for the professional and the student control of locomotion. Most of these putativealike. The resulting synthesis of cognitive, etho- specializations remain unknown or unexplored.logical, and neurobiological approaches to pri- This realization suggests that comparative stu-mate behavior yields a richer understanding of dies of the neuroethology of locomotion in pri-our primate cousins that also sheds light on the mates may offer unique insights into motorevolution of human behavior and cognition. control, and such insights may have implications for fields as diverse as robotics and the clinical treatment of paralysis with brain–machine inter-ORGANIZATION face devices. Following this discussion of primate locomo-This book brings together the latest information ¨ tion, Janmaat and Zuberbuhler review recenton primate behavior, cognition, and neuro- studies of primate foraging behavior in the

INTRODUCTION 5wild. The authors suggest that the information- neuroscience. He argues that, although theseprocessing problems primates encounter in models are often incompatible, they can be ren-foraging, particularly searching for ripe edible dered more coherent by incorporating corefruits, may have provided the impetus for the principles that include an evolutionary perspec-evolution of enhanced cognitive skills such as tive. Together, the chapters in the first section ofcognitive mapping and forecasting the ripeness the book clearly endorse the notion that under-of fruits based on the recent history of weather. standing the neurobiology of primate behaviorNext, Cheney and Seyfarth review our under- and cognition will profit from an evolutionarystanding of vocal communication in primates. and ethological approach.They contend that primate communication calls The second section of the book reviews ourconvey information about both the caller’s affec- knowledge of the brain circuits that underlietive state and objects and events in the world. behavior and cognition in human and non-Crucially, this mixed referential signaling human primates. First, Kaas outlines the majormechanism appears to be fundamentally social organizational features of the sensory and motorin nature and thus crucial for the representation systems in primates. Comparison of these sys-of goals, intentions, and knowledge. tems with respect to other mammals suggests Synthesizing the prior chapters on foraging their likely organization in ancestral primatesand vocal communication, Stevens reviews deci- and reveals an adaptive diversity in extant pri-sion-making behavior in primates. Evidence mates that is unique among mammals. Haydenfrom both human and nonhuman primates builds on this comparative analysis of sensorydemonstrates that decision makers often fail to and motor systems with a detailed neuroanato-behave as predicted by economic principles of mical and neurophysiological account of visualrational utility maximization. Based on this evi- processing in the primate brain. Hayden con-dence, Stevens contends that understanding tends that consideration of the natural require-decision making and its underlying mechanisms ments for detecting and identifying behaviorallywill be most successfully advanced by an evolu- meaningful stimuli such as insects, fruits, andtionarily informed framework termed ‘‘bounded the facial identities and expressions of otherand ecological rationality,’’ which emphasizes individuals likely played an important role inthe match between decision mechanisms and the evolution of visual processing and, by exten-the natural environment. Central to under- sion, the evolution of cognition in primates.standing the mechanisms underlying decision Moore and Noudoost focus on one salientmaking is defining the role of intentionality. aspect of visual behavior with a review of theRosati, Santos, and Hare review evidence that neural mechanisms mediating selective visualmonkeys and great apes understand the psycho- attention. They describe a body of evidencelogical states of others. They conclude that some that strongly implicates specific neural circuitsapes, and perhaps some monkeys as well, under- in controlling visual orienting behavior.stand behavior in terms of goals, intentions, and Following this discussion of vision and atten-even knowledge. tion, Miller and Cohen describe our under- Next, Brannon, Jordan, and Jones provide standing of how the primate brain parsescompelling evidence for a homologous cognitive vocalizations as auditory objects. The authorsand neural system supporting numerical argue that in the primate auditory system, evo-approximation in lemurs, monkeys, apes, and lution selected for those neural mechanisms thathumans. This approximate number system bind the acoustic features of vocalizations intoappears to form the backbone upon which sym- behaviorally meaningful units that can be actedbolically mediated numerical computation and upon, just like objects in the visual domain.mathematical operations are built in humans. Hatsopoulous, Saleh, and Mattiello build onFinally, Gintis reviews contemporary models of the preceding reviews of sensory mechanismshuman behavior in various fields including eco- with a review of the neural mechanisms under-nomics, biology, anthropology, sociology, and lying the production of movement. Based on

6 PRIMATE NEUROETHOLOGYcurrent evidence, they contend that motor neural mechanisms mediating species-typicalcortex does not encode any simple physical social behavior. He concludes that neuromodu-variable like velocity or direction but rather latory influences on specific neural circuits,encodes elementary action fragments that can including amygdala, orbitofrontal cortex, andbe assembled into simple behaviors. This hippocampus, underlie specific patterns ofhypothesis is consistent with a neuroethological affiliation, dominance behavior, and socialapproach, which predicts that neural mechan- tolerance in different species and among indivi-isms will be organized to produce adaptive duals within a species.behavior rather than follow arbitrary physical The last two chapters of the second section ofprinciples. Groh and Pai take this discussion the book cover aspects of primate cognition thatone step further by looking at the neural are often assumed to be uniquely human. First,transformations that mediate sensory-motor Nieder reviews the evidence that neurons in theintegration. They suggest that the brain parietal cortex and prefrontal cortex selectivelytransforms head-centered auditory information encode the numerical values of objects or eventsinto a rate-coded format anchored to a hybrid in the environment. The tuning properties ofreference frame that is suitable for guiding these neurons directly parallel the psychophy-movements of the eyes, but may also permit sical properties of numerical judgments.extrapolation of sound location for guiding Moreover, many of these neurons also appearother types of movements. to encode spatial extent—thus suggesting a The foregoing review of sensory and motor single cortical system dedicated to representingsystems is followed by reviews of the neural approximate quantity derived from multiplemechanisms underlying cognitive processes features of particular stimuli or events. Finally,including emotion, reward, memory, social Wallis examines the role of prefrontal cortex inbehavior, numerosity, and executive control. controlling complex, flexible behavior. He con-First, Gothard and Hoffman examine the cludes that homologous mechanisms mediateneural circuits that process emotion in the pri- the abstraction of rules, strategies, and task setsmate brain. They argue that two nested cortical in human and nonhuman primates, and arguesand subcortical circuits mediate emotional eva- that this system likely evolved in concert withluation of behaviorally meaningful stimuli. increasing behavioral complexity in primates.Although these systems are shared with other The final section of the book sketches anmammals, they appear to be further specialized outline of the neuroethology of primatefor social behavior in primates. Schultz builds on behavior and cognition. This portion of thethis discussion by reviewing the neural mechan- book is the most speculative, but builds uponisms underlying reward learning in primates. He the firm foundations of behavioral descriptionargues that structures involved in learning and and basic neurobiology described in thereward, particularly midbrain dopamine neu- preceding sections of the book. The chapter byrons, appear to calculate the difference between Preuss advocates an evolutionary approach toexpected and received rewards. This prediction understanding comparative brain anatomy inerror signal appears to be fundamental to primates. He argues that deep understanding oflearning and decision making in primates and the relationships between brain and behaviorother mammals. Following these discussions of requires determining how evolution modifiesemotion and reward, Naya and Suzuki review specific systems of neurons and their intercon-the role of the medial temporal lobe in associa- nections, and not just relating brain size to grosstive memory. They argue that parallel, but dis- measures of cognition or behavior. Such neu-tinct, mechanisms mediate the formation of roethological studies will require active manage-long-term associations between stimuli versus ment of captive and wild populations ofassociations between stimuli and action. primates needed for detailed comparison.Synthesizing the discussions of emotion, Following this charge, Graziano reviewsreward, and memory, Maestripieri reviews the evidence that motor cortex in primates is not

INTRODUCTION 7organized according to topographic maps mediated by relatively simple rules that use therelated to the body surface, but is organized structure of the social environment as a scaffold.according to species-typical motor behavior. This is in opposition to the notion that socialHe finds that microstimulation with behavio- knowledge must be explicitly represented byrally relevant time courses evokes basic move- specialized neural circuits. The complexity ofments such as bringing food to the mouth, the social environment is, in essence, an emer-climbing, or defensive responses. He concludes gent property of these simple rules of socialthat primate motor cortex serves as an interface engagement.functionally specialized for producing species- The final two chapters of the book examinetypical actions. Tsao, Cadieu, and Livingstone the ability of primates to use tools. Hopkinsthen take a neuroethological approach to object reviews behavioral and neurobiological data onand face recognition. They argue that the specia- tool use in primates. He finds that great apes, inlization of the primate brain for identifying and particular chimpanzees, excel at tool use—espe-assigning meaning to faces—a ubiquitous and cially generalizing principles to new tool-usingsalient social stimuli—may provide a roadmap tasks and contexts. He finds limited evidencefor understanding how the primate brain iden- that monkeys, even highly manual species suchtifies and extracts information about other types as capuchins, do so as readily. Hopkins arguesof objects. Synthesizing what we know about that this behavior is strongly associated withface and voice processing in the primate brain, neuroanatomical changes that include theRomanski and Ghazanfar argue that under- expansion of the cerebellum and interhemi-standing the evolution of human communica- spheric connectivity. Finally, Iriki, Yamazaki,tion requires the recognition that and Sakura review neurophysiological studiescommunication is fundamentally multimodal of how primates learn to extend their actionsin nature. They review behavioral, anatomical, with tools. They find that learning to use toolsand neurophysiological data to support this con- modifies not only the response properties oftention. Shepherd and Platt review the neural neurons involved in motor planning and sen-mechanisms underlying social attention in pri- sory-motor transformation, but also their ana-mates. They suggest that the neural systems tomical connections. Moreover, they contendmediating visual orienting behavior are that tool learning prepares and adapts the pri-intrinsically sensitive to social cues in the mate brain to learn more complex combinatorialenvironment, thereby promoting the adaptive tool use techniques. The authors speculate thatacquisition of behaviorally relevant social tool use learning in primates may provide theinformation. scaffolding upon which other more complex Lee follows these sensory-level discussions aspects of cognition are built.with a review of the neuroethology of decisionmaking in primates. He argues that primatesevolved more complex decision-making circuits THE WAY FORWARDto deal with the increasing complexitiesassociated with social interactions. He reports The primate brain is not a generalized informa-evidence that primates can treat interactions tion-processing device, or simply a differentlywith a computerized opponent strategically, a scaled version of a prototypical mammalianbehavior that requires the representation of cur- brain. Our hope is that the juxtaposition ofrent goals as well as prior rewards and prior these various ideas from the ethological, cogni-actions. Neurons in the prefrontal cortex, in tive, and neurobiological literature will lead toparticular, seem to encode these variables. the recognition that we cannot understand theBarrett and Rendall present an alternative view evolution of primates (and humans, in parti-to the notion that complex social behaviors in cular) without understanding the sophisticatedprimates require complex brain processes. They relationships between species-typical behaviorsargue that social behavior in primates may be and neural processes.

8 PRIMATE NEUROETHOLOGY Recognition of this relationship leads to repertoire of manual behaviors. Unlike otherseveral ideas about the way forward. We offer a New World monkeys, but much like thefew of these ideas here. First and foremost (and macaque monkey, Cebus monkeys possess aas illustrated in our cover illustration), the Order proprioceptive cortical area 2 and a well-Primates is a very diverse taxon, comprising developed area 5, which is associated withnumerous different species with different motor planning and the generation of internalhabitats and social systems. Unfortunately, very body coordinates necessary for visually guidedfew primate species are used as subjects in reaching, grasping, and manipulation (Padbergbehavioral and neurobiological studies. That is, et al., 2007). These types of data suggest thatthe ‘‘comparative approach’’ has not been taken parallel evolution of brain areas and behaviorsto heart, and without it, we cannot accurately can be driven (or at least paralleled) by changesconstruct the evolution of any particular trait or in body morphology (Rose, 1996).identify what is unique to our species (Preuss, Finally, a real synthesis of the emerging ideas2000). from ethology and neurobiology will require Another omission in most discussions of the better experimental paradigms for the latter. Asevolution of primate brains and behaviors is it currently stands, most primate neurophysio-the role of development. To understand the logical studies are carried out under conditionsevolutionary origins of a phenotype, we must of restraint while the subjects view static presen-understand the relationship between ontoge- tations of stimuli, trial after trial. This, of course,netic and phylogenetic processes (Gottlieb, is nothing like the real world. Future studies will1992; Gould, 1977). This relationship can get around this artificiality (at least partially) ininform questions about homology and help two ways. First, the use of interactive paradigmsdetermine whether putative homologies reflect between two (albeit restrained) primates holdsthe operation of the same or different mechan- great promise for understanding the neural basesisms (Schneirla, 1949). Are the developmental of social interactions, including dominanceprocesses leading to the emergence of particular interactions (Fujii et al., 2008). Related to this,behaviors similar or different across primate a second method of simulating dyadic interac-species? The answer will likely determine to tions is through the use of synthetic agents,what extent homologies at the neural level either computer animations or robots. Thesemake sense. For example, the rate of neural afford the experimenter the ability to controldevelopment in Old World monkeys and one side of the social interaction (and thushumans differs considerably—all sensorimotor explore experimentally different questions in atracts are heavily myelinated by 2 to 3 months tightly controlled manner). Finally, telemetricafter birth in rhesus monkeys, but not until 8 to technology is getting increasingly more refined,12 months after birth in human infants. These allowing the recording of multiple channels ofdifferences are paralleled at the behavioral neural signals remotely with lightweight, andlevel in the emergence of species-specific long-lasting, battery packs (Obeid et al., 2004).motor, socioemotional, and cognitive abilities This allows the monitoring of free-ranging(Antinucci, 1989; Konner, 1991). primates in a limitless variety of scenarios. The Between brains and the environment, there is added realism of these emerging techniquesa body. How the body shapes brain processes offers great promise for full realization of anand vice versa during development and integrated, evolutionarily motivated neu-experience with the environment is almost roethology of primate behavior and cognition.completely ignored (see Schmitt, this volume,for an exception). For example, whereas other ACKNOWLEDGMENTSNew World monkeys are not very dexterous andpossess a poorly developed area 5, Cebus mon- Many people were instrumental in the translationkeys are the only New World primate known to of a germ of an idea into a full-fledged volume onuse a precision grip, and thus have an extended primate neuroethology. This project began as a

INTRODUCTION 9symposium at the Society for Neuroscience Gottlieb, G. (1992). Individual development &(SFN) meeting in Washington, DC, in 2005. evolution: The genesis of novel behavior.Our editor, Catherine Carlin, eagerly solicited New York: Oxford University Press.this book from us at that meeting (and later Gould, S. J. (1977). Ontogeny and phylogeny.sealed the deal with fine wine, food, and conver- Cambridge, MA: Belknap-Harvard. Heiligenberg, W. (1991). Neural nets in electric fish.sation at the following SFN meeting in Atlanta). Cambridge, MA: MIT Press.Of course, the members of our labs contributed in Hinde, R. A. (1982). Ethology: Its nature and relationsmany ways to the ideas guiding this project, and with other sciences. New Zealand: Harper Collins.also read and commented on various chapters. Konner, M. (1991). Universals of behavioralFor this, Michael extends his gratitude to Robert development in relation to brain myelination.O. Deaner, Jamie Roitman, Karli Watson, Jeffrey In: K. R. Gibson & A. C. Petersen (Eds.), BrainKlein, Heather Dean, Alison McCoy, Michael maturation and cognitive development:Bendiksby, Sarah Heilbronner, Rebecca Ebitz, Comparative and cross-cultural perspectivesJohn Pearson, Arwen Long, and Amit Khera. (pp. 181–223). New York: Aldine de Gruyter.Michael also thanks his colleagues for their Obeid, I., Nicolelis, M. A. L., & Wolf, P. D. (2004).strong support and encouragement, including A low power multichannel analog front end for portable neural signal recordings. Journal ofDale Purves, Larry Katz, Rich Mooney, David Neuroscience Methods, 133, 27–32.Fitzpatrick, Scott Huettel, and Ron Mangun. Padberg, J., Franca, J. G., Cooke, D. F., Soares, J. G.Asif expresses his gratitude to his lab, including M., Rosa, M. G. P., Fiorani, M., et al. (2007).Joost Maier, Chandramouli Chandrasekaran, Parallel evolution of cortical areas involved inHjalmar Turesson, and Stephen Shepherd, as skilled hand use. Journal of Neuroscience,well as to his mentors (in the temporal order of 27, 10106–10115.their impact), Matthew Grober, Miguel Nicolelis, Preuss, T. M. (2000). Taking the measure ofMarc Hauser, and Nikos Logothetis. diversity: Comparative alternatives to the model-animal paradigm in cortical neuroscience. Brain Behavior and Evolution, 55, 287–299. Rose, M. D. (1996). Functional morphologicalREFERENCES similarities in the locomotor skeleton ofAntinucci, F. (1989). Systematic comparison of miocene catarrhines and platyrrhine monkeys. early sensorimotor development. In: Folia Primatologica, 66, 7–14. F. Antinucci (Ed.), Cognitive structure Schneirla, T. C. (1949). Levels in the psychological and development in nonhuman primates capacities of animals. In: R. W. Sellars, (pp. 67–85). Hillsday, NJ: Lawrence Erlbaum V. J. McGill, & M. Farber (Eds.), Philosophy for Associates. the future (pp. 243–286). New York: Macmillan.Darwin, C. (1872). The expression of the emotions in Simmons, J. A. (1989). A view of the world through animals and man. London: John Murray. the bat’s ear: The formation of acoustic imagesFujii, N., Hihara, S., & Iriki, A. (2008). Social in echolocation. Cognition, 33, 155–199. cognition in premotor and parietal cortex. Suga, N. (1990). Biosonar and neural computation Social Neuroscience, 3, 250–260. in bats. Scientific American, 262, 60–68.

CHAPTER 2Primate Classification and DiversityMatt CartmillA TAXONOMY OF imposed by Linnaeus’s own creationist assump-CLASSIFICATIONS tions. We have no reason for thinking that milk glands take precedence over feet in the mind ofThere are basically two ways of grouping things: God.by their properties or by their connections. The underlying justification for LinnaeanDescriptive groupings are defined by shared classification became apparent when historicalproperties of their members. An example is the and causal linkages were introduced into theclass of diamonds, which includes all and only system after the emergence of Darwinism in thethose objects composed mainly of carbon atoms mid-1800s. Linnaeus’s nested sets were real enti-arranged in a cubical crystal lattice. Historical ties because they corresponded to successivegroupings, by contrast, are defined by causal branching points on the phylogenetic tree. Milklinkages among their members—for example, glands were relevant properties for defining thethe class of all the ancestors of GeorgeWashington, from the Precambrian down to 1758 class Mammalia because they were inher-Washington’s parents. These are different kinds itances from a common ancestor shared by allof sorting criteria. Diamonds are not connected mammals and by no other organisms. Glabrouswith each other in any way, and Washington’s skin and fins—ancient vertebrate traits lost inancestors have no special properties in common the ancestors of the first mammals, and subse-apart from those shared with other organisms. quently re-evolved in the Cetacea as special Biologists have tried classifying organisms by adaptations to life in the sea—did not reflecttheir observable properties, by their genealogical the geometry of evolutionary relationships, andconnections, and by a mixture of both. Each therefore were not relevant properties forapproach has both merits and defects. The defining a taxon (the 1735 class Pisces).chief difficulty with a purely descriptive system For a century after Darwin, biological classi-is that each descriptor defines a different set of fication was dominated by mixed systems thatorganisms. These sets overlap, and it is not clear combined phylogenetic and essentialist criteria.why some descriptors should have priority over In such systems, sometimes referred to as evolu-others in making sequential cuts. For example, tionary systematics, taxa were defined by thein the first (1735) edition of Systema Naturae, acquisition of key evolutionary novelties.Linnaeus classified the whales as fish because Within a taxon, subtaxa were distinguishedthey had glabrous skin and lacked feet; but in from each other by clusters of shared adaptivethe canonical 10th edition of 1758, he reclassi- novelties that evolved later than the key traitsfied them as mammals because they had milk defining the larger taxon. One of these subtaxaglands and bore live young. This correction usually constituted a basal ‘‘wastebasket taxon’’seems warranted to us, but there was no way of comprising early, little-differentiated membersjustifying it in the purely descriptive framework of the larger taxon, together with later forms 10

PRIMATE CLASSIFICATION AND DIVERSITY 11that had remained persistently primitive. Such uniquely shared derivation) and homoplasieswastebasket taxa included Amphibia within (parallelisms, convergences, and coincidencesTetrapoda, Reptilia within Amniota, Prosimii that constitute noise in the phylogeneticwithin Primates, and so on. signal), using criteria of maximum parsimony In these mixed systems, taxonomic practice (minimizing the number of assumed changeswas constrained by phylogenetic or historical from primitive to derived states) or maximumcriteria as well. Taxa were usually required to probability. This process generates hierarchicallybe ‘‘monophyletic,’’ which meant that they arranged groupings defined by nested synapo-were supposed to contain only descendants of morphies. Each group is required to be holophy-the first species having the taxon’s defining letic, encompassing all and only the descendantsproperties. The lower boundary of each taxon of its LCA. Such groupings are called clades.was defined by the point of acquisition of those Wastebasket taxa, which contain only theproperties on the lineage leading to its last LCA’s descendants but not all of them, arecommon ancestor (LCA). If the key properties termed paraphyletic and are not admitted towere acquired independently in two separate the system. All taxa are defined exclusively bylineages, the descriptive grouping that they synapomorphies, and every grouping must bedefined was ‘‘polyphyletic.’’ Such taxa were gen- distinguished from its nearest relative—itserally forbidden, though some systematists ‘‘sister group’’—by at least one synapomorphyaccepted low levels of polyphyly (Simpson, represented in the sister by a more primitive1945, 1961). state of the same character. The nonintersecting During the 1950s and ’60s, the primacy of sets generated by this sort of analysis can be readthis mixed approach was challenged by two out either as hierarchical Linnaean classifica-rivals, one of which ultimately swallowed the tions or as atemporal branching tree diagramsother and replaced evolutionary systematics in known as cladograms. The classification is iso-a classical Kuhnian paradigm shift (Cartmill, morphic with the cladogram.1999). The defeated challenger was a strictlydescriptive system known as ‘‘numerical tax-onomy’’ (Sokal & Sneath, 1963), which excluded SOME PROBLEMS WITHphylogeny as a classificatory criterion on the not CLADISTICSunreasonable grounds that it is not an obser- The theoretical rigor and fascinating technicalvable property. Numerical taxonomists began intricacies of the cladistic approach have helpedby enumerating the species included in the clas- to give it a position of unchallenged dominancesification and then analyzing their properties in modern systematic practice. Apart from itsinto characters with varying states. The resulting esthetic appeal, cladistics has contributed tocharacter-state matrix was processed to generate biology by bringing the logic of phylogenetictaxonomic groupings that maximized the overall reconstruction into sharper focus. But a strictlysum of shared intrataxon resemblances. phylogenetic systematics encounters pro- The victorious challenger and currently reg- blems—in theory, in systematic practice, and innant style in systematics, known as cladistics or practical utilization.phylogenetic systematics (Hennig, 1966), groups The deepest theoretical problem is that someorganisms solely on the basis of phylogenetic organisms are in fact wholly primitive relative torelationships. The determination of those rela- others, and thus lack defining apomorphiestionships begins with a character-state analysis (nonprimitive traits). Therefore, they cannot beof the species in the group being classified, as in classified. For example, all character states in annumerical taxonomic practice. All character ancestral species are by definition primitive rela-states that are primitive within the group (sym- tive to other states of the same characters in itsplesiomorphies) are discarded as irrelevant to the descendants. This problem is particularly irk-determination of phylogeny. The remainder are some for paleontologists, who occasionally dis-sorted out into synapomorphies (traits of cover extinct organisms that left recognizable

12 PRIMATE NEUROETHOLOGYdescendants. Cladistic systematists tend to deal ‘‘absent,’’ it counts as a synapomorphy linkingwith this difficulty by seeking unique apomor- anthropoids to tarsiers (Cartmill, 1980;phies in apparently primitive organisms that Cartmill & Kay, 1978). But if we score it asexclude them from an ancestral status, or by ‘‘complete’’ versus ‘‘incomplete’’ (Beard &sneaking wastebasket taxa back into the system MacPhee, 1994), the septum counts as a uniquein various ways—erecting unranked taxa called synapomorphy of the anthropoids that distin-‘‘plesions,’’ referring to primitive extinct forms guishes them from tarsiers, thereby skewing theinformally as ‘‘stem groups,’’ or using old was- analysis in the opposite direction. Neithertebasket-taxon names like ‘‘reptile’’ inside quo- dichotomization seems obviously preferable.tation marks to show that they are being Analyzing the character into multiple statesnaughty. arrayed in a linear or branching transformation Another deep problem involves the identifi- series involves arbitrary choices of other sorts.cation of morphological characters and char- The weighting of morphological charactersacter states. The words we use to describe also raises serious questions for cladistic prac-morphology are not objectively determined by tice. Most cladistic methodologies apply equalnature. Choosing such words involves what can weighting to all characters, so that the differenceonly be called an act of poetic imagination; and between (say) ‘‘tail long’’ and ‘‘tail short’’ countsdifferent terminologies with similar information as much in judging phyletic affinities as thatcontent can yield different cladograms when between ‘‘six cervical vertebrae’’ and ‘‘seven cer-they are fed through the machineries of cladistic vical vertebrae.’’ But variation in some charac-analysis (Cartmill, 1982, 1994a). In primate sys- ters is known to be far more significant than intematics, a simple example of this sort of pro- others. Among mammals, the number of caudalblem is provided by the postorbital septum, a vertebrae is highly variable, but the number ofbony partition separating the orbital contents cervical vertebrae is virtually invariant andfrom the chewing muscles in monkeys and apes appears to be controlled by regulatory genes(Anthropoidea). The small carnivorous Asian acting at a much deeper level early in the processprimates called tarsiers have a septum with a of segmentation. Interspecies differences in thegap in it. Other primates lack the septum number of neck vertebrae should therefore(Fig. 2.1). Not having a septum is primitive. If count much more heavily in assessing groupwe score the septum as ‘‘present’’ versus affinities than differences in the number of tailFigure 2.1 The inside of the right orbit in three primates: semidiagrammatic front views. In the primitivecondition (Galago, A), the frontal, zygomatic, and maxillary bones (f, z, m) form a postorbital bar. In theanthropoid arrangement (Saimiri, C), outgrowths from the alisphenoid (a) and zygomatic combine withflanges of the maxillary and frontal bones to form a complete bony postorbital septum separating the orbitalcontents from the temporal fossa. The condition in Tarsius (B) is intermediate. L, lateral orbital fissure.From Cartmill, M., & Smith, F. H. (2009). The human lineage. New York: Wiley-Blackwell. Used withpermission.

PRIMATE CLASSIFICATION AND DIVERSITY 13vertebrae. But how much more? We can feel parasitized locus as a permanent marker of acertain that the arbitrary imposition of equal unique character-state change; and (3) parallelweighting here is a mistake that introduces mutations to the derived character state anderror into the reconstruction, but alternative reversions to the primitive character state areweightings seem equally arbitrary. When the vastly less likely than they are in the case ofunknown errors due to arbitrary weighting are single-nucleotide mutations.multiplied across the hundreds of characters Molecular analyses of mammalian phylogenycommonly tallied up in morphological char- have revealed some highly corroborated cladesacter-state matrices, it is hard to have much that had gone undetected by morphologistsconfidence in the details of the resulting (Fig. 2.2). Primitive eutherian (placental) mam-cladogram. mals first appear in the Lower Cretaceous of Asia around 125 million years ago (Mya), and were present in both Asia and North America by 110WHALES AND HIPPOS AND Mya (Ji et al., 2002). Molecular-clock analysesCOWS, OH MY suggest that the divergence of the extantMany of these problems can be obviated by eutherian orders dates back to about this timegiving up on morphology and going directly to (Eizirik et al., 2001; Kumar & Hedges, 1998;the genome. Although identifying homologous Murphy et al., 2001). The initial split seems toparts of different genomes is not a simple matter, have been between the South American eden-the conceptual basis of homology is clearer for tates (Xenarthra) and all other eutherians, fol-DNA data than it is for morphological data lowed by the divergence of an Africa-based(Cartmill, 1994b), and the boundaries of char- supraordinal clade (Afrotheria) comprising theacters (nucleotide positions in a homologous elephants, sea cows, hyraxes, tenrecs, and somesequence) and character states (the four nucleo- other originally African groups. The remainingtides) are unambiguous. Over the course of the eutherian orders form a clade (Boreotheria) withpast two decades, DNA data have increasingly two major subdivisions: (1) primates and relatedsupplanted phenotypic data in reconstructing groups plus the rodents and rabbitsthe phylogenetic relationships of living organ- (Euarchontoglires), and (2) everybody elseisms. For the most part, phylogenies inferred (Laurasiatheria, including carnivores, ungulates,from the genome have corroborated those whales, shrews, hedgehogs, and bats).inferred from the phenotype; but some intract- Parts of some of these clusters had beenable disputes of long standing have been decisi- glimpsed by morphologists and paleontolo-vely resolved, and there have been surprises. gists—for example, the primate-treeshrew- The use of SINEs (‘‘Short Interspersed colugo group (Archonta: Gregory, 1910) andNuclear Elements’’) as lineage markers has the elephant-seacow-hyrax group (Paenungu-enhanced the consistency and reliability of mole- lata: Simpson, 1945)—but even in these casescular phylogenetic analyses (Cook & Tristem, the affiliations of the tenrecs and the rodent-1997; Shedlock et al., 2004). A SINE can be rabbit group (Glires) were unexpected. Perhapsthought of as a transfer RNA molecule that has the biggest surprise was the deeply imbeddedbecome parasitic by introducing a retrotran- position of the whales and dolphins (Cetacea)scribed DNA copy of itself into the nuclear within the even-toed hoofed mammals or Artio-DNA. The introduced sequence is capable of dactyla, where they fall out as the sister group ofmaking new RNA copies, which then reproduce hippopotamuses and as successively more dis-themselves at new target loci. SINE insertions are tant relatives of ruminants-plus-pigs and camels.particularly reliable markers of lineage relation- The molecular identification of the cetaceans asships for three reasons: (1) the primitive state at artiodactyls (Arnason et al., 2000; O’Leary &the parasitized locus is always ‘‘SINE absent’’; Gatesy, 2007) has since been corroborated by(2) the template DNA copy is not excised during the discovery of early fossil whales that retainedreplication, so that it remains indefinitely at the hindlimb bones with distinctively artiodactyl

14 PRIMATE NEUROETHOLOGYFigure 2.2 Phylogenetic relationships of major clades of extant eutherian mammals, as inferred fromretroposon insertions and other molecular data. Based on Kriegs, J., Churakov, G., Kiefmann, M.,Jordan, U., Brosius, J., & Schmitz, J. (2006). Retroposed elements as archives for the evolutionary historyof placental mammals. PLoS Biology, 4(4), e91. (Cetartiodactyl branching details added fromO’Leary, M. A., & Gatesy, J. [2007]. Impact of increased character sampling on the phylogeny ofCetartiodactyla (Mammalia): Combined analysis including fossils. Cladistics, 23, 1–46.)morphology (Gingerich et al., 2001; Thewissen Artiodactyla and Cetacea as cognate orders, withet al., 2001). Artiodactyla being retained as a paraphyletic but These revelations bring into focus some ques- adaptively coherent grouping ancestral to thetions about the utility of classifications that are Cetacea. But in a cladistic classification, we arebased solely on genealogical connections. obliged to erect a sequence of nested taxa of con-Phylogenetic relationships are usually correlated tinually diminishing biological import to expresswith the distribution of phenotypic properties, the successive furcations of the cladogram:but not always. Whales, cows, pigs, and camels Superorder hippos-and-whales, Hyperorderare all artiodactyls in a cladistic sense, but there hippos-whales-and-cows, Grandorder hippos-is virtually nothing that they have in common whales-cows-and-camels, and so on. Theapart from some shared DNA sequences. A killer resulting groupings are not very useful for talkingwhale and a sheep are as different in anatomy, about anything except genealogy.ecology, and way of life as it is possible for two The foregoing exposition on systematics isplacental mammals to be. Classifying them offered by way of an apology for what fol-together as ‘‘cetartiodactyls’’ encodes no useful lows—namely, an annotated partial classifica-information about their biology. It is not entirely tion of the order Primates (Table 2.1) that isclear why this is supposed to be a desirable out- neither entirely descriptive nor entirely genealo-come of biological systematics. In a mixed or gical. The classification includes extinct primateevolutionary system of classification, this sort groups, but I have given them short shrift, forof situation would be handled by drawing an two reasons. First, their phylogenetic relation-ordinal boundary across the artiodactyl lineage ships are often unclear and inferable only fromleading to the ancestral whales and treating morphological data, with all the inherent defects

Table 2.1 A Partial Classification of the Order PrimatesSUBORDER STREPSIRRHINIInfraorder †Adapiformes Superfamily †Adapoidea Family †Adapidae (†Adapis, †Adapoides, †Cryptadapis, †Leptadapis, †Palaeolemur, etc.) Family †Notharctidae (†Notharctus, †Cantius, †Cercamonius, †Periconodon, etc.) Family †Sivaladapidae (†Sivaladapis, †Guangxilemur, †Indraloris, †Sinoadapis, etc.)Infraorder Lemuriformes Superfamily Lemuroidea Family Daubentoniidae (Daubentonia) Family Indriidae (Indri, Avahi, Propithecus, etc.) Family Lepilemuridae (Lepilemur) Family Lemuridae (Lemur, Eulemur, Hapalemur, Varecia, †Pachylemur) Family Cheirogaleidae (Cheirogaleus, Microcebus, Mirza, Allocebus) Family †Megaladapidae (†Megaladapis) Family †Paleopropithecidae (†Paleopropithecus, †Mesopropithecus, †Babakotia, etc.) Family †Archaeolemuridae (†Archaeolemur, †Hadropithecus) Superfamily Lorisoidea Family Lorisidae (Loris, Nycticebus, Perodicticus, Arctocebus, †Karanisia, etc.) Family Galagidae (Galago, Galagoides, Euoticus, Otolemur, †Saharagalago, etc.)SUBORDER HAPLORHINIInfraorder Tarsiiiformes Superfamily †Omomyoidea Family †Omomyidae (†Omomys, †Tetonius, †Necrolemur, etc.) Superfamily Tarsioidea Family Tarsiidae (Tarsius)Infraorder Anthropoidea SECTION PLATYRRHINI Superfamily Ceboidea Family Callitrichidae (Callithrix, Cebuella, Callimico, Saguinus, Leontopithecus) Family Cebidae (Cebus, Saimiri) Family Aotidae (Aotus, †Tremacebus) Family Atelidae (Ateles, Lagothrix, Brachyteles, Alouatta, †Protopithecus) Family Pitheciidae (Pithecia, Cacajao, Chiropotes, Callicebus, †Homunculus, etc.) SECTION CATARRHINI Superfamily †Propliopithecoidea (†Propliopithecus, †Aegyptopithecus, †Moeripithecus) Superfamily †Pliopithecoidea (†Pliopithecus, †Epipliopithecus, †Crouzelia, etc.) Superfamily †Proconsuloidea Family †Proconsulidae (†Proconsul, †Afropithecus, †Kenyapithecus, etc.) Family †Sugrivapithecidae (†Sivapithecus, †Ankarapithecus, †Gigantopithecus, etc.) Family †Dendropithecidae ( †Dendropithecus, †Micropithecus, †Simiolus, etc.) Superfamily Hominoidea Family †Dryopithecidae (†Dryopithecus, †Pierolapithecus, †Oreopithecus, etc.) Family Hylobatidae (Hylobates) Family Pongidae (Pongo, †Khoratpithecus?) Family Hominidae (Homo, †Australopithecus, Pan, Gorilla) Superfamily Cercopithecoidea Family †Victoriapithecidae (†Victoriapithecus, †Prohylobates) (continued) 15

16 PRIMATE NEUROETHOLOGY Table 2.1 (Continued) Family Cercopithecidae (Cercopithecus, Chlorocebus, Macaca, Papio, etc.) Family Colobidae (Colobus, Presbytis, Trachypithecus, Nasalis, etc.) Anthropoidea incertae sedis: Families †Proteopithecidae, †Oligopithecidae, †Parapithecidae HAPLORHINI INCERTAE SEDIS: Family †Eosimiidae PRIMATES INCERTAE SEDIS: Families †Plesiopithecidae, †Amphipithecidae. Extinct groups (including paraphyletic stem groups) are indicated by daggers.of such data magnified by the scrappy nature of woodlands, though humans and some Oldthe fossil record. Second, they are of secondary World monkeys range into drier and colderinterest in the context of the present volume. habitats. Distinctive morphological synapomor-Recent reviews of the systematics, diversity, and phies of the crown group (living primates andevolution of extinct primates can be found in extinct descendants of their LCA) include large,Hartwig (2002) and Cartmill and Smith (2009). forward-facing eyes set in complete bony rings; The following classification is generally con- elaboration of the visual apparatus and visualservative and utilizes taxon names that are com- parts of the brain; a tympanic bulla formed bymonly employed and should be widely an extension of the bone (petrosal) surroundingunderstood. It follows phylogenetic relation- the membranous labyrinth; and grasping hindships where it seems convenient and biologically feet with a divergent hallux bearing a flat nail.useful to do so, but employs paraphyletic group- The claws of the other digits are also modifiedings where phylogeny is unclear or where deeply into flattened nails in most primates, which isimbedded groups seem sufficiently distinct probably another crown-group synapomorphy.adaptively to warrant distinction by elevating Grasping specializations of the hand havetheir rank (e.g., Callitrichidae). However, evolved secondarily in several primate lineages.I have followed cladistic principles and current Molecular data show that primates are particu-fashion in sinking gorillas and chimpanzees into larly close relatives of treeshrews and ‘‘flyingthe human family (Hominidae) rather than lemurs’’ or colugos (Fig. 2.2). Most paleontolo-adopting the now quaint-seeming practice of gists regard the Plesiadapiformes, an extinctusing the orangutan family (Pongidae) as a taxo- (Cretaceous-late Eocene) group of primarilynomic wastebasket for the great apes. What arboreal, vaguely rodent-like mammals, asseems to me to be the most probable pattern of stem-group primates (that is, the extinct sisterphyletic relationships among the extant pri- of the crown group). However, plesiadapiformsmates, as judged from molecular evidence, is lack the primate cranial synapomorphies listedpresented in Figures 2.3 and 2.4. previously, and share apomorphies of their own that exclude them from the direct ancestry of the crown-group primates (Bloch et al., 2007). InORDER PRIMATES what follows, the term ‘‘primate’’ will beThe primates are a moderately diverse order of restricted to members of the crown group.mainly arboreal eutherians known as fossils For most practical purposes, primates can befrom the earliest Eocene onward in both the divided into a wastebasket ‘‘prosimian’’ groupNew and Old Worlds. Almost all extant primates (Prosimii) of so-called ‘‘lower’’ primates com-inhabit tropical and subtropical forests and prising early and persistently primitive forms,

PRIMATE CLASSIFICATION AND DIVERSITY 17Figure 2.3 Phylogenetic relationships of major clades of extant strepsirrhine primates (lorises andlemurs), as inferred from molecular data. After Roos, C., Schmitz, J., & Zischler, H. (2004). Primatejumping genes elucidate strepsirrhine phylogeny. Proceedings of the National Academy of Sciences, 101,10650–10654, with compatible addenda from Horvath, J., Weisrock, D., Embry, S., Fiorentino, I., Balhoff, J.,Kappeler, P., et al. (2008). Development and application of a phylogenomic toolkit: Resolving theevolutionary history of Madagascar’s lemurs. Genome Research, 18, 489–499.and a holophyletic clade of ‘‘higher’’ primates characters of the central nervous system and the(Anthropoidea) comprising monkeys, apes, and visual and nasal apparatus. Strepsirrhine symple-humans. One prosimian group, the tarsiers siomorphies include the comma-shaped nostrils(Tarsiidae), is linked to the anthropoids by var- and wet dog-like rhinarium that give the taxon itsious morphological synapomorphies (Fig. 2.1: name. However, the living strepsirrhines areCartmill & Kay, 1978; Kay et al., 1997, 2004) bound together as a clade by a few apparentand by molecular apomorphies including pat- synapomorphies, the most obvious being theterns of SINE insertion (Schmitz & Zischler, modification of the lower incisors and canines2004). A subordinal cut between Haplorhini into a ‘‘toothcomb’’ (Fig. 2.6) used for grooming(tarsiers-plus-anthropoids) and Strepsirrhini the fur (and for specialized feeding activities in(nontarsier prosimians), rather than a cut some species). Strepsirrhines also differ frombetween Prosimii and Anthropoidea, is adopted other primates in having an epitheliochorialhere on cladistic grounds. rather than a hemochorial placenta. This is counted as a synapomorphy by some and a sym- plesiomorphy by others (Wildman et al., 2006).SUBORDER STREPSIRRHINI † Infraorder AdapiformesExtant strepsirrhine primates can be described aslooking rather like monkeys with the heads of This extinct group of lemur-like primates includesdogs (Fig. 2.5). Most of the traits in which they most of the larger and more herbivorous primatesdiffer systematically from other primates are pri- known as fossils from the Eocene. Adapiforms lackmitive (plesiomorphic) states of various the toothcomb, and it is not certain that they are

18 PRIMATE NEUROETHOLOGYFigure 2.4 Phylogenetic relationships of major clades of extant haplorhine primates (tarsiers andanthropoids), as inferred from molecular data. After Xing, J., Wang, H., Han, K., Ray, D., Huang, C.,Chemnick, L., et al. (2005). A mobile element based phylogeny of Old World monkeys. MolecularPhylogenics and Evolution, 37, 872–880, with compatible addenda from Goodman, M., Porter, C.,Czelusniak, J., Pages, S., Schneider, H., Shoshani, J., et al. (1998). Toward a phylogenetic classification ofprimates based on DNA evidence complemented by fossil evidence. Molecular Phylogenics and Evolution, 9,585–598, and Tosi, A., Disotell, T., Morales, J., & Melnick, D. (2003). Cercopithecine Y-chromosome dataprovide a test of competing morphological evolutionary hypotheses. Molecular Phylogenics and Evolution,27, 510–521.more closely related to lemurs than they are to some heavily built, slow-moving forms that maymonkeys (Gebo, 2002; Rasmussen, 1986). If they have resembled sloths in their ecology.are not, then Strepsirrhini as defined here is a Notharctidae were more active, lemur-like, run-wastebasket taxon. ning and leaping animals. The Sivaladapidae Adapiforms constitute a single superfamily, were an Asian radiation of adapiforms that cul-Adapoidea, divisible into three families. The minated in some largish (ca. 7 kg), deep-jawed,Adapidae comprise several genera of medium- monkey-convergent prosimians known from thesized, primarily folivorous primates, including Miocene of Southeast Asia.

PRIMATE CLASSIFICATION AND DIVERSITY 19Figure 2.5 The face of a strepsirrhine primate (Varecia juvenile). From Cartmill, M., & Smith, F. H.(2009). The human lineage. New York: Wiley-Blackwell. Used with permission.Figure 2.6 Dentition of the cheirogaleid lemuroid Cheirogaleus medius, showing the toothcomb (T).From Cartmill, M., & Smith, F. H. (2009). The human lineage. New York: Wiley-Blackwell, afterJames, W. (1960). The jaws and teeth of primates. London: Pitman Medical Publishing. Used withpermission.Infraorder Lemuriformes the cheek teeth and the middle ear. LorisoidsThis grouping comprises the toothcomb pri- are known from late Eocene deposits in themates, divided into the lemurs of Madagascar Fayum Depression of Egypt (Seiffert et al.,(Lemuroidea) and the lorises and galagos or 2003) and from the Miocene of Africa andbushbabies (Lorisoidea) of Africa and Asia. southern Asia. There is essentially no fossilThe two groups differ systematically in a record of lemuroid evolution; the oldestnumber of anatomical details, particularly of extinct lemuroids are Holocene species that

20 PRIMATE NEUROETHOLOGYoverlapped in time with the early human estimated divergence from the other Malagasycolonists of Madagascar. primates lead some systematists to assign it to a separate infraorder of its own, Chiromyiformes.Superfamily Lemuroidea Family Indriidae The indriid genera Indri,The Malagasy primates constitute a holophyletic Propithecus, and Avahi comprise several speciesclade of considerable antiquity. Recent mole- of lemurs specialized for a diet of leaves and acular-clock estimates date the first divergence ‘‘vertical clinging and leaping’’ pattern of posi-within the lemuroids (between Daubentonia tional behavior. They have long, powerful hin-and the others) to around the time of the K-T dlimbs capable of propelling them from tree toboundary (Horvath et al., 2008). The available tree in 10-meter leaps, and their long hands anddata point to a late Cretaceous or early Tertiary feet have robust, widely divergent first digitscolonization of Madagascar by a single ancestral adapted to grasping large vertical supports. Thelemuroid species, presumably derived from the indriids include the largest extant strepsirrhine,same undocumented proto-lemuriform radia- Propithecus diadema (>7 kg), as well as one oftion in Africa that gave rise to the ancestral the smallest folivorous primates, Avahi lanigerlorisoids (Yoder et al., 1996). Like other small, (%1 kg). Avahi is nocturnal in its activity, butisolated land masses, Madagascar has a taxono- Propithecus and Indri are mainly diurnal ani-mically impoverished fauna whose endemic mals, and some species have strikingly patternedgroups have radiated to fill ecological spaces and colored pelage, presumably correlated withpreempted by other taxa on the major conti- color vision.nents. Most of the extant lemuroids occupyniches broadly similar to those filled by primates Family Lepilemuridae Seven or more speciesin other tropical forests, but Daubentonia and of Lepilemur are distinguished by taxonomists,some of the large extinct lemurs developed unu- mainly on the basis of pelage variants. All aresual specializations. small (500 to 1,000 g), long-tailed, nocturnal, arboreal folivores with a moderate degree ofFamily Daubentoniidae The aye-aye Dau- specialization for leaping locomotion. Despitebentonia madagascariensis is the sole primate its adaptive convergence with Avahi, Lepilemurrepresentative of a foraging guild that has been shows no special morphological similarity to thelabeled ‘‘mammalian woodpeckers’’ (Beck, indriids, and recent molecular data (Horvath2009) or ‘‘woodpecker avatars’’ (Cartmill, et al., 2008) position it as the phyletic sister of1974). The extant members of this guild—the the mouse- and dwarf-lemur familyaye-aye and the Australasian marsupials of the Cheirogaleidae (Fig. 2.3).genus Dactylopsila—inhabit areas devoid ofbirds that feed on wood-boring insect larvae. Family Lemuridae Of the four genera ofThey have developed similar convergent adapta- extant lemurids, three (Lemur, Eulemur, andtions for exploiting this resource, including Varecia) are rather similar-looking medium-enlarged incisors for cutting into infested wood sized animals with long foxy muzzles, generallyand an elongated, clawed finger used for probing fruit-centered dietary preferences, and dailyexposed tunnels and snagging the grubs. The activity cycles ranging from semi-nocturnalaye-aye’s complex foraging habits (Erickson, (Eulemur mongoz) to strictly diurnal (Lemur).1994) are probably related to the large size of Lemur catta, the sole species in its genus, has aits brain, which is as big relative to body size as more terrestrial activity pattern and larger groupthose of some monkeys (Stephan, 1972; sizes (up to 20þ animals) than other livingStephan & Andy, 1969). The aberrant morpho- lemuroids. Its phyletic sister Hapalemur islogical specializations of the aye-aye (including another small folivore (700 to 2,400 g), with ainguinal nipples, claws on all digits except the suite of anatomical, behavioral, and biochemicalhallux, and permanently growing incisor teeth specializations for feeding on bamboo. Althoughlike those of rodents) and the antiquity of its the Lemuridae as a whole lack obvious

PRIMATE CLASSIFICATION AND DIVERSITY 21morphological synapomorphies, SINE inser- sometimes likened to a giant koala. Thetions (Roos et al., 2004) identify them as a holo- Archaeolemuridae (Archaeolemur, Hadropithecus)phyletic clade and the sister group of the indriids include the most terrestrial offshoots of the(Fig. 2.3). lemuroid radiation, convergent in various respects with certain ground-feeding Old World monkeys.Family Cheirogaleidae This family com- Superfamily Lorisoideaprises five genera of small nocturnal primatesranging in size from %300 g (Mirza coquereli) Although far less diverse than the primates ofto 30 g (Microcebus myoxinus, the smallest living Madagascar, the continental lemuriforms areprimate). All feed on high-energy foods—fruit, nevertheless divided into two sharply differen-nectar, gums, and animal prey—and some spe- tiated subgroups. Both molecular and fossil datacies of Microcebus and Cheirogaleus conserve point to a late Eocene divergence of the two. Theenergy during lean seasons of the year by esti- family Lorisidae (lorises) comprises four generavating or by metabolizing fat stored in the tail. of African (Perodicticus, Arctocebus) and AsianMany experts have pointed to Microcebus as a (Loris, Nycticebus) prosimians that range inplausible living model for the last common size from 1,200 g (N. coucang) to around 200 gancestor of the crown-group primates. Cheiro- (A. aureus). All four lorisid genera share noc-galeids share certain potentially apomorphous turnal habits, diets featuring animal prey, and aanatomical traits with galagos, including elonga- suite of striking apomorphies of locomotortion of the tarsal bones and peculiar specializa- anatomy and behavior—vestigial tails, vise-liketions of the carotid arterial circulation. Some hands and feet, and cautious, often weirdly slowsystematists have accordingly suggested that patterns of locomotion in which at least one handthey may be more closely related to the lorisoids or foot remains in contact with the support at allthan to the other Malagasy lemurs (Cartmill, times, even during running (Schmitt et al., 2006).1975; Schwartz, 1986; Szalay & Katz, 1973). The members of the other lorisoid family, theHowever, molecular data consign the Galagidae (Galago, Otolemur, Galagoides,cheirogaleids to a deeply imbedded position Euoticus), are also nocturnal animals with mixedwithin the lemuroid clade (Fig. 2.3: Goodman diets, but their locomotor specializations are justet al., 1998; Horvath et al., 2008; Roos et al., the opposite: galagos are adapted to leaping and2004; Yoder et al., 1996). Any features they have correspondingly long tails, long andshare with galagos must therefore be conver- powerful hindlimbs, and elongated tarsal bonesgences, or symplesiomorphies retained from that add leverage and speed at takeoff. Ranging inthe lemuriform LCA. size from Otolemur (%1,100 g) to the diminutive Three families of large-bodied lemuroids Galagoides (%60 g), galagos are limited in distri-became extinct around the time of the initial bution to Africa.human colonization of Madagascar some 2,000years ago. Paleopropithecidae (Paleopropithecus, SUBORDER HAPLORHINIMesopropithecus, Babakotia) were suspensoryarboreal leaf eaters convergent in their ecology Although tarsiers and anthropoids are united byand locomotor anatomy with the tree sloths of the a substantial list of shared derived features ofNew World. The largest of them, Archaeoindris, is morphology, many systematists continue toknown mainly from a single skull, which was as big resist their assimilation into a common suborderas that of a male gorilla. This gigantic ‘‘sloth lemur’’ Haplorhini (Shoshani et al., 1996.) There aremay have been a terrestrial form resembling a three main reasons for this resistance: (1) someground sloth. Another huge extinct lemur, of the candidate haplorhine synapomorphiesMegaladapis (family Megaladapidae), appears to (e.g., the hemochorial placenta) may in fact behave been a slow-moving (but not suspensory), symplesiomorphies, (2) most of the candidategreat-ape-sized (40 to 80 kg) folivore. It is synapomorphies are either unknown or lacking

22 PRIMATE NEUROETHOLOGYin the fossil prosimians often regarded as pos- faunivorous primates, and their antemolar den-sible tarsier relatives, and (3) some molecular tition is uniquely specialized for seizing anddata suggest that tarsiers have closer ties to holding prey, with stabbing, dagger-like incisorslemurs and lorises than to anthropoids (Eizirik supplementing the canines and a battery ofet al., 2004). However, the bulk of the molecular pointed premolars (Fig. 2.7). The tarsier post-data, including SINE insertions, confirm the cranium is highly specialized for vertical clingingmonophyly of the extant Haplorhini and leaping, with long hindlimbs; elongated(Schmitz & Zischler, 2004). tarsal bones (which give tarsiers their name); highly stabilized, hinge-like hip and ankle joints; and long stiff tails that serve as props inInfraorder Tarsiiformes clinging to vertical supports. Tarsiers’ bulgingThe only animals universally admitted to this eyes, elongated legs and feet, and long fingersinfraorder are the five to eight species in the and toes tipped with expanded pads give them aextant genus Tarsius (tarsiers), which inhabit vaguely froggy appearance despite their longseveral islands of the Malay Archipelago tails and big, galago-like ears.including Sumatra, Borneo, Sulawesi, and the Living tarsiers are here placed in aPhilippines. The affinities of tarsiers are Superfamily Tarsioidea of their own, distin-obscured by their grotesque specializations, guished from a Superfamily †Omomyoideawhich collectively justify their German name of containing an array of extinct Paleogene prosi-‘‘goblin lemurs’’ (Koboldmaki). Visual predators mians often regarded as tarsiiforms. Omo-par excellence, these small animals (%100 g) myoids comprise most of the nonadapoidhave enormous eyeballs—each as large as the primates known from the Eocene. Most ofbrain, in some species. Their retinas lack cones them are small animals known exclusively frombut have foveas (Wolin & Massopust, 1970), teeth and jaws, which evince a spectrum ofsuggesting descent from a diurnal ancestor with dietary adaptations ranging from insectivory tosmaller eyes. Tarsiers are the only exclusively frugivory. A few of the larger omomyoids mayFigure 2.7 Antemolar dentition of Tarsius. I, upper central incisor (I1); C, lower canine. From Cartmill,M., & Smith, F. H. (2009). The human lineage. New York: Wiley-Blackwell. Used with permission.

PRIMATE CLASSIFICATION AND DIVERSITY 23have been folivores. Known skulls and postcra- or imply that these stem-group anthropoidsnial remains variously resemble those of cheir- (Families †Oligopithecidae, †Proteopithecidae,ogaleids and galagos, with varying degrees of and †Parapithecidae) had distinctively anthro-limb specializations for leaping and generally poid teeth, ear regions, and orbits, but lackedlarge eye sockets that suggest nocturnal habits. some of the synapomorphies of the crown groupSeveral omomyoid genera exhibit apomorphies (e.g., big brains and fused mandibular sym-peculiar to tarsiers among living primates, but physes). Other candidate Eocene anthropoidsthese are scattered in distribution and are not include two groups from eastern Asia, thehierarchically nested in a way that would imply †Amphipithecidae and †Eosimiidae, which areordinated degrees of phyletic relationship to dismissed by some authorities as anthropoid-Tarsius. convergent adapoids and omomyoids, respec- The first omomyoids known, from the early tively (Beard, 2002; Beard et al., 2005;Eocene, are very similar in molar morphology to Ciochon & Gunnell, 2004; Ciochon et al., 2001;the earliest adapoids (Ni et al., 2004; Rose & Gunnell & Miller, 2001; Rasmussen, 2002).Bown, 1991), intimating that these fossils maybe sampling the initial stages in the divergence of Section Platyrrhini ¼ Superfamily Ceboideahaplorhines from strepsirrhines. However, thisinference conflicts with molecular-clock esti- Living anthropoids are divided into two groups:mates that date the haplo-strepsirrhine split to the monkeys of the New World tropicssome 80 Mya, well back in the Mesozoic (Platyrrhini) and the monkeys, apes, and(Horvath et al., 2008; Martin, 1993). Some tiny humans of the Old World (Catarrhini). Thebut strikingly tarsier-like molars and a fragmen- platyrrhines are almost wholly plesiomorphoustary maxilla from the middle Eocene of China relative to the crown-group catarrhines, lackinghave been attributed to the genus Tarsius (Beard such distinctive catarrhine synapomorphies as aet al., 1994; Rossie et al., 2006). If Tarsius itself tubular extension of the bony ring (ectotym-was already present in the Eocene, then most of panic) around the eardrum and the loss of thethe isolated tarsier-like features found among anterior premolar. Nevertheless, there are a fewomomyoids must be convergences or paralle- candidate synapomorphies of the New Worldlisms. At present, the haplorhine affinities of monkeys, mainly in details of the dentition,these extinct ‘‘tarsiiforms’’ remain uncertain and they consistently sort out as a clade in ana-(Beard, 1988). lyses based on molecular data. The origins of the New World monkeys are mysterious. They first appear in OligoceneInfraorder Anthropoidea deposits of South America, at a time whenLiving anthropoids are distinguished from tar- South America was separated from other con-siers and other prosimians by a host of synapo- tinental land masses by oceanic gaps of hundredsmorphies, including a complete postorbital of kilometers. Some systematists who think thatseptum, fusion of the two halves of the lower anthropoids originated in Asia postulate anjaw, vertically implanted spatulate incisors, a entrance into South America via Beringea andfoveate retina with a cone-rich area centralis, North America, but the general consensus is thatenlargement of the brain and elaboration of the they probably descend from some African basalvisual centers, and numerous details of the cheek anthropoid that somehow managed to get acrossteeth, ear region, and postcranium. Molecular the South Atlantic. Floating rafts of coastal vege-phylogenetic studies uniformly confirm the tation torn loose by a tropical storm are a pop-holophyly of the anthropoid clade. The earliest ular fantasy vehicle for this sea crossingfossils universally acknowledged as anthropoids (Chiarelli & Ciochon, 1980). Origin by vicar-come from Eocene deposits of the Egyptian iance (continental rifting yielding a cladisticFayum (Seiffert et al., 2005; Simons & split in the attached fauna) is unlikely,Rasmussen, 1994). The known specimens show because estimated molecular dates for the

24 PRIMATE NEUROETHOLOGYplatyrrhine-catarrhine divergence (32–36 Mya) Of special interest to neurobiologists, thepostdate the Cretaceous zoogeographic isolation night or owl monkey Aotus is the only nocturnalof South America from the other continents by anthropoid. Its eyes are enlarged for purposes ofat least 30 million years (Bocxlaer et al., 2006; light-gathering under scotopic conditions, andGlazko & Nei, 2003). its retinas are afoveate and devoid of cones. This Morphological and molecular data concur in small monkey (%1 kg) is sometimes classed insegregating the platyrrhines into five clusters (Fig. the family Cebidae, along with Cebus (capuchin2.4). The marmosets and tamarins (Family monkeys) and Saimiri (squirrel monkeys), but isCallitrichidae) are the smallest living anthropoids, here assigned to its own family (Aotidae)ranging in size from 500 g (Callimico) down to because of its markedly divergent specializa-%110 g (Cebuella). Their small size correlates with tions. Aotus has an average-sized brain for aa dietary preference for insects, plant gums, and New World monkey, whereas Cebus andother high-energy foods. All their digits except the Saimiri are the most highly encephalized of thefirst toe are furnished with sharp claws, which living platyrrhines. This may be a synapo-facilitate clinging to thick tree trunks in feeding morphy linking the two. However, brainon exudates. The callitrichids’ small size is also growth follows different ontogenetic trajectoriescorrelated with uniquely high reproductive rates in the two genera, occurring mainly before birthfor anthropoids—up to four offspring annually in Saimiri and mainly after birth in Cebusper female. This high birth rate is made possible by (Hartwig, 1999). The large brain of Cebus corre-consistent twinning and by a cooperative breeding lates with some ‘‘advanced’’ behavioral apomor-system in which males and juveniles carry and phies, including tool use in both captivity andtend infants, handing them over at intervals to the wild (Fragaszy et al., 2004).the group’s reproductive female to nurse. The remaining platyrrhine genera (Pithecia,Callimico, which lacks the twinning pattern and Chiropotes, Cacajao, Callicebus) constitute a fifththe reduced dental formula seen in the rest of the family of New World anthropoids, the Pithe-callitrichids, looks like it ought to represent a cidae, distinguished by several synapomorphiesprimitive outgroup of the others—but the mole- including specializations of the anterior teeth forcular data belie this interpretation (Fig. 2.4). feeding on hard, unripe fruit. Some systematists At the opposite end of the platyrrhine size regard Aotus as part of the pithecid clade, butspectrum, the Atelidae are a family of this assignment is not supported by the mole-medium-sized (%5 to 10 kg) arboreal plant cular data (Fig. 2.4).eaters, including the New World’s only pri-marily folivorous monkey, Alouatta. The pla- Section Catarrhinityrrhine stock has not given rise to a radiationof arboreal leaf eaters like those that evolved The earliest fossil anthropoids from the Oldamong the lemurs of Madagascar and the Old World are no more clearly related to modernWorld anthropoids, perhaps because that catarrhines than they are to platyrrhines. Theniche was preempted in South America by oldest taxon widely accepted as a catarrhinethe tree sloths. All the atelids have prehensile stem group is the early Oligocene familytails, and all but Alouatta spend a significant †Propliopithecidae from the Fayum.amount of time hanging and swinging under- Propliopithecids have the reduced dental for-neath branches, suspended by their tails and mula characteristic of later Old World anthro-hands. As adaptations to this sort of arm- poids (2.1.2.3/2.1.2.3) and share some otherswinging locomotion, they have evolved some catarrhine dental synapomorphies (e.g., loss ofconvergently ape-like apomorphies of the the paraconid cusp on the lower molars).limbs and trunk (Erikson, 1963). Some extinct Otherwise, they appear to have been persistentlyPleistocene atelids (Caipora, Protopithecus) primitive anthropoids, with relatively smallapproached the great apes in body size, with brains, ring-shaped ectoympanics, and general-weights of up to 25 kg. ized arboreal-quadruped postcranial anatomy.

PRIMATE CLASSIFICATION AND DIVERSITY 25 Modern catarrhines fall into two holophyletic the head with the elbow fully extended. Thesuperfamilies: apes-plus-humans (Hominoidea) thorax is transversely broadened and the cla-and Old World monkeys (Cercopithecoidea), vicle elongated (Fig. 2.8), redirecting the gle-distinguished from each other by conspicuous noid socket of the shoulder joint to face moresynapomorphies peculiar to each group. The laterally and thereby facilitating swinging andoldest known cercopithecoids occur as fossils in hanging from supports above and behind thethe early Miocene, whereas the first fossil apes head. Because flexion and extension of the backknown to exhibit the distinctive shared peculia- no longer contribute to locomotion as they dorities of the living hominoids are Dryopithecus in a galloping quadruped, the lumbar part ofand related genera (family †Dryopithecidae) the hominoid vertebral column is reduced infrom the late Miocene of Europe. The remaining length and its epaxial extensor muscles areMiocene catarrhines are sometimes described as reduced in volume and cross-sectional area,‘‘dental apes’’ because they had ape-like denti- resulting in characteristic changes in the mor-tions but monkey-like limb and trunk skeletons, phology of the lumbar vertebrae.adapted to quadrupedal locomotion rather than Extant hominoids are here divided intoarm swinging. They are assigned here to two three families. The lesser apes or gibbonsextinct superfamilies: the paraphyletic (Hylobatidae) are medium-sized (5 to 12 kg),†Proconsuloidea (which have a complete tub- strictly arboreal inhabitants of tropical forests inular ectotympanic) and the probably holophy- Southeast Asia and Indonesia. Hylobatids areletic †Pliopithecoidea (which do not, and swift, ricochetal arm-swingers that often hurlpresumably branched off at an earlier point themselves in acrobatic, arm-propelled leapsfrom a stem catarrhine resembling the proplio- across gaps of several meters between trees. Theypithecids). Both cercopithecoids and hominoids seldom descend to the ground, and walk bipedallyprobably originated from proconsuloid with their hands in the air on the rare occasionsancestors. when they do so. From the Miocene onward, catarrhines have The remaining apes are considerably largerexhibited two evolutionary tendencies not evi- animals and correspondingly more cautious indent in other primates: (1) a tendency to evolve their arboreal locomotion. Orangutansmating systems involving high levels of male– (Pongidae) are the largest strictly arboreal pri-male competition and marked sexual mates, with body weights ranging from %36 kgdimorphism and (2) a tendency to occupy ter- in females up to around 80 kg in big males.restrial niches. The two tendencies are probably Their evolutionary history is disputed. Theadaptively correlated with each other. Substantial Miocene sugrivapithecid Sivapithecus had aradiations of terrestrially adapted forms have strikingly orangutan-like facial skeleton, andappeared in all three of the post-Oligocene catar- most authorities regard it as an early represen-rhine superfamilies—in the large-bodied, thick- tative of the Pongidae. However, known post-enameled proconsuloids here classified as the cranials of Sivapithecus seem to have beenfamily †Sugrivapithecidae, in the cercopithecids generally monkey-like, lacking key hominoidamong the Cercopithecoidea, and in the homi- apomorphies of the hand and shouldernids among the Hominoidea. (Pilbeam, 2002). Either the orangutan-like cranial features of Sivapithecus are convergences orSuperfamily Hominoidea else most of the hominoid postcranial apomor-The living apes and humans are distinguished phies evolved separately three times—in theby a suite of postcranial synapomorphies ancestors of gibbons, of orangutans, and ofthought to have originated as adaptations to the African apes and humans. The jury is stillan arm-swinging form of suspensory locomo- out on this issue.tion. Hominoids have long, limber forelimbs The final hominoid family, the Hominidae,with modifications of the joints that maximize comprises humans (Homo) and theirflexibility and permit the arm to be held over Plio-Pleistocene relatives (†Australopithecus),

26 PRIMATE NEUROETHOLOGYFigure 2.8 Cranial views of the thorax and right shoulder girdle of a cercopithecoid (Papio, A) andhominoid (Homo, B), showing the elongated clavicle, transversely broad thorax, and reoriented shouldersocket (arrows) characteristic of the crown-group Hominoidea. From Cartmill, M., & Smith, F. H. (2009).The human lineage. New York: Wiley-Blackwell, after Schultz, A. (1969). The life of primates. London:Weidenfeld and Nicholson. Used with permission.chimpanzees (Pan), and gorillas (Gorilla). spread into Asia around the end of the Miocene.Chimpanzees and gorillas exhibit a so-called The first cercopithecoids (†Victoriapithecidae)‘‘knuckle-walking’’ pattern of quadrupedal loco- are already distinguished from hominoids by sev-motion, in which the weight of the upper body is eral characteristic cercopithecoid synapomor-borne on the backs of the middle phalanges, and phies, including a ‘‘bilophodont’’ molar patternthey share some functionally related weight- (in which the four main cusps of the molar teethbearing specializations of the hand and wrist are joined together by transverse crests) and var-not seen in humans. Despite these apparent ious postcranial traits that appear to be function-synapomorphies, molecular data strongly indi- ally related to an increase in the amount of timecate that chimpanzees are more closely related to spent on the ground (Benefit & McCrossin, 2002;humans than to gorillas, and all three genera are McCrossin et al., 1998).now conventionally lumped into the human The extant cercopithecoids are divisible intofamily on cladistic grounds. two differently adapted families. The Colobidae or leaf monkeys are primarily folivores, withSuperfamily Cercopithecoidea correspondingly specialized teeth and saccu-Molecular-clock studies suggest a date of 23–25 lated, somewhat ruminant-like stomachs. MostMya for the divergence of Old World monkeys extant colobids are largely or exclusivelyfrom the ‘‘dental ape’’ ancestors of the arboreal. The Cercopithecidae are moreHominoidea. Cercopithecoids first appear in the diversely adapted, omnivorous animals thatlate early Miocene of North and East Africa and manifest a wider range of life habits and body

PRIMATE CLASSIFICATION AND DIVERSITY 27sizes, from the small arboreal form Miopithecus Benefit, B., & McCrossin, M. (2002). The(%1,200 g) up to the large-bodied, mainly ter- Victoriapithecidae, Cercopithecoidea. In:restrial quadrupeds Papio and Mandrillus (over W. Hartwig (Ed.), The primate fossil record30 kg in big males). Colobids find their greatest (pp. 241–253). Cambridge: University ofdiversity in Asia, with one genus (Colobus) Cambridge Press. Bloch, J. I., Silcox, M. T., Boyer, D. M., & Sargis, E. J.native to Africa; conversely, cercopithecids con- (2007). New Paleocene skeletons and thestitute a largely African radiation that has given relationship of plesiadapiforms to crown-grouprise to a single cosmopolitan genus (Macaca) primates. Proceedings of the National Academy ofextending beyond Africa eastward across Sciences, 104, 1159–1164.southern Asia into Indonesia. Perhaps the most Bocxlaer, I., Roelants, K., Biju, S., Nagaraju, J., &successful and speciose of living primate Bossuyt, F. (2006). Late Cretaceous vicariancefamilies, the cercopithecids comprise a bewil- in Gondwanan amphibians. PLoS ONE, 1, e74.dering variety of variously adapted arboreal Cartmill, M. (1974). Daubentonia, Dactylopsila,and terrestrial forms, including some 20 recog- woodpeckers, and klinorhynchy. In: A. Martin,nized species in each of the genera Macaca and G. Doyle, & A. Walker (Eds.), Prosimian biologyCercopithecus. (pp. 655–670). London: Duckworth. Cartmill, M. (1975). Strepsirhine basicranial structures and the affinities of the Cheirogaleidae. In: W. Luckett & F. S. SzalayREFERENCES (Eds.), Phylogeny of the primates (pp. 313–354).Arnason, U., Gullberg, G., Gretarsdottir, S., New York: Plenum. Ursing, B., & Janke, A. (2000). The Cartmill, M. (1980) Morphology, function and mitochondrial genome of the sperm whale and evolution of the anthropoid postorbital a new molecular reference for estimating septum. In: A. B. Chiarelli & R. L. Ciochon eutherian divergence dates. Journal of (Eds.), Evolutionary biology of the New World Molecular Evolution, 50, 569–578. monkeys and continental drift (pp. 243–274).Beard, K. C. (1988). The phylogenetic significance New York: Plenum. of strepsirrhinism in Paleogene primates. Cartmill, M. (1982). Assessing tarsier affinities: International Journal of Primatology, 9, 83–96. Is anatomical description phylogeneticallyBeard, K. C. (2002). Basal anthropoids. In: ´ neutral? Geobios, memoire special, 6, W. Hartwig (Ed.), The primate fossil record 279–287. (pp. 133–149). Cambridge: University of Cartmill, M. (1994a), Anatomy, antinomies, and Cambridge Press. the problem of anthropoid origins. In:Beard, K. C., Jaeger, J-J, Chaimanee, Y., Rossie, J. F. Fleagle & R. F. Kay (Eds.), Anthropoid J., Soe, A., Tun, S., et al. (2005). Taxonomic origins (pp. 549–566). New York: Plenum. status of purported primate frontal bones Cartmill, M. (1994b). A critique of homology as a from the Eocene Pondaung Formation of morphological concept. American Journal of Myanmar. Journal of Human Evolution, 49, Physical Anthropology, 94, 115–123. 468–481. Cartmill, M. (1999). Revolution, evolution, andBeard, K. C., & MacPhee, R. D. E. (1994). Cranial Kuhn: A response to Chamberlain and anatomy of Shoshonius and the antiquity of the Hartwig. Evolution and Anthropology, 8, Anthropoidea. In: J. F. Fleagle & R. F. Kay 45–47. (Eds.), Anthropoid origins (pp. 55–97). New Cartmill, M., & Kay, R. F. (1978). Craniodental York: Plenum. morphology, tarsier affinities, and primateBeard, K. C., Qi, T., Dawson, M., Wang, B., & Li, C. suborders. In: D. J. Chivers & K. A. Joysey (1994). A diverse new primate fauna from (Eds.), Recent advances in primatology (vol 3, middle Eocene fissure-fillings in southeastern pp. 205–214). London: Academic Press. China. Nature, 368, 604–609. Cartmill, M., & Smith, F. H. (2009). The humanBeck, R. M. D. (2009). Was the Oligo-Miocene lineage. New York: Wiley-Blackwell. Australian metatherian Yalkaparidon a Chiarelli, A., & Ciochon, R. (Eds). (1980). ‘mammalian woodpecker’? Biological Journal of Evolutionary biology of the New World monkeys the Linnaean Society of London, 97, 1–17. and continental drift. New York: Plenum.

28 PRIMATE NEUROETHOLOGYCiochon, R., Gingerich, P., Gunnell, G., & Simons, complemented by fossil evidence. Molecular E. L. (2001). Primate postcrania from the late Phylogenics and Evolution, 9, 585–598. middle Eocene of Myanmar. Proceedings of the Gregory, W. K. (1910). The orders of mammals. National Academy of Sciences, 99, 7672–7677. Bulletin of the American Museum of NaturalCiochon, R., & Gunnell, G. (2004). Eocene large- History, 27, 1–524. bodied primates from Myanmar and Thailand: Gunnell, G., & Miller, W. (2001). Origin of morphological considerations and phylogenetic Anthropoidea: Dental evidence and affinities. In: C. Ross & R. F. Kay (Eds.), recognition of early anthropoids in the fossil Anthropoid origins: New visions (pp. 249–282). record, with comments on the Asian New York: Kluwer/Academic. anthropoid radiation. American Journal ofCook, J. M., & Tristem, M. (1997). ‘SINEs of the Physical Anthropology, 114, 177–191. times’ — transposable elements as clade Hartwig, W. (1999). Perinatal life history traits in markers for their hosts. Trends in Ecology and New World monkeys. American Journal of Evolution, 12, 295–297. Primatology, 40, 99–130.Eizirik, E., Murphy, W. J., & O’Brien, S. J. (2001). Hartwig, W. (Ed.). (2002). The primate fossil record. Molecular dating and biogeography of the early Cambridge: University of Cambridge Press. placental mammal radiation. Journal of Hennig, W. (1966). Phylogenetic systematics. Heredity, 92, 212–219. Urbana: University of Illinois Press.Eizirik, E., Murphy, W. J., Springer, M. S., & Horvath, J., Weisrock, D., Embry, S., Fiorentino, I., O’Brien, S.J. (2004) Molecular phylogeny and Balhoff, J., Kappeler, P., et al. (2008). dating of early primate divergences. In: C. Ross Development and application of a & R. F. Kay (Eds.), Anthropoid origins: New phylogenomic toolkit: Resolving the visions (pp. 45–64). New York: Kluwer/ evolutionary history of Madagascar’s lemurs. Academic. Genome Research, 18, 489–499.Erickson, C. J. (1994). Tap-scanning and extractive James, W. (1960). The jaws and teeth of primates. foraging in aye-ayes, Daubentonia London: Pitman Medical Publishing. madagascariensis. Folia Primatologica, 62, Ji, Q., Luo, Z.-X., Yuan, C.-X., Wible, J. R., Zhang, 125–135. J.-P., & Georgi, J. A. (2002). The earliestErikson, G. (1963). Brachiation in the New World eutherian mammal. Nature 416, 816–822. monkeys. Symposia of the Zoological Society of Kay, R., Ross, C., & Williams, B. (1997). London, 10, 135–164. Anthropoid origins. Science, 275, 797–804.Fragaszy, D., Izar, P., Visalberghi, E., Ottoni, E., & Kay, R., Willams, B., Ross, C., Takai, M., & Gomes de Oliviera, M. (2004). Wild capuchin Shigehara, N. (2004). Anthropoid origins: A monkeys (Cebus libidinosus) use anvils and phylogenetic analysis. In: C. Ross & R. F. Kay stone pounding tools. American Journal of (Eds.), Anthropoid origins: New visions (pp. 91– Primatology, 64, 359–366. 135). New York: Kluwer/Academic.Gebo, D. (2002). Adapiformes: Phylogeny and Kriegs, J., Churakov, G., Kiefmann, M., Jordan, U., adaptation. In: W. Hartwig (Ed.), The primate Brosius, J., & Schmitz, J. (2006). Retroposed fossil record (pp. 21–43). Cambridge: University elements as archives for the evolutionary of Cambridge Press. history of placental mammals. PLoS Biology,Gingerich, P., ul Haq, M., Zalmout, I., Khan, I., & 4(4), e91. Malkani, M. (2001). Origin of whales from early Kumar, S., & Hedges, S. B. (1998). A molecular artiodactyls: hands and feet of Eocene timescale for vertebrate evolution. Nature, 392, Protocetidae from Pakistan. Science, 293, 917–920. 2239–2242. Linne, K. (1735). Systema naturae sive regna triaGlazko, G. V., & Nei, M. (2003). Estimation of naturae systematice proposita per classes, ordines, divergence times for major lineages of primate genera, et species. Leiden: T. Haak. 1964 species. Molecular Biology and Evolution, 20, facsimile reprint. Nieuwkoop: de Graaf. 424–434. Martin, R. (1993). Primate origins: Plugging theGoodman, M., Porter, C., Czelusniak, J., Pages, S., gaps. Nature, 363, 223–234. Schneider, H., Shoshani, J., et al. (1998). McCrossin, M., Benefit, B., Gitau, S., Palmer, A., & Toward a phylogenetic classification of Blue, K. (1998). Fossil evidence for the origins primates based on DNA evidence of terrestriality among Old World higher

PRIMATE CLASSIFICATION AND DIVERSITY 29 primates. In: E. Strasser, J. Fleagle, Seiffert, E., Simons, E., Clyde, W., Rossie, J., Attia, A. Rosenberger, & H. McHenry (Eds.), Y., Bown, T., et al. (2005). Basal anthropoids Primate locomotion: Recent advances (pp. from Egypt and the antiquity of Africa’s higher 353–396). New York: Plenum. primate radiation. Science, 310, 300–304.Murphy, W., Eizirik, E., O’Brien, S., Madsen, O., Shedlock, A. M., Takahashi, K., & Okada, N. Scally, M., Douady, C., et al. (2001). Resolution (2004). SINEs of speciation: Tracking lineages of the early placental mammal radiation using with retroposons. Trends in Ecology and Bayesian phylogenetics. Science, 294, Evolution, 19, 545–553. 2348–2351. Shoshani, J., Groves, C., Simons, E., & Gunnell, G.Ni, X., Wang, Y., Hu, Y., & Li, C. (2004). (1996). Primate phylogeny: Morphological vs. A euprimate skull from the early Eocene of molecular results. Molecular Phylogenics and China. Nature, 427, 65–68. Evolution, 5, 102–154.O’Leary, M. A., & Gatesy, J. (2007). Impact of Simons, E., & Rasmussen, D. (1994). A whole new increased character sampling on the phylogeny world of ancestors: Eocene anthropoideans of Cetartiodactyla (Mammalia): Combined from Africa. Evolution and Anthropology, 3, analysis including fossils. Cladistics, 23, 1–46. 128–139.Pilbeam, D. (2002). Perspectives on the Miocene Simpson, G. G. (1945). The principles of Hominoidea. In: W. Hartwig (Ed.), The primate classification and a classification of mammals. fossil record (pp. 303–310). Cambridge: Bulletin of the American Museum of Natural University of Cambridge Press. History, 85, 1–350.Rasmussen, D. (1986). Anthropoid origins: Simpson, G. G. (1961). Principles of animal A possible solution to the Adapidae- taxonomy. New York: Columbia University Omomyidae paradox. Journal of Human Press. Evolution, 15, 1–12. Sokal, R. R., & Sneath, P. H. A. (1963). Principles ofRasmussen, D. (2002). Early catarrhines of the numerical taxonomy. San Francisco: African Eocene and Oligocene. In: W. Hartwig W. H. Freeman & Co. (Ed.), The primate fossil record (pp. 203–220). Stephan, H. (1972). Evolution of primate brains: Cambridge: University of Cambridge Press. A comparative anatomical investigation. In:Roos, C., Schmitz, J., & Zischler, H. (2004). R. H. Tuttle (Ed.), The functional and Primate jumping genes elucidate strepsirrhine evolutionary biology of primates (pp. 155–174). phylogeny. Proceedings of the National Academy Chicago: Aldine-Atherton. of Sciences, 101, 10650–10654. Stephan, H., & Andy, O. (1969). QuantitativeRose, K., & Bown, T. (1991). Additional fossil comparative neuroanatomy of primates: An evidence on the differentiation of the earliest attempt at a phylogenetic interpretation. euprimates. Proceedings of the National Annals of the New York Academy of Science, Academy of Sciences, 88, 98–101. 167, 370–387.Rossie, J., Ni, X., & Beard, K. C. (2006). Cranial Szalay, F. S., & Katz, C. (1973). Phylogeny of remains of an Eocene tarsier. Proceedings of the lemurs, galagos, and lorises. Folia Primatology, National Academy of Sciences, 103, 4381–4285. 19, 88–103.Schmitt, D., Cartmill, M., Griffin, T., Hanna, J. B., Thewissen, J., Williams, M., Roe, L., & Hussain, S. & Lemelin, P. (2006). Adaptive value of ambling (2001). Skeletons of terrestrial cetaceans and the gaits in primates and other mammals. Journal of relationship of whales to artiodactyls. Nature, Experimental Biology, 209, 2042–2049. 413, 277–281.Schmitz, J., & Zischler, H. (2004). Molecular cladistic Tosi, A., Disotell, T., Morales, J., & Melnick, D. markers and the infraordinal phylogenetic (2003). Cercopithecine Y-chromosome data relationships of primates. In: C. Ross & R. F. Kay provide a test of competing morphological (Eds.), Anthropoid origins: New visions (pp. evolutionary hypotheses. Molecular 65–77). New York: Kluwer/Academic. Phylogenics and Evolution, 27, 510–521.Schultz, A. (1969). The life of primates. London: Wildman, D., Chen, C., Erez, O., Grossman, L., Weidenfeld and Nicholson. Goodman, M., & Romero, R. (2006).Seiffert, E., Simons, E., & Attia, Y. (2003). Fossil Evolution of the mammalian placenta revealed evidence for an ancient divergence of lorises and by phylogenetic analysis. Proceedings of the galagos. Nature, 422, 421–424. National Academy of Sciences, 103, 3203–3208.

30 PRIMATE NEUROETHOLOGYWolin,L., & Massopust, L. (1970). Morphology of the Molecular Phylogenics and Evolution, 37, primate retina. In: C. Noback and W. Montagna 872–880. (Eds.), The primate brain (pp. 1–28). New York: Yoder, A., Cartmill, M., Ruvolo, M., Smith, K., & Appleton-Century-Crofts. Vilgalys, R. (1996). Ancient single origin forXing, J., Wang, H., Han, K., Ray, D., Huang, C., Malagasy primates. Proceedings of Chemnick, L., et al. (2005). A mobile element the National Academy of Sciences, 93, based phylogeny of Old World monkeys. 5122–5126.

CHAPTER 3Primate Locomotor Evolution:Biomechanical Studies of Primate Locomotionand Their Implications for UnderstandingPrimate NeuroethologyDaniel SchmittPrimate locomotor diversity is extraordinary 1996 for a review of this work). Primates arewhen compared to most other vertebrate critical models for human clinical research notgroups with at least three forms of locomotion only because they are our closest relatives, butnot seen among any other mammals alive today. also because primates (including humans)One of those unique forms of locomotion is our show patterns of locomotion and locomotorown upright striding bipedal gait. The 65 mil- control that are different from all other mam-lion–year story of how our lineage departed mals. For example, the important (albeit con-from a tiny mammalian ancestor and evolved troversial) theory that primates as a group lackthe locomotor variation we see today involves a central pattern generators and rely more onseries of profound shifts in the way primates use supraspinal control of motion compared totheir limbs. These dramatic changes in limb other mammals has stimulated research thatfunction reflect major adaptive shifts and loco- has led to a deeper understanding and treat-motor innovations during primate and human ment of spinal injury (see Duysens & van deevolution. These changes in limb function may Crommert, 2007; Grillner and Wallen 1985;also reflect profound differences in the neural Vilensky & Larson, 1989; Vilensky &control of locomotion between primates and O’Connor, 1997, 1998).almost all other animals. In addition to the profound clinical and In the past 50 years primate models have basic science insights, biomechanical studiesplayed an important role in studies of brain that compare primates to other mammalsand spinal cord injury and pathology (i.e., have allowed us to better understand andCourtine et al., 2005a,b, and Xiang et al., 2007, reconstruct the evolution of locomotor beha-for some of the most recent work, and vior in our Order and the underlyingCapitanio and Emborg, 2008, for a review of adaptive foundations of patterns of primatethe history of this research). Beginning with the gait. Table 3.1 provides a summary of biome-pioneering work of Harvard neurobiologist chanical studies of primate locomotion thatDerek Denny-Brown, studies of locomotor pat- may be a resource for those neuroethologiststerns in healthy primates and those with spinal interested in the underlying differencesand brain lesions have led to significant insight both within primates and across orders. Thisinto the neural control of human locomotion chapter describes the extreme locomotor(see Gilman, 1982, and Vilensky et al., 1994a, diversity found within primates and describes 31

Table 3.1 A Representative* List of Experimental Studies of Primate LocomotionSource Taxa Data Movement(s)Cartmill et al., 2002, 2007a,b All T TQ, AQHildebrand, 1967 All T TQLarson, 1998; Larson et al. 1999, 2001 All K TQ, AQLemelin & Schmitt, 1998 All K TQ, AQReynolds, 1985 All T, FP TQReynolds, 1987 All T, K TQ, TBVilensky, 1987, 1989; Vilensky & Gehlsen, 1984; All T, K, EMG TQ Vilensky & Larson, 1989Aerts et al., 2000 Hom T TQ, TBChang et al., 1997, 2000; Bertram & Chang, 2001 Hom FP AS ˆD’Aout et al., 2002 Hom T, K TQ, TBElftman, 1944; Elftman & Manter, 1935 Hom K, T TBJenkins, 1972 Hom K TBKimura, 1990, 1991, 1996 Hom T, En TQLarson & Stern, 1986, 1987 Hom EMG TQ, AQ, RLarson et al., 1991 Hom EMG AS, TQ, RLarson, 1988, 1989 Hom EMG ASOkada & Kondo, 1982; Okada, 1985 Hom EMG TBProst, 1967, 1980 Hom K, T TQ, TB, VCShapiro et al., 1997 Hom EMG, T TQStern & Larson, 2001 Hom EMG TQ, ASStern & Susman, 1981 Hom EMG TQ, TB, VCSusman, 1983 Hom K TQ, TBSwartz et al., 1989 Hom BS ASTardieu et al., 1993 Hom K TBTuttle & Basmajian, 1974a,b,c, Hom EMG TQ, TB, AS 1977, 1978a,b; Tuttle et al., 1983, 1992Vereecke et al., 2003, 2006 Hom FP TQWunderlich & Jungers, 2009; Hom Pr TQ, AQ Wunderlich & Ford, 2000Yamazaki & Ishida, 1984 Hom K, T TB, VCJenkins et al., 1978 NWM K, C, ASProst & Sussman, 1969 NWM K, T IQSchmitt, 2003a NWM FP, K, T AQ, TQTurnquist et al., 1999 NWM K ASVilensky & Patrick, 1985 NWM T, K TQVilensky et al., 1994 NWM T, K IQFleagle et al., 1981 NWM, Hom EMG, BS VC, TQ, TBIshida et al., 1985 NWM, Hom EMG TQ, TBJungers & Stern, 1980, 1981, 1984 NWM, Hom EMG ASStern et al., 1977, 1980 NWM, Hom EMG AQ, VCTaylor & Rowntree, 1973 NWM, Hom EnHirasaki et al., 1993, 1995, 2000 NWM, OWM T, K, FP, EMG VCProst, 1965, 1969 NWM, OWM T TQKimura et al., 1979; Kimura, 1985, 1992 NWM, OWM, Hom FP TQKimura et al., 1983 NWM, OWM, Hom T TQ, TBSchmitt & Larson, 1995 NWM, OWM, Hom K TQ, AQVangor & Wells, 1983 NWM, OWM, Hom EMG TQ, TB, VC (continued) 32

PRIMATE LOCOMOTOR EVOLUTION 33Table 3.1 (Continued)Source Taxa Data Movement(s)Wunderlich & Schmitt, 2000 NWM, OWM, Hom K TQ, AQDemes et al., 1994 OWM BS TQLarson & Stern, 1989; 1992 OWM EMG TQMeldrum, 1991 OWM K, T AQ, TQPolk, 2002 OWM T, FP, K TQRollinson & Martin, 1981 OWM T AQ, TQSchmitt et al., 1994 OWM EMG TQShapiro & Raichlen, 2005 OWM K TQWells & Wood, 1975 OWM K TQ, LSchmitt, 1994, 1998, 1999, 2003b OWM K, FP TQ, AQVilensky, 1980, 1983, 1988; Vilensky & OWM K, T TQ Gankiewicz, 1986, 1990, Vilensky et al., 1986,1990, 1991Whitehead & Larson, 1994 OWM K, C, EMG TQAlexander & Maloiy, 1984 OWM, Hom T TQShapiro & Jungers, 1988, 1994 OWM, Hom EMG, T TQ, TB, VCAnapol & Jungers, 1987 Pro EMG, T TQ, LCarlson et al., 2005 Pro FP TQ, AQDemes et al., 1990 Pro T AQDemes et al., 1998, 2001 Pro FP LFranz et al., 2005 Pro K, FP TQ, AQGunther, 1991 Pro FP, EMG LIshida et al., 1990 Pro T, FP AQJouffroy, 1983; Jouffroy & Gasc, 1974; Pro K, C AQ Jouffroy et al., 1974Jouffroy & Petter, 1990 Pro T, K, AQJouffroy & Stern, 1990 Pro EMG AQJungers & Anapol, 1985 Pro T, EMG TQSchmidt & Fischer, 2000 Pro K, C AQSchmitt & Lemelin, 2002 Pro FP TQ, AQShapiro et al., 2001 Pro K AQStevens, 2001; Stevens et al. 2001 Pro K, T AQ, IAQTaxa: All, representative species from all of the taxa; Hom, hominoid; NWM, New World monkey; OWM, Old Worldmonkey; Pro, prosimian.Data: BS, bone strain; C, cineradiography; EMG, electromyography; En, energetics; FP, force plate; K, kinematics;Pr, pressure; T, temporal characters.Movements: AQ, arboreal quadrupedalism; AS, arm-swinging; IAQ, inclined quadrupedalism (pole); IQ, inclinedquadrupedalism (flat substrate); L, leaping; R, reaching; TB, terrestrial bipedalism; TQ, terrestrial quadrupedalism.*This is not an exhaustive list of all studies on primate locomotion. I have included those studies that focus specifically onprimate locomotor mechanics primarily in a laboratory setting. I apologize to anyone who was excluded. A review ofmany experimental studies can be found in Fleagle (1979), Jouffroy (1989), Churchill and Schmitt (2003), and Lemelinand Schmitt (2007). The above table does not include studies by anthropologists that focus solely on human bipedalismlike those of Li and colleagues (1996), Schmitt and colleagues (1996, 1999), or Crompton and colleagues (1998).laboratory and field studies that have illumi- significance of fundamental changes in the rolenated significant differences between primates of the forelimb associated with the origin ofand other animals. These two aspects of pri- primates, the evolution of suspensory beha-mate locomotion—diversity and difference— vior, and the evolution of upright, stridingare brought together to interpret the adaptive bipedalism.

34 PRIMATE NEUROETHOLOGYPRIMATE LOCOMOTOR are some primates that swing like pendulumsDIVERSITY through the trees and others that walk on their fingers, their fists, or even their knuckles. Finally,Classifying the locomotion of any animal is dif- a select group of primates walks on two legsficult. Primates are especially difficult. Stern and exclusively. This wide is made more complexOxnard (1973) argued that a system of classifi- by the reality that many primate species cancation must be an aid to thinking and commu- use several of those different modes equallynication. But with primates this is an almost adroitly. That flexibility was the key to the suc-impossible goal. There are many primates that cessful radiation of our Order.can effectively walk, run, leap, climb, and swing At its core, the Order Primates is an arborealthrough the trees. What name can we give to that radiation. Although several species in Africa andrepertoire? What category would such an animalfit into? Asia spend most of their time on the ground The Order Primates contains between 100 (Table 3.1), the vast majority of primates,and 400 (see Chapter 2 for a review) recognized including all those in South America, restrictspecies (Table 3.2). Figure 3.1 shows a small their movements to the trees, and all primatessample of this diversity. Our Order includes move into the tress to sleep and escape predators.many primates that walk and run in a manner The physical characteristics that we use tothat is superficially similar to dogs and cats. But define primates—grasping, prehensile handsthere are also many that leap large distances with nails instead of claws and forward-facingbetween horizontal or vertical supports. There eyes (Figure 3.2)—have long been associatedTable 3.2 List of Primates and Their Most Commonly Used Substrates and Locomotor Modes SubstrateCommon Name Species Name Used Locomotor ModeSUBORDER STREPSIRRHINIINFRAORDER LEMURIFORMESSuperfamily Lemuroidea Family Cheirogaleidae Fat-tailed dwarf lemur Cheirogaleus medius Arboreal Quadruped/leaper Greater dwarf lemur Cheirogaleus major Arboreal Quadruped/leaper Lesser mouse lemur Microcebus murinus Arboreal Quadruped/leaper Coquerel’s mouse lemur Mirza coquereli Arboreal Quadruped/leaper Hairy-eared dwarf lemur Allocebus trichotis Arboreal Quadruped/leaper Fork-crowned lemur Phaner furcifer Arboreal Quadruped/clingerFamily Lemuridae Ring-tailed lemur Lemur catta Terrestrial/ Quadruped/leaper arboreal Black lemur Eulemur macaco Arboreal Quadruped/leaper Brown lemur Eulemur fulvus Arboreal Quadruped/leaper Mongoose lemur Eulemur mongoz Arboreal Quadruped/leaper Crowned lemur Eulemur coronatus Arboreal Quadruped/leaper Red-bellied lemur Eulemur rubriventer Arboreal Quadruped/leaper Gray gentle lemur Hapalemur griseus Arboreal Quadruped/leaper Golden gentle lemur Hapalemur aureus Arboreal Quadruped/leaper Broad-nosed gentle lemur Hapalemur simus Arboreal Quadruped/leaper Ruffed lemur Varecia variegata Arboreal Quadruped (continued)

Table 3.2 (Continued) SubstrateCommon Name Species Name Used Locomotor ModeFamily Lepilemruidae Weasel sportive lemur Lepilemur mustelinus Arboreal Vertical clinger & leaperFamily Indriidae Indri Indri indri Arboreal Vertical clinger & leaper Eastern woolly lemur Avahi laniger Arboreal Vertical clinger & leaper Western woolly lemur Avahi occidentalis Arboreal Vertical clinger & leaper Diademed sifaka Propithecus diadema Arboreal Vertical clinger & leaper Verreaux’s sifaka Propithecus verreauxi Arboreal Vertical clinger & leaperFamily Daubentoniidae Aye-aye Daubentonia Arboreal Quadruped madagascariensisSuperfamily Lorisoidea Family Loridae Golden potto Arctocebus aureus Arboreal Quadruped potto Perodicticus potto Arboreal Quadruped Slender loris Loris tardigradus Arboreal Quadruped Slow loris Nycticebus coucang Arboreal Quadruped Lesser slow loris Nycticebus pygmaeus Arboreal QuadrupedFamily Galagidae Greater galago Otolemur crassicaudatus Arboreal Quadruped/leaper Garnett’s galago Otolemur garnettii Arboreal Quadruped/leaper Southern needle-clawed Euoticus elegantulus Arboreal Quadruped/leaper galago Lesser galago Galago senegalensis Arboreal Quadruped/leaper Southern lesser galago Galago moholi Arboreal Quadruped/leaper Allen’s galago Galago alleni Arboreal Quadruped/leaper Demidoff’s galago Galago demidoff Arboreal Quadruped/leaperSUBORDER HAPLORHINIINFRAORDER TARSLLFORMESFamily Tarsiidae Philippine tarsier Tarsius syrichta Arboreal Vertical clinger & leaper Horsfield’s tarsier Tarsius bancanus Arboreal Vertical clinger & leaper Spectral tarsier Tarsius spectrum Arboreal Vertical clinger & leaperINFRAORDER PLATYRHINISuperfamily Ceboidea Family Cebidae Dusky titi Callicebus moloch Arboreal Quadruped Northern owl monkey Aotus trivirgatus Arboreal Quadruped White-faced saki Pithecia pithecia Arboreal Quadruped Monk saki Pithecia monachus Arboreal Quadruped White-nosed saki Chiropotes albinasus Arboreal Quadruped Black-headed uakari Cacajao melanocephalus Arboreal Quadruped Red uakari Cacajao calvus Arboreal Quadruped White-throated capuchin Cebus capucinus Arboreal Quadruped White-fronted capuchin Cebus albifrons Arboreal Quadruped (continued) 35

Table 3.2 (Continued) SubstrateCommon Name Species Name Used Locomotor Mode Weeper capuchin Cebus olivaceus Arboreal Quadruped Black-capped capuchin Cebus apella Arboreal Quadruped Common squirrel monkey Saimiri sciureus Arboreal Quadruped/leaper Family Atelidae Mantled Alouatta palliata Arboreal Quadruped howler monkey Red howler monkey Alouatta seniculus Arboreal Quadruped Brown howler monkey Alouatta fusca Arboreal Quadruped Black howler monkey Alouatta caraya Arboreal Quadruped Black spider monkey Ateles paniscus Arboreal Arm-swinger/quadruped Long-haired spider monkey Ateles belzebuth Arboreal Arm-swinger/quadruped Brown-headed spider Ateles fusciceps Arboreal Arm-swinger/quadruped monkey Black-handed spider Ateles geoffroyi Arboreal Arm-swinger/quadruped monkey Woolly spider monkey Brachyteles arachnoides Arboreal Quadruped/arm-swinger Humboldt’s woolly monkey Lagothrix lagotricha Arboreal Quadruped/arm-swinger Yellow-tailed woolly Lagothrix flavicauda Arboreal Quadruped/arm-swinger monkeyFamily Callitrichidae Goeldi’s marmoset Callimico goeldii Arboreal Vertical clinger & leaper Common marmoset Callithrix jacchus Arboreal Vertical clinger & leaper Geoffroy’s tufted-ear Callithrix geoffroyi Arboreal Vertical clinger & leaper marmoset Buffy-headed Marmoset Callithrix flaviceps Arboreal Vertical clinger & leaper Silvery marmoset Callithrix argentata Arboreal Vertical clinger & leaper Pygmy marmoset Cebuella pygmaea Arboreal Vertical clinger & leaper Golden lion tamarin Leontopithecus rosalia Arboreal Vertical clinger & leaper Golden-headed lion tamarin Leontopithecus Arboreal Vertical clinger & leaper chrysomelas Midas tamarin Saguinus midas Arboreal Vertical clinger & leaper Black-mantled tamarin Saguinus nigricollis Arboreal Vertical clinger & leaper Saddle-back tamarin Saguinus fuscicollis Arboreal Vertical clinger & leaper Black-chested moustached Saguinus mystax Arboreal Vertical clinger & leaper tamarin Red-bellied tamarin Saguinus labiatus Arboreal Vertical clinger & leaper Emperor tamarin Saguinus imperator Arboreal Vertical clinger & leaper Pied tamarin Saguinus bicolor Arboreal Vertical clinger & leaper Cotton-top tamarin Saguinus oedipus Arboreal Vertical clinger & leaperINFRAORDER CATARRHINISuperfamily Cercopithecoidea Family Cercopithecidae Subfamily Cercopithecinae Allen’s swamp monkey Allenopithecus Terrestrial Quadruped nigroviridis Talapoin monkey Miopithecus talapoin Arboreal Quadruped Patas monkey Erythrocebus patas Terrestrial Quadruped (continued) 36

Table 3.2 (Continued) SubstrateCommon Name Species Name Used Locomotor Mode Vervet monkey Cercopithecus aethiops Terrestrial/ Quadruped arboreal Diana monkey Cercopithecus diana Arboreal Quadruped Greater white-nosed Cercopithecus nictitans Arboreal Quadruped monkey Blue monkey Cercopithecus mitis Arboreal Quadruped Mona monkey Cercopithecus mona Arboreal Quadruped Campbell’s monkey Cercopithecus campbelli Arboreal Quadruped Crowned guenon Cercopithecus pogonias Arboreal Quadruped Wolf’s monkey Cercopithecus wolfi Arboreal Quadruped Lesser white-nosed monkey Cercopithecus petaurista Arboreal Quadruped Redtail monkey Cercopithecus ascanius Arboreal Quadruped L’hoest’s monkey Cercopithecus lhoesti Terrestrial/ Quadruped arboreal Preuss’s monkey Cercopithecus preussi Arboreal Quadruped DeBrazza’s monkey Cercopithecus neglectus Terrestrial/ Quadruped arboreal Barbary macaque Macaca sylvanus Terrestrial Quadruped Lion-tailed macaque Macaca silenus Terrestrial/ Quadruped arboreal Pigtailed macaque Macaca nemestrina Terrestrial Quadruped Tonkean macaque Macaca tonkeana Terrestrial Quadruped Celebes macaque Macaca nigra Terrestrial Quadruped Crab-eating macaque Macaca fascicularis Arboreal Quadruped Stumptailed macaque Macaca arctoides Terrestrial/ Quadruped arboreal Rhesus macaque Macaca mulatta Terrestrial/ Quadruped arboreal Japanese macaque Macaca fuscata Terrestrial/ Quadruped arboreal Bonnet macaque Macaca radiata Terrestrial/ Quadruped arboreal Gray-cheeked mangabey Lophocebus albigena Arboreal Quadruped Black mangabey Lophocebus aterrimus Arboreal Quadruped Hamadryas baboon Papio hamadryas Terrestrial Quadruped Guinea baboon Papio papio Terrestrial Quadruped Olive baboon Papio anubis Terrestrial Quadruped Yellow baboon Papio cynocephalus Terrestrial Quadruped Chacma baboon Papio ursinus Terrestrial Quadruped Gelada baboon Theropithecus gelada Terrestrial Quadruped Sooty mangabey Cercocebus atys Terrestrial/ Quadruped Arboreal White-collared mangabey Cercocebus torquatus Terrestrial/ Quadruped Arboreal Agile mangabey Cercocebus agilis Terrestrial/ Quadruped Arboreal Tana river mangabey Cercocebus galeritus Arboreal Quadruped (continued) 37

Table 3.2 (Continued) SubstrateCommon Name Species Name Used Locomotor Mode Mandrill Mandrillus sphinx Arboreal/ Quadruped Terrestrial Drill Mandrillus leucophaeus Terrestrial/ Quadruped arborealSubfamily Colobinae Black colobus monkey Colobus satanas Arboreal Quadruped Angolan black-and-white Colobus angolensis Arboreal Quadruped colobusWestern black-and-white Colobus polykomos Arboreal Quadruped colobusAbyssinian black-and-white Colobus guereza Arboreal Quadruped colobus Western red colobus Procolobus badius Arboreal Quadruped monkey Olive colobus monkey Procolobus verus Arboreal Quadruped Hanuman langur Semnopithecus entellus Terrestrial/ Quadruped arboreal Purple-faced langur Trachypithecus vetulus Arboreal Quadruped Silvered leaf monkey Trachypithecus cristatus Arboreal Quadruped Tenasserim langur Trachypithecus barbei Arboreal Quadruped Dusky leaf monkey Trachypithecus obscurus Arboreal Quadruped Delacour’s langur Trachypithecus delacouri Arboreal Quadruped Mitered leaf monkey Presbytis melalophos Arboreal Quadruped Douc langur Pygathrix nemaeus Arboreal Quadruped Snub-nosed langur Pygathrix roxellana Arboreal Quadruped Proboscis monkey Nasalis larvatus Arboreal QuadrupedSuperfamily Hominoidea Family Hylobatidae White-handed gibbon Hylobates lar Arboreal Brachiator Siamang Hylobates syndactylus Arboreal BbrachiatorFamily Hominidae Orangutan Pongo pygmaeus Arboreal Quadrumanous/ arm-swinger Gorilla Gorilla gorilla Terrestrial Knuckle-walker Pygmy chimpanzee Pan paniscus Arboreal/ Knuckle-walker/ (bonobo) terrestrial arm-swinger Common chimpanzee Pan troglodytes Terrestrial/ Kknuckle-walker/ arboreal arm-swinger Human Homo sapiens Terrestrial BipedThis list of primates is derived from a review of animals listed by Fleagle (1999), Szalay and Delson (1979), andRowe (1996). This is not meant as a definitive or inarguable phylogenetic arrangement of primates (see Cartmill,this volume for a critical review of primate phylogeny). This list is serve as a basis for describing the variation inlocomotor behavior in an organized list. I have chosen to list a subset of the nearly 400 species that one might beincluded in a comprehensive list. For example, the many identified species of night monkey are not listed becausetheir locomotion does not differ. The locomotor classification is based on Rose (1973), Oxnard et al. (1990), andFleagle (1999). 38

PRIMATE LOCOMOTOR EVOLUTION 39Figure 3.1 This painting (‘‘Darwin and Friends,’’ Stephen Nash, 1985) illustrates the remarkable locomotordiversity in the order. A terrestrial quadrupedal lemur and monkey are shown on the ground to the left, theknuckle-walking apes are on the ground to the right, arboreal quadrupeds are on the middle and high branches,an arm-swinging monkey and ape are hanging in the center, and the vertical clingers and leapers are shown onthe primary trunk. Finally, our own bipedalism is represented on the right. Used with permission.Figure 3.2 This figurehighlights two pieces of a largersuite of features that defineprimates. (A) Forward-facingeyes in the primate (right) allowdetailed stereoscopic vision anddepth perception. (B) Graspinghands with nails in primates(right) allow effective gripping ofbranches as well as single-handed, prehensile capture andmanipulation of objects.Modified from Cartmill, M.(1992). New views on primateorigins. Evolution andAnthropology, 1, 105–111. Usedwith permission.with arboreal habits. Smith (1912) and Jones suggested that grasping hands and stereoscopic(1916) were among the first to relate some of vision met the critical need to leap great distancesthe unique anatomical and behavioral character- and land safely. Their views were further pro-istics of primates to living in the trees. They moted in writings of LeGros Clark (1959) but

40 PRIMATE NEUROETHOLOGYlater challenged and refined by Cartmill (1970, behavioral, and neurological qualities that define1972, 1974a,b, 1992), who suggested that the for- primates. As Jenkins (1974: 112) noted concisely,ward-facing eyes and grasping extremities of pri- ‘‘The adaptive innovation of ancestral primatesmates should be interpreted not as adaptations was therefore not the invasion of the arborealfor leaping, but rather as adaptations to cautious, habitat, but their successful restriction to it.’’ Sonocturnal foraging for insect prey on thin, flexible it is reansonable to begin this exercise of classi-branches. Recent morphological and laboratory- fying primate locomotion by saying that thebased data support the latter argument (Cartmill underlying primate bauplan reflects a history ofet al., 2002; Hamrick, 1998; Lemelin, 1999; arboreal quadrupedal locomotion on relativelySchmitt, 1999; Schmitt & Lemelin, 2002). thin supports.Others have suggested that primates evolved Another broad statement that can be madetheir unusual physical characteristics in associa- is that most primates rely primarily on a quad-tion with foraging for fruits rather than insects rupedal mode of locomotion (Fig. 3.3). Of the(Sussman, 1991). Exactly which aspects of arbore- 149 living species of primates listed in Table 3.2,ality influenced the development of primate at least 107 them rely heavily on quadrupedalanatomy remains a subject of continued debate locomotion (Rose, 1973). However, as has been(Cartmill, 1992; Cartmill et al. 2002, 2007a,b; recognized by many authors, the term ‘‘quad-Lemelin & Schmitt, 2007; Raichlen & Shapiro, ruped’’ masks a remarkable diversity(Rose2006; Schmitt & Lemelin, 2002; Sussman, 1991; 1973,Hunt et al.1996). Quadrupedal primatesSzalay & Dagosto, 1988; Szalay et al., 1987). But can be broken down, as Fleagle and Mittermierthe fact remains that the challenges of moving (1980) did, into ‘‘quadrupedal runners’’ andand foraging in a thin, flexible, terminal branch ‘‘quadrupedal leapers.’’ This difference is subtlemilieu appear to be at the core of the anatomical, but has important implications for anatomy andFigure 3.3 Quadrupedalism is the most common mode of locomotion among primates (Rose, 1973;Table3.2). Quadrupedalism is found in strepsirrhines, New and Old World monkeys, and apes as isillustrated here moving counter clockwise from left by this ring-tailed lemur, yellow-tailed woollymonkey, diana monkey, and gorilla. Quadrupedalism is found in terrestrial and arboreal primates and inprimates as small as 50 grams and as large as 500 kg. Drawings by Stephen Nash; courtesy of ConservationInternational.

PRIMATE LOCOMOTOR EVOLUTION 41ecology. This was made clear in a seminal study between horizontal supports and show signifi-in which Fleagle (1977a,b) was able to show that cant differences in the detailed anatomy oftwo closely related species of Asian colobines muscle markings and joint shape in their lowerthat we readily classify as arboreal quadru- limb (Fig. 3.4).peds—the dusky leaf monkey and the banded The primates that we label as quadrupedsleaf monkey—use different amounts of leaping show greater diversity than simple variation inFigure 3.4 The banded leaf monkey (Presbytis melalophos) (left) and dusky leaf monkey (Trachypithecusobscura) (right) are sympatric and closely related but they exhibit dramatic differences in the percentage oftime spent leaping relative to the amount of time spent walking quadrupedally (A). These differences in thefunctional role of the hindlimbs are reflected in joint surface shape and area (B) as well as limb length andback muscle weight (C). Modified from Fleagle, J. G. (1999). Primate adaptation and evolution. New York:Academic. Used with permission.

42 PRIMATE NEUROETHOLOGYthe amount of leaping. For example, Byron and There are of course some primates that areCovert (2004) have recently reported that, dedicated to specialized locomotor modes. I willdespite having upper and lower limbs of nearly simplify this down to four groups: (1) verticalequal length, the rare red-shanked douc langur clinging and leaping (VCL), (2) arm-swinging/(Fig. 3.5a) can swing by its arms as elegantly as brachiation, (3) knuckle and fist walking,dedicated arm-swingers like the spider monkey and (4) bipedalism. The first two categoriesof South America (Fig. 3.5a) and the gibbon of represent opposites on the spectrum of fore-Southeast Asia (Fig. 3.5b). Even the more pro- limb-dominated versus hindlimb-dominatedsaic quadrupedal monkeys of the South locomotion (Fig. 3.6). The second two categoriesAmerican and African rainforest show distinct represent solutions to the problem of using fore-differences in which parts of the canopy they limbs highly modified for the demands ofoccupy (Fleagle & Mittermeir, 1980; Gebo & arboreal movement during terrestrial locomo-Chapman, 1995; McGraw, 1996, 1998a,b,c) and tion (Fig. 3.7).anatomical differences associated with those Vertical clinging and leaping behavior ischoices. Oxnard and colleagues (1990) and found among at least five living strepsirrhineDagosto (1995) have shown great variation in genera and in three living haplorhine generathe nature of quadrupedalism in Madagascar (the tarsier, marmosets, and tamarins). In thislemurs. To try to make sense out of all this, remarkable locomotor mode, animals cling to aHunt and colleagues (1996) identified a wide vertical support and launch themselvesvariety of locomotor modes among quadrupedal with powerful extension of the lower limbmonkeys. Though they have standardized the (Fig. 3.6a). In the case of small leapers likedescriptions of these categories, their lengthy galagos, the leap is driven primarily by extensionlist of types of movement can be difficult to at the ankle, whereas in larger leapers the exten-penetrate. That fact alone reveals how complex sion is primarily at the hip Demes and Gunther,primate locomotion is. 1995; (Jouffroy & Lessertisseur, 1979). SmallFigure 3.5 The douc langur (A) arm-swinging through trees (from Byron, C. D. and Covert H. H.(2004). Unexpected locomotor behaviour: brachiation by an Old World monkey (Pygathrix nemaeus)from Vietnam. Journal of Zoology, 263, pp. 101–106) in a manner similar to that of gibbons or spidermonkeys (B) (from Turnquist, J. E., Schmitt, D., Rose, M.D., & Cant, J. G. [1999]. Pendular motion inthe brachiation of captive Lagothrix and Ateles. American Journal of Primatology, 48, 263–281). Used withpermission.

PRIMATE LOCOMOTOR EVOLUTION 43Figure 3.6 Primate locomotion exhibits extremes in a continuum of ‘‘hindlimb-dominated’’ to‘‘forelimb-dominated’’ modes of locomotion. Sifaka (A) exhibit powerful leaps using long muscular legs.Gibbons (B) exhibit acrobatic brachiation with their relative long forelimbs. Drawings by Stephen Nash;courtesy of Conservation International.Figure 3.7 Differences in digit form and hand posture during terrestrial locomotion in the baboon (A) andthe chimpanzee (B). The baboon has short, thick fingers and adopts a digitigrade posture. The chimpanzeehas long, curved fingers and adopts knuckle-walking hand postures when walking quadrupedally (Imagesmodified from Fleagle, J. G. (1999). Primate adapation and evolution. New York: Academic). Used withpermission.leapers, in contrast, have extended tarsal bones, Though the distinction between arm-swinging andwhereas large leapers have exceptionally long brachiation is imprecise, it is generally argued thatfemora (Demes and Gunther, 1995; Jouffroy & brachiation involves whole body rotation (Cant,Lessertisseur, 1979). During the leap the animals 1987a,b; Larson, 1988). Arm-swinging is seenrotate and land on the next vertical support, among South American primates like the spidergripping it with their feet (Dunbar, 1988; monkey (Fig. 3.5b) and wooly spider monkey, theNapier & Walker, 1967). douc langur, and orangutans. Brachiation is seen in In contrast, arm-swinging locomotion is driven gibbons and chimpanzees. Ricochetal brachiation,entirely by hand-over-hand motion of the forelimb. in which there is a whole body aerial phase, is found

44 PRIMATE NEUROETHOLOGYin gibbons (Fig. 3.6b) and sometimes chimpanzees. DIFFERENCES BETWEENIn all cases, the limbs are used in tension and are PRIMATE AND NONPRIMATEgenerally long and gracile. When such highly spe- LOCOMOTOR BEHAVIORcialized animals walk on branches or the ground,they bear very little weight on their forelimbs com- The fact that we are unique among primates inpared to their hindlimbs (Kimura et al., 1979; using a striding, bipedal form of locomotionReynolds, 1985; Schmitt, 1994) or even none at all begs a number of critical questions, most ofin the case of the bipedal gibbon. which are debated in detail elsewhere (see Digit morphology is one of the aspects of pri- Schmitt, 2003c; Stern 2000; and Ward, 2002 formate anatomy most sensitive to locomotor a review), not directly relevant to neuroethology,demands. The feet and hands are in direct contact and well beyond the scope of this chapter. Thesewith the substrate and thus strongly reflect the questions include the following: Which apes arecomplex interaction between locomotor behavior our immediate ancestors? In what environmentand the environment. The relationship between did bipedalism first evolve? Why did humansforces produced by the animal and substrate reac- become bipedal?tion forces is strongly influenced by the position Embedded in these questions are additionaland design of the hand. There is a strong need to questions that this chapter will address: Whatbalance bone length, which increases leverage and are the fundamental differences between thedistance, against bone strength, which increases locomotion of primates compared to other ani-safety but also mass and cost. Animals that swing mals? Did these differences, which probablyby their arms develop long and curved phalanges accrued early in the evolution of the primate(Richmond et al., 2002) that are relatively gracile. lineage, facilitate the evolution of this startlingTerrestrial quadrupeds like the baboon shorten array of specialized locomotor behaviors,and thicken their finger bones and bear weight including our own, that we see today?directly at the base of the fingers and heads of If we start with the model that the basic pri-metacarpal bones (Fig. 3.7a). That option is una- mate bauplan is that of an arboreal quadruped,vailable to arm-swinging primates. When they we can ask what aspects of locomotion havecome to the ground or large supports, animals changed over time to yield vertical clingers andlike the orangutan ball up their fists and bear leapers, brachiators, and bipeds. From this per-weight on the outer edge of the metacarpals and spective, it becomes clear that primate locomotorphalanges. In contrast, chimpanzees choose to evolution is characterized by dramatic changes inwalk on the dorsal surface of the middle phalanx the functional role of the forelimbs. Rather than(See Richmond et al. 2002 for a review) (Fig. 3.7b). have a near-equal division of labor between fore- Finally, we have our own characteristic limbs and hindlimbs as in almost all other leggedform of locomotion: upright, striding biped- vertebrates, there has been a change such that wealism. Rather than bear weight on gracile might describe primates as ‘‘hindlimb domi-limbs, our earliest ancestors removed their nated,’’ relying heavily on the hindlimbs toforelimbs completely from a weight-bearing power locomotion (Kimura et al., 1979; but seerole during locomotion and walked around Demes et al., 1994 for a revision of the concept ofon two legs. The exact nature of bipedalism ‘‘hindlimb-drive’’). The forelimbs of primates, inin our early ancestors is a matter of serious contrast, may be described as ‘‘free’’ to providedebate (see Latimer et al., 1987; Schmitt, stability and guidance (‘‘steering’’) as well as2003a,b; Stern, 2000; Susman et al., 1984; grasping and manipulation. This changed func-and Ward, 2002 for a balanced set of argu- tional relationship between the forelimb and hin-ments), but the fact that they did so and the dlimb is highlighted in the many ways in whichadaptive pathway that primates followed to the walking gaits of primates differ from those ofget there is really the key to understanding other mammals. In every case described later thisthe underlying neural mechanisms of loco- division of labor between forelimbs and hin-motion in primates. dlimbs appears to relate to the biomechanical

PRIMATE LOCOMOTOR EVOLUTION 45challenges of arboreal locomotion and reflects a al., 1999; 2001; Lemelin et al., 2003; Rollinson &forelimb used less in compressive weight support Martin, 1981; Schmitt 1999, 2003a,b; Schmitt &and more in complex movement of guidance and Lemelin, 2002). These unusual gait choicesmanipulation. appear to provide smooth movement, flex- Figure 3.8 illustrates the ways in which most ibility, and security for an animal with noprimates differ from most other mammalian claws walking on relatively thin arboreal sub-quadrupeds (see Demes et al. 1994; Larson, strates. These ecological/behavioral factors1998; Lemelin Schmitt, 1999, 2003a,b,c; Schmitt become critical targets for neural controland Lemelin, 2002; Vilensky, 1989; and Vilensky because of their presumed adaptive value. Byand Larson, 1989 for additional review). The examining these one at a time, a picturequadrupedal walking gaits of most primates can develops of the functional role of each unusualbe distinguished from most other mammals by aspect of primate locomotion.the presence of:1. Diagonal sequence footfall patterns Footfall Pattern2. High degrees of limb protraction Diagonal sequence (DS) walks are symmetrical gaits3. Relatively low peak forelimb forces in which the contact of each hindfoot is followed by4. Relatively low forelimb spring stiffness that of the contralateral forefoot (Fig. 3.8a). During5. The lack of a running trot and the use of an a DS walk the right hindfoot (RH) contact is fol- ambling gait instead lowed by that that of the left forefoot (LF) so that a6. Reduced oscillations of the center of mass. DS gait can be summarized as follows: RH, LF, LH, All of these aspects of primate gait have, at one RF in series. Most primates consistently use DStime or another, been argued to be part of a basal walking gaits, although they can and do use lateraladaptation to locomotion on thin flexible branches sequence (LS = RH, RF, LH, LF) gaits on occasion.(see Cartmill et al., 2002; Larson, 1998; Larson et Most other mammals exclusively use LSFigure 3.8 Summary of the commonly accepted differences that are believed to distinguish the walkinggaits of most primates from those of most nonprimate mammals. Nonprimates generally use (A) lateralsequence walking gaits, (B) have a humerus that at ground contact is retracted relative to a horizontal axispassing through the shoulder, and (C) have greater peak vertical forces on their forelimbs than they do ontheir hindlimbs. Primates show the opposite pattern. From Schmitt, D., & Lemelin, P. (2002). The origins ofprimate locomotion: Gait mechanics of the woolly opossum. American Journal of Physical Anthropology, 118,231–238. Used with permission.

46 PRIMATE NEUROETHOLOGYgaits (Cartmill et al., 2002, 2007a,b; De La Croix, to select both DS and LS gaits. In this model1936; Dunbar & Badam, 2000; Hildebrand, 1967; primates choose to use DS gaits possibly for oneLemelin et al., 2003, 2008; Muybridge, 1887; of the advantages described previously. OtherSchmitt & Lemelin, 2002; Vilensky & Larson, animals do not have that choice. This argument1989; White, 1990). This pattern has been recog- is intuitively appealing and corresponds withnized since Muybridge (1887) first filmed locomo- important well-documented differences in pri-tion of the baboon and described what Magna De La mate locomotor control that have implicationsCroix (1936) would later call the ‘‘pithecoid gait.’’ for treatment of spinal cord injuries. But the pre-Since then many arguments have been advanced to sence of DS gaits in arboreal opossums (Lemelinexplain the preference for DS gaits among primates et al., 2003; Pridmore, 1994), kinkajous (Lemelin(Gray, 1959; Meldrum, 1991; Rollinson & Martin, et al., 2008; McClearn, 1992), and even, about half1981; Vilensky & Larson, 1989). The recent finding the time, the Virginia opossum (White, 1990)by Schmitt and Lemelin (2002) and Lemelin and calls this model into question.colleagues (2003) that the woolly opossum(Caluromys philander), a dedicated fine-branch (2) Limb position at touchdown:arborealist, uses DS gaits almost exclusively sup-ports a link between fine-branch arboreality and A second character believed to be typical of pri-footfall sequence. The closely related terrestrial mar- mate gaits—an arm (humerus) that is protractedsupial, the short-tailed opossum (Monodelphis much further than 90 degrees relative to a hor-domestica), exclusively uses LS gaits (Lemelin et al., izontal axis at touchdown of the forelimb2003). Based on these data on primates and marsu- (Fig. 3.8b)—has also been related to arborealpials, Cartmill and colleagues (2002, 2004, 2007a,b) quadrupedalism (Larson, 1998; Larson et al.,articulated a model that suggested that DS walks 1999, 2001; Lemelin and Schmitt, 2007; Schmitt,were adopted by primates to ensure that a grasping 1998, 1999, 2003a,d). It has been argued that earlyhindfoot is placed on a tested support when the primates, having first evolved flattened nails,contralateral forefoot touches down on an untested required long limbs with large excursions tosupport. Although there has been a small amount of reach above their head or around a trunk duringdebate on this subject (Shapiro & Raichlen, 2005, climbing and to use long smooth, strides that2007), the recent findings of DS gaits in a highly would not oscillate thin branches (Demes et al.,arboreal carnivore (the kinkajou) and the increased 1990; Larson, 1998; Larson et al., 1999, 2001;prevalence of DS gaits on arboreal supports in Schmitt, 1995, 1998, 1999). Lemelin andcapuchin monkeys (Wallace & Demes, 2008) Schmitt (2007) recently compared arm protrac-further supports the association of this footfall pat- tion between Caluromys and Monodelphis. Theytern with locomotion and foraging on thin found that Monodelphis protracts its arm slightlybranches. Thus, the current explanation places a beyond 90 degrees relative to its body axis at‘‘testing’’ role for the forelimb and a ‘‘safety’’ role forelimb touchdown during quadrupedalfor the hindlimb. walking. Lemelin and Schmitt (2007) observed Vilensky and Larson (1989) have argued pre- significantly greater arm protraction at forelimbviously that the presence of DS gaits reflects a touchdown in Caluromys, exceeding that offundamental change in the neural control of loco- Monodelphis by nearly 20 degrees. From thesemotion. They argued that other animals are data, it appears that opossums, like other mar-restricted by central pattern generators to adopt supials in general, are characterized by higheronly LS gaits. Primates, in contrast, are given degrees of arm protraction at forelimb touch-credit for greater supraspinal control, as evi- down than most nonprimate mammals (Larsondenced by the inability of decerebrate primates et al., 1999; Lemelin & Schmitt, 2007). The factto initiate stepping patterns (Vilensky & Larson, that woolly opossums have much greater arm1989; Vilensky & Oconnor, 1997, 1998). It is protraction at forelimb touchdown under-argued that this greater control allows primates scores its close link with arboreality,

PRIMATE LOCOMOTOR EVOLUTION 47particularly locomotion on thin branches. arboreal locomotion (Larney & Larson, 2004;Once again, this difference reflects a changing Larson, 1998; Schmitt, 1998, 1999, 2003a,b,c;role of the forelimb as a primate reaches around Schmitt & Lemelin, 2002). Furthermore, varia-to grasp. This has been argued to also reflect tion in the ratio of forelimb to hindlimb Vpkincreased supraspinal control of locomotion in within primates supports this intuitive assump-primates in general (Vilensky & Larson, 1989), tion. In general, primates that spend most ofbut now we would have to argue the same for their locomotor time in trees have the lowestthe woolly opossum. FL/HL Vpk ratios, whereas the most terrestrial primates have a nearly equal distribution of forelimb and hindlimb vertical peak forcesWeight Distribution (Demes et al., 1994; Kimura, 1985, 1992;A third feature (Fig. 3.8c) thought to distinguish Kimura et al., 1979; Reynolds, 1985; Schmittprimates from other mammals is the presence of & Lemelin, 2002). Kimura (1985, 1992) recog-higher vertical peak (Vpk) substrate reaction nized this pattern and argued strongly that theforces on the hindlimbs than on the forelimbs degree to which forelimb forces are reduced is(Demes et al., 1994; Kimura, 1985, 1992; Kimura directly related to substrate preference. It is easyet al., 1979; Reynolds, 1985; Schmitt & Lemelin, to see the benefits that this pattern would pro-2002). There is variation in this feature, and not vide. As early primates moved and foraged inall primates show statistically significant differ- the trees, they would need to reach out and grabences between forelimb and hindlimb peak food placed off of the path of locomotion andforces (Ishida et al., 1990; Polk, 2000, 2001; they would have to reach out in order to effectSchmitt & Hanna, 2004; Schmitt & Lemelin, often abrupt changes of direction along2002). However, most primates experience branches. In those cases a highly mobile fore-lower forelimb peak forces. Moreover, this dis- limb that was not responsible for bearing 60%parity between forelimb and hindlimb peak of the animal’s body weight would provide aforces is exaggerated on arboreal supports distinct advantage.(Schmitt & Hanna, 2004). This pattern is pro-found and separates most primates from most Limb Stiffnessother animals (Fig. 3.9). The exceptions from this pattern of force It is tempting to argue that this shift in weight-distribution are also informative. Schmitt and bearing role reflects an anatomical redistributionLemelin (2002) have shown that the woolly of weight, but there is no evidence to supportopossum has relatively low peak vertical forces such a claim. Studies of center of mass positionon the forelimbs compared to the hindlimbs show no differences across taxa (Vilensky &(Fig. 3.9) and concluded this was adaptively Larson, 1989). Moreover, short-tailed macaquesadvantageous for locomotion and foraging on show the same pattern and apes, which have nothin, flexible branches. Conversely, Schmitt tails and huge forequarters, are among the most(2003b) reported recently that the common extreme in showing relatively high hindlimbmarmoset (Callithrix jacchus), a clawed primate forces. The difference between forelimb andthat spends much of its time clinging on large hindlimb peak vertical forces must be antree trunks, predominantly uses LS gaits, active shift of load posteriorly and must reflectmore retracted arm positions, and higher peak different patterns of limb use.vertical forces on the forelimbs relative to the So the question remains as to what the actualhindlimbs (Fig. 3.9). mechanism is. Reynolds (1985) argued that These kinetic data support the widely held activity in the hindlimb retractor muscles whileassumption that the difference in weight distri- the hindlimb was highly protracted (reaching for-bution between primates and nonprimate ward) actively shifted weight posteriorly. Schmittmammals represents a basal adaptation to (1998, 1999) and Schmitt and Hanna (2004)

48 PRIMATE NEUROETHOLOGY Squirrel Chipmunk Horse Goat Sheep Dog Cat Short-tailed opossum Woolly opossum (runway) Woolly opossum (pole) Common marmoset Fat-tailed dwarf lemur Brown lemur Patas monkey Talapoin monkey Vervet monkey Macaque Baboon Spider monkey Orangutan Chimpanzee 0.4 0.6 0.8 1 1.2 1.4 1.6 HL > FL FL > HL : Nonprimates : PrimatesFigure 3.9 Bar graph of the ration of peak vertical forces on the forelimb divided by peak vertical forces onthe hindlimb in representative primates and nonprimate mammals. A ratio of 1.0 represents equal weightdistribution. Most primates (red) fall to the side that indicates increased peak vertical forces on thehindlimb, whereas most nonprimate mammals (yellow) fall to the opposite side. The exceptions to thispattern, the highly arboreal woolly opossum and the claw-bearing common marmoset, are seen asindicating the importance of arboreality in driving this functional differentiation. The figure is modifiedfrom Demes, B., Larson, S. G., Stern, J.T., Jr., Jungers, W. L., Biknevicius, A. R., et al. (1994). The kinetics ofprimate quadrupedalism: Hindlimb drive reconsidered. Journal of Human Evolution, 26, 353–374, toinclude data on marsupials (Schmitt, D., & Lemelin, P. [2002]. The origins of primate locomotion: gaitmechanics of the woolly opossum. American Journal of Physical Anthropology, 118, 231–238; Lemelin, P., &Schmitt, D. (2007). The orgins of grasping and locomotor adaptations in primates: Comparative andexperimental approaches using an opossum model. In: M. Dagosto & M. Ravosa (Eds.), Primate origins(pp. 329–380). New York: Kluwer.) and the common marmoset (Schmitt, D. [2003b]. The relationshipbetween forelimb anatomy and mediolateral forces in primate quadrupeds: Implications for interpretationof locomotor behavior in extinct primates. Journal of Human Evolution 44, 49–60).agreed that ‘‘Reynolds’ mechanism’’ could also adjust their limb stiffness in order to moderateplay a substantial role in shifting weight posteriorly loads on arboreal supports has become an increas-but in addition, changes in limb stiffness could also ingly frequent area of research and debate inplay a role in reducing forelimb peak loads. This anthropology and has important and broad impli-notion that arboreal mammals such as primates cations for neural control of locomotion (Franz

PRIMATE LOCOMOTOR EVOLUTION 49et al., 2005; Larney & Larson, 2004; Li et al., 1996; limbs (Blickhan, 1989; Li et al., 1996; McMahon etSchmitt, 1995, 1998, 1999, 2003a,b; Schmitt & al., 1987; Schmitt, 1995, 1998, 1999). Schmitt andHanna, 2004; Schmitt & Lemelin, 2002; Schmitt Hanna (2004) found a similar but less pronouncedet al., 2007, 2008; Wallace & Demes, 2008). This is pattern for the hindlimb of primates.not a new concept for biology in general. Twenty Recently, Larney and Larson (2004) providedyears before physical anthropologists incorporated an extensive dataset on forelimb and hindlimbthis idea, limb stiffness was recognized as a critical joint yield for a wide variety of mammals. Theyvariable for understanding the relationship confirmed the high values of forelimb compli-between limb design and locomotor behavior in ance in most primates, but they also reporteda wide variety of animals (Ahn et al., 2004; that marsupials have forelimb compliance asAlexander, 1992; Cavagna et al., 1977; Farley et great, if not greater, than that of most primates.al., 1993; Full & Tu, 1990; Griffin et al., 2004; Schmitt and colleagues (n.d.) recently confirmedMcMahon, 1985; McMahon et al., 1987). that woolly opossums, like primates, have high Under experimental conditions, Schmitt values of elbow yield, while the short-tailed(1998, 1999) showed that Old World monkeys opossum does not.reduce forelimb stiffness by increasing forelimb‘‘compliance’’—a term borrowed from general Lack of a Running Trot and the Use of ananimal biomechanics (Alexander, 1992; AmbleMcMahon, 1985; McMahon et al., 1987)—whenwalking on relatively thin poles compared to a As most quadrupedal mammals increase speed,runway. In these studies, forelimb compliance they shift from a walking gait with no aerialwas measured in terms of joint yield. By increasing phase (Fig. 3.10a) to a running gait with ajoint yield, it was argued that the forelimb operates whole body aerial phase (Fig. 3.10b) (Cartmillrelatively more like a spring, which increases con- et al., 2002; Gambaryan, 1974; Hildebrand, 1985;tact time and reduces peak vertical forces on the Howell, 1944; Muybridge, 1957). At their fastestFigure 3.10 A vervet monkey using two common gaits. At slow speeds most quadrupedal primates adopta walking gait (A) in which a forelimb contact is followed by a hindlimb contact and there is always at leasttwo limbs on the substrate at one time with no whole body aerial phase. At faster speeds most quadrupedalprimates adopt a gallop in which two hindlimb contacts are followed by two forelimb contacts and there is awhole body aerial phase. Many primates use a canter, which is a slow gallop with no aerial phase. Modifiedfrom Schmitt, D., Larson, S. G., & Stern, J. T., Jr. (1994) Serratus ventralis function in vervet monkeys: Areprimate quadrupeds unique? Journal of Zoology, 232, 215–230. Used with permission.

50 PRIMATE NEUROETHOLOGYspeeds, quadrupedal mammals generally use a Schmitt and colleagues (2006) argued thatrunning gait such as a gallop (Gambaryan, 1974; ambling ensures continuous contact of theHildebrand, 1985; Howell, 1944; Muybridge, body with the substrate while dramatically1957). But at speeds between that of a walk and reducing vertical oscillations of the center ofa gallop, quadrupedal mammals often use sym- mass. This may explain why ambling appearsmetrical running gaits that have an aerial phase to be preferable to trotting for extremely largeand in which the feet strike down in diagonal terrestrial mammals such as elephants and forpairs (trot) or unilateral pairs (pace) (Cartmill arboreal mammals like primates that move onet al., 2002; Gambaryan, 1974; Hildebrand, 1985; unstable branches.Howell, 1944; Muybridge, 1887). These gaits arefaster than walking gaits but still provide rela- Reduced Oscillations of the Center of Masstively longer periods of support by both a fore-limb and a hindlimb than does galloping Both the changes in spring stiffness (#4) and(Cartmill et al., 2002). the use of ambles rather than trots (#5) appear Unlike other mammals, primates almost never to have implications for the vertical oscilla-adopt a running trot or pace (Demes et al., 1994, tions of primates as they walk. The movements1994; Hildebrand, 1967; Preuschoft and Gunther, of the animal’s body, reflected in the move-1994; Rollinson & Martin, 1981; Schmitt, 1995; ments of the animal’s center of mass (COM),Schmitt et al., 2006; Vilensky, 1989). Instead, have implications for loading, stability, andSchmitt and colleagues (2006) showed that pri- energy exchange. The latter represents anmates adopt a highly unusual running gait called important target of selection that helps explainan ‘‘amble’’ (Fig. 3.11). These gaits are referred to the postural and gait choices made byas ‘‘grounded running gaits’’ (Rubenson et al., primates.2004) because they do not involve a whole body Schmitt (2003) argued that forelimb compli-aerial phase. Ambles are exhibited by almost all ance moderates vertical oscillations of the bodyprimates (Schmitt et al., 2006) as well as certain and peak vertical forces on the limbs. He and hisbreeds of horses (Barrey, 2003; Biknevicus et al., colleagues (Schmitt et al., 2006) argued that2004; Muybridge, 1887) and elephants moderating vertical oscillations of the center of(Gambaryan, 1974; Hutchinson et al., 2003). mass are a critical control target during primateFigure 3.11 A primate adopting an amble, a gait in which a forelimb contact is followed by a hindlimbcontact and there is always at least two limbs on the substrate. Unlike a walk, however, there are alternatingaerial phases for the hindlimb (A) and forelimb (B) pairs. The amble is a rare gait adopted only by somebreeds of horses, elephants, kinkajous, and primates. This gait reduces vertical oscillations of the center ofmass and increases stability compared to a running trot. Schmitt, D., Cartmill, M., Hanna, J., Griffin, T., andLemelin, P. (2006). The adaptive value of ambling in primates. Journal of Experimental Biology 209,2042–2049. Used with permission.

PRIMATE LOCOMOTOR EVOLUTION 51locomotion at both normal and fast walking 1977; Farley & Ko, 1997; Farley et al., 1993;gaits and at slow running gaits. However, the Full, 1991; Full & Tu, 1990; Griffin, 2000,movements of the COM discussed in these stu- 2002; Manter, 1938). The data derived fromdies were assumed and had not been quantified. these studies allow researchers to exploreThey were simply inferred from the limb stiff- underlying mechanics of various mammalianness itself, which in turn was inferred from joint gaits and allow for comparison across a wideyield. variety of taxa. They have revealed that the The movements of the COM have important walking and running gaits of most animals,implications for understanding the targets of regardless of phylogeny and morphologicallocomotor control. Any animal must balance design, operate with the same basic mechanicalneeds of stability, speed, and energetic efficiency. principles and that gaits may be defined byThe control of COM influences all three factors those principles.to varying degrees. The ways in which the COM To understand the implications of thisoscillates in nonprimate bipeds and quadrupeds finding, it is worth reviewing gait definitions.has been a critical variable both in defining gaits Classifications of gait and analyses of gaitas speed increases and in understanding the choice have traditionally been based on visualadaptive tradeoffs between stability and effi- rather than biomechanical criteria. This isciency of any gait (see Ahn et al., 2004; useful and appropriate but also omits impor-Biewener, 2003; Cavagna et al., 1977; Griffin tant information about the mechanicalet al., 2004). It appears that most animals costs and benefits of specific gaits and thefollow the same the basic underlying governing underlying principles that guide gait selectionprinciples for walking and running. Nonhuman in animals.primates, however, may represent an exception Under visual schemes of gait classification,to this broadly conserved pattern (Schmitt, quadrupedal walking, trotting, and running2003). gaits have been defined in the following way. Direct measures of COM movements in pri- A walk (Fig. 3.10a) is a symmetrical gait inmates are very limited. Wells and Wood (1975) which hindlimb footfalls alternate with fore-described the movements of the COM during limb footfalls and there are at least two feet onleaping in vervet monkeys. Using videorecords, the ground at all times. Thus, in a walk theVilensky (1979) provided data on the COM in duty factor (contact time of any limb dividedmacaques at a wide range of speeds on a tread- by total stride time) is at least 0.5. Gaits with amill. His data suggested that changes in limb duty factor of less than 0.5 have an aerialkinematics minimized the movements of the phase.COM at different speeds. Few studies, with the Some gaits, like an amble (Fig. 3.11), areexception of Cavagna and colleagues (1977) and symmetrical gaits in which there is no wholeKimura (1990, 1991, 1996), have used force plat- body aerial phase but there is an aerial phase inform data to infer the behavior of the center of either the forelimbs or the hindlimbs (Barrey,mass for either whole animal or individual limb 2001; Gambarayan, 1974; Howell, 1944;girdles. Hutchinson et al., 2003; Schmitt et al., 2006). In contrast to primate studies, the analysis of Thus, the body is always supported by at leastCOM movements is a common method of ana- one limb.lysis in biomechanical studies of other animals A running trot is a symmetrical gait in whichincluding cockroaches, crabs, frogs, lizards, diagonal limb pairs (right hindlimb and leftostriches, penguins, sheep, horses, dogs, and forelimb) swing and contact the ground simul-humans (Ahn et al., 2004; Alexander, taneously and there is a whole body aerial2003; Biewener, 2003; Bishop et al., n.d.; phase. Finally, galloping is an asymmetricalBlickhan & Full, 1992; Cavagna et al., 1976, gait with a whole body aerial phase

52 PRIMATE NEUROETHOLOGY a b c A m ∇ B ∇ v C) Gravitational h m m Potential Energy v ∇ = mg h b ∇ c Horizontal a Kinetic Energy ∇ = (1/2 mv2) a b c TimeFigure 3.12 The inverted pendulum model of walking. The limb is modeled as a massless strut andwith the center of mass (red circle) concentrated at one end. In both (A) and (B) the center of mass(COM) is at its low point at the beginning of the step (a). At the midpoint of the step (b) the center ofmass is at its highest point with a large amount of potential energy. The change in height (Dh) isindicated in (B). In the second half of the step (c), as the body moves forward the center of mass dropsand potential energy (PE) is converted into kinetic energy (KE). If fluctuations in PE and KE are out ofphase and of the same amplitude (C), then there is a highly effective conversion of PE into KE. Thisenergy exchange (recovery) reduces the work that needs to be done by muscles to accelerate anddecelerate the center of mass. A high value of exchange of PE and KE (between 50% and 75%) isfound in the walking gaits of a diverse array of quadrupeds and bipeds and is seen as a potentiallyimportant target of selection.(Fig. 3.10b). During galloping, forelimb and however, to define walking (duty factor >0.5)hindlimb contacts do not alternate but rather and running gaits (duty factor <0.5) in mechan-contacts of the hindlimbs are followed by con- ical terms. Walking and running gaits show dis-tacts of the forelimb. crete differences in the behavior of the COM and These definitions work very well for visual in the exchange of potential and kinetic energy.distinction, but in reality gaits defined in this This is most easily explained using a bipedfashion are part of a continuum (Cartmill et model but defines the gaits of quadrupeds asal., 2002). Using the definitions given previously, well (Fig. 3.12).gaits with a duty factor (contact time of a limb When modern humans walk, we vault overdivided by total stride time) of 0.51 compared to relatively stiff lower limbs in such a way that0.53 are both walks. However, if the duty factors our center of mass is at its lowest point atare 0.51 and 0.49, one animal is running while heel-strike and rises to its highest point atthe other is walking. It has been argued that gaits midstance (Fig. 3.12a,b) (Cavagna et al.,should not be defined in this way, but rather 1976; Lee & Farley, 1998). This type ofdefined by discrete differences. It is possible, inverted pendulum gait is common to almost

PRIMATE LOCOMOTOR EVOLUTION 53all known quadrupeds and bipeds (Ahn et al., 1977). During running, in contrast, KE and2004; Alexander, 1977, 2003; Biewener, 2003; PE are in phase and cannot exchange energy.Blickhan & Full, 1992; Cavagna et al., 1976, During running much of this energy is con-1977; Farley & Ko, 1997; Farley et al., 1993; verted to elastic energy in spring elements ofFull, 1991; Full & Tu, 1990; Gatesy & the lower limb.Biewener, 1991; Griffin & Kram, 2000; This exchange mechanism works when fluc-Heglund et al., 1982). In contrast, when tuations in PE and KE are equal in magnitude.humans and other animals run, their limbs Excessively large amounts of PE due to highoperate more like they were part of a spring- vertical oscillations would be wasted. Similarly,mass system in which the limb is compressed excessively low amounts of PE due to a smooth,during support phase, and the center of mass nonoscillating gait would be insufficient for gen-is at its lowest point during the middle of eration of enough KE. These constraints maystance. explain the relatively stiff-legged oscillating Center of mass movements can be directly gaits that are found in many legged mammalscalculated from force curves of all three com- and even explain the waddle of penguins (Griffinponents of the ground reaction forces exerted et al., 2000).by an animal. Forces reflect accelerations of the Primates, however, are different. In the casebody’s center of mass. Those accelerations can of ambling and compliant walking, previousbe easily converted to velocities and displace- studies have suggested that primate gaits arements from which both kinetic energy and characterized by low oscillations of the COM.potential energy can be calculated. The mathe- Thus, it has been assumed that primates havematics for calculating these movements are given up this energy-saving mechanism indetailed in textbooks (Biewener, 2003) and exchange for security while running onrecent publications (Ahn et al., 2004). branches. This notion received support from The relationship between the potential recent findings (Bishop et al., 2008) that catsenergy (PE) and kinetic energy (KE) of the show low values of recovery when executingCOM provides information about the relative stealthy, crouched gaits. It is expected that pri-efficiency of different gaits. If the fluctuations mates would show the same pattern. However,of the PE and KE are mirror images of each recent preliminary analysis of lemurs has shownother (have the same amplitude, shape, and that the story is more complex (Schmitt et al.,opposite direction) (Fig. 3.12C), then PE 2008). Although it is true that lemurs show lowerstored at midstance can be converted to KE values of recovery than dogs (highest averageduring the second half of stance. The recovered value = 75%; mean recovery value = 60; GriffinPE converted into KE can be used to drive the et al. (2004). and other quadrupeds (Cavagna etCOM forward. If the leg is held stiff, as most al., 1977), the values are not always as low asanimals do, then the COM will also be driven would have been expected. The ring-tailedupward and regain PE than can be converted lemur, a highly terrestrial strepsirrhine, showsinto KE. This mechanism of energy recovery values of recovery near that of dogs with anreduces the amount of muscular work required average recovery of 49% and maximum valuesto accelerate and decelerate the COM and thus of 70%. Schmitt and colleagues (2008) arguedmay reduce the costs of locomotion. This that by combining long limbs and deep compli-amount of exchange is referred to as the per- ance, these lemurs achieved relatively acceptablecentage of recovery. This is a minimum esti- levels of energy exchange. Had they not beenmate of the energy used by the system. In compliant, then their long limbs would havewalking, PE and KE are equal in amplitude caused a large amount of change in height ofand shape but are largely out of phase. As a the COM (Dh in Fig. 3.12b) and PE would haveresult, as much as 70% of the energy needed to been wasted. In contrast, the brown lemur, amove forward can be conserved through an much more arboreal primate compared to theexchange of PE and KE (Cavagna et al., ring-tailed lemur, showed excessively low values

54 PRIMATE NEUROETHOLOGYof energy recovery with an average value of 36% These animals adopted highly protracted limbsand no value greater than 50% recovery. These and deep elbow flexion to effect long strides andlatter data suggest that much of the serious dif- smooth out their gait. This was also associatedferences between primate locomotion and that with a reduction in forelimb compressiveof other animals are specifically related to weight bearing that facilitated mobility of therequirements of locomotion in an arboreal forelimb. This posterior weight shift freed theenvironment. limbs to be used in tension (Stern, 1976; Stern & Oxnard, 1973) and facilitated the evolution of suspensory locomotion.PATHWAY OF PRIMATE As presented, it would follow that our pre-LOCOMOTOR EVOLUTION bipedal ancestors simply descended to the ground and, like gibbons, walked bipedallyIt is overly simplistic to try to draw a straight line because their limbs were too gracile to supportfrom the origin of primates to the origin of weight. This is a logical but untestable conclu-bipedalism by connecting the dots with each sion. But one question we can address isnew functional change in the forelimb. The whether the bipedal locomotion of our ances-reality is that there have been many detours tors was compliant like all nonhuman primatesand dead ends along the way. Nonetheless, or relatively stiff like our own. Since nonhumanwhen these data on locomotion are takentogether, an intriguing picture emerges that primates typically utilize compliant gaits whenmay explain why our mode of locomotion they walk either quadrupedally or bipedally, itevolved in our lineage only. The model I am seems plausible, then, that early bipedal homi-sketching here is one in which adaptations to nins would have retained a compliant walkinglife in an arboreal environment, specifically on style typical of other nonhuman primates.thin, flexible branches, drove fundamental Postcranial anatomy of early hominins suggestschanges in the functional role of the forelimb that some of them walked with a deeply yieldingand may have involved significant changes in knee and hip (Stern & Susman, 1983). Butneural control. These changes allowed the beyond being simply a primitive retention,exploration of new, previously unavailable compliant walking in prehominins may havelocomotor niches. Although we now have had several advantages. Among quadrupedalmore details and more data, this scenario is nonhuman primates, low peak forces andnot new. It was first articulated by Jones reduced stride frequencies make their locomo-(1916) and tested and elaborated on by later tion relatively smooth, which helps them avoidresearchers (Clark, 1959; Jenkins, 1972; shaking flexible branches, thus enhancing theirLarson, 1998; Lemelin, 1999; Lemelin and stability and helping them escape the notice ofSchmitt, 2007; Schmitt, 1998, 1999, 2003a,c,d; predators (Demes et al., 1990; Schmitt, 1998,Schmitt & Lemelin, 2002; Stern, 1976). In this 1999). These features may have also allowedscenario the earliest primates in the Paleocene early human ancestors to maintain mobile,60 to 65 million years ago would have moved loosely stabilized forelimb joints. Kinematic,and foraged frequently on thin, flexible, term- force plate, and accelerometer studies oninal branches. Their locomotion would have human compliant bipedalism show thatbeen characterized by changes in the relation- humans who adopted a complaint gait achievedship between the forelimb and the hindlimb in longer stride lengths, faster maximum walkingorder to promote smooth and secure locomo- speeds, lower peak vertical forces, andtion. These early primates, unlike their nonpri- improved impact shock attenuation betweenmate ancestors, would have relied on diagonal shank and sacrum compared to normal walkingsequence footfall patterns in order to ensure the (Schmitt et al., 1996, 1999, 2003). These dataplacement of the forelimb on a tested support. are consistent with findings of several other

PRIMATE LOCOMOTOR EVOLUTION 55studies (Li et al., 1996; Yaguramaki, 1995) and bonobos (Pan paniscus). American Journal ofsupport claims by Stern and Susman (1983) Physical Anthropology, 111, 503–517.that compliant bipedalism may have been an Ahn, A. N., Furrow, E., & Biewener, A. A. (2004).effective gait for a small biped with relatively Walking and running in the red-legged running frog, Kasina maculata. Journal of Experimentalsmall and weakly stabilized joints, which had Biology, 207, 399–410.not yet completely forsaken arboreal Alexander, R. McN. (1977). Mechanics and scalinglocomotion. of terrestrial locomotion. In: T. Pedley (Ed.), In 1998, Susan Larson suggested, based on Scale effects in animal locomotion (pp. 93–110).electromyographic data (Larson, 1989; Larson London: Academic Press.& Stern, 1987, 1989, 1992; Schmitt et al., 1994), Alexander, R. McN. (1992). A model of bipedalthat primates were fundamentally different locomotion on compliant legs. Philosophicalfrom all other mammals. She argued that Translations of the Royal Society of Londonwhile motor patterns were conserved across Series B, 338, 189–198.broad orders of vertebrates, primates had Alexander, R. M., & Maloiy, G. M. (1984). Stridedeviated from this pattern and had adopted lengths and stride frequencies of primates. Journal of Zoology of London, 202, 577–582.different patterns of muscle recruitment Anapol, F. C., & Jungers, W. L. (1987).(Larson, 1998). This was a profound conclusion Telemetered electromyography of the fast andthat suggested that basal primates had not just slow extensors of the leg of the brown lemurslightly modified their locomotor behavior, but (Lemur fulvus). Journal of Experimentalhad instead adopted a derived locomotor pat- Biology, 130, 341–358.tern in association with arboreal movement. Barrey, E. (2001). Inter-limb coordination. In: W.This conclusion was supported by data on foot- Back & H. M. Clayton (Eds.), Equine locomotionfall patterns (Cartmill et al., 2002), forelimb (pp. 77–94). London: W.B. Saunders.kinematics (Larson et al., 1999, 2001), and Bertram, J. E. A., & Chang, Y. (2001) Mechanicalground reaction forces (Demes et al., 1994; energy oscillations of two brachiation gaits:Kimura et al., 1979; Reynolds, 1985; Schmitt Measurement and simulation. American Journal of Physical Anthropology, 115, 319–326.2003a; Schmitt & Lemelin, 2002). Now it Blickhan, R. (1989). The spring mass model forappears that even the fundamental underlying running and hopping. Journal of Biomechanics,mechanisms that characterize walking gaits of 22, 1217–1227.most animals are not conserved by primates. Byron & Covert. (2003).This lack of conservation of basic locomotor Carlson, K. J., Demes, B. and Franz, T. M. (2005).behavior and control is not a trivial mathema- Mediolateral forces associated with quadrupedaltical distinction, nor is it simply a difference of gaits of lemurids. J. Zool. 266, 261–273.degree. Instead, the way in which primates dif- Cartmill, M. (1970). The orbits of arborealferent in their locomotor behavior from most mammals: A reassessment of the arboreal theoryother mammals represents an abrupt beha- of primate evolution. Ph.D. Thesis, Chicago,vioral shift that suggests major ecological, ana- University of Chicago. Cartmill, M. (1972). Arboreal adaptations and thetomical, and neurological changes in our origin of the order primates. In: R. H. Tuttleearliest ancestors that set the stage for own (Ed.), The functional and evolutionary biology ofevolution and present condition. primates (pp. 3–35). Chicago: Aldine-Atheton. Cartmill, M. (1974a). Pads and claws in arboreal locomotion. In: F. A. Jenkins Jr. (Ed.), PrimateREFERENCES locomotion (pp. 45–83). New York: Academic Press.Aerts, P., Van Damme, R., Van Elsacker, L., & Cartmill, M. (1974b). Rethinking primate origins. Duchene V. (2000). Spatiotemporal gait Science, 184, 436–443. characteristics of the hind-limb cycles during Cartmill, M. (1992). New views on primate origins. voluntary bipedal and quadrupedal walking in Evolution and Anthropology, 1, 105–111.

56 PRIMATE NEUROETHOLOGYCartmill, M., Lemelin, P., & Schmitt, D. (2002). Demes, B., Jungers, W. L., & Nieschalk, U. (1990). Support polygons and symmetrical gaits in Size- and speed-related aspects of quadrupedal mammals. Zoological Journal of the Linnean walking in slender and slow lorises. In: F. K. Society, 136, 401–420. Jouffroy, M. H. Stack, & C. Niemitz (Eds.),Cartmill, M., Lemelin, P., & Schmitt, D. (2007a). Gravity, posture and locomotion in primates Primate gaits and primate origins. In: M. (pp. 175–198). Florence: Il Sedicesimo. Dagosto & M. Ravosa (Eds.), Primate origins Demes, B., Larson, S. G., Stern, J. T. Jr., Jungers, W. (pp. 403–436). New York: Kluwer. L., Biknevicius, A. R., & Schmitt, D. (1994). TheCartmill, M., Lemelin, P., & Schmitt, D. (2007b). kinetics of primate quadrupedalism: hindlimb Understanding the adaptive value of diagonal- drive reconsidered. Journal of Human. sequence gaits in primates: A comment on Evolution, 26, 353–374. Shapiro and Raichlen (2005). American Demes, A. B., Qin, Y., Stern, J. T. Jr., Larson, S. G., Journal of Physical Anthropology, 133, & Rubin, C. T. (2001). Patterns of strain in the 822–825. macaque tibia during functional activity.Cavagna, G. A., Heglund, N. C., & Taylor, C. R. American Journal of Physical Anthropology, 116, (1977). Mechanical work in terrestrial 257–265. locomotion: Two basic mechanisms for Demes, A. B., Stern, J. T. Jr., Hausman, M. R., minimizing energy expenditure. American Larson, S. G., McLeod, K. J., & Rubin, C. T. Journal of Physiology, 233, R243–261. (1998). Patterns of strain in the macaque ulnaCavagna, G. A., Thys, H., & Zamboni, A. (1976). during functional activity. American Journal of Sources of external work in level walking and Physical Anthropology, 106, 87–100. running. Journal of Physiology of London, 262, Elftman, H. (1944). The bipedal walking of the 639–657. chimpanzee. Journal of Mammals, 25, 67–71.Chang Y. H., Bertram J. E., & Ruina A. (1997). Elftman, H., & Manter, J. (1935). Chimpanzee and A dynamic force and moment analysis system human feet in bipedal walking. American for brachiation. Journal of Experimental Biology, Journal of Physical Anthropology, 20, 69–79. 200, 3013–3020. Farley, C. T., Glasheen, J., & McMahon, T. A.Chang, Y., Bertram, J. E. A., & Lee, D. V. (2000). (1993). Running springs: Speed and animal External forces and torques generated by the size. Journal of Experimental Biology, 185, brachiating white-handed gibbon (Hylobates 71–86. lar). American Journal of Physical Anthropology, Fleagle, J. G. (1977a). Locomotor behavior and 113, 201–216. skeletal anatomy of sympatric Malaysian leaf-Churchill S. E. and Schmitt D. (2003) Biomechanics monkeys (Presbytis obscura and Presbytis in paleoanthropology: engineering and melalophos). Yearbook of Physical Anthropology, experimental approaches to the investigation of 20, 440–453. behavioral evolution in the genus Homo. In C Fleagle, J. G. (1977b). Locomotor behavior and Harcourt and R Crompton (eds.): New muscular anatomy of sympatric Malaysian Perspectives in Primate Evolution and leaf-monkeys (Presbytis obscura and Presbytis Behavior. London: Linnaean Society, pp. melalophos). American Journal of Physical 59–90. Anthropology, 46, 297–308.Crompton, R. H., Li, Y., Wang, W., Gunther, Fleagle, J. G. (1979). Primate positional behavior M. M., & Savage, R. (1998). The mechanical and anatomy: Naturalistic and experimental effectiveness of erect and bent-hip, bent-knee approaches. In: M. E. Morbeck, H. Preuschoft, bipedal walking in Australopithecus & N. Gomberg (Eds.), Environment, behavior, afarensis. Journal of Human Evolution, 35, and morphology: Dynamic interactions in 55–74. primates (pp. 313–326). New York: Wenner- ˆD’Aout, K. D., Aerts, P., De Clercq, D., De Meester, Gren Foundation. K., & Van Elsacker, L. (2002). Segment and joint Fleagle, J. G. (1999). Primate adaptation and angles of hind limb during bipedal and evolution. New York: Academic. quadrupedal walking of the bonobo (Pan Fleagle, J. G., Stern, J. T., Jr., Jungers, W. L., paniscus). American Journal of Physical Susman, R. L., Vangor, A. K., & Wells, J. P. Anthropology, 119, 37–51 (1981). Climbing: A biomechanical link

PRIMATE LOCOMOTOR EVOLUTION 57 with brachiation and with bipedalism. the spider monkey and the Japanese macaque. Symposium of the Zoological Society of American Journal of Physical Anthropology, 113, London, 48, 359–375. 455–472.Franz, T. M., Demes, B., & Carlson, K. J. (2005). Howell, A. B. (1944). Speed in animals. Chicago: Gait mechanics of lemurid primates on University of Chicago Press. terrestrial and arboreal substrates. Journal of Hutchinson, J. R., Famini, D., Lair, R. and Kram, R. Human Evolution, 48, 199–217. (2003). Are fastmoving elephants reallyFull, R. J., & Tu, M. S. (1990). Mechanics of six- running? Nature 422, 493–494. legged runners. Journal of Experimental Biology, Ishida, H., Jouffroy, F. K., & Nakano, Y. (1990). 148, 129–146. Comparative dynamics of pronograde andGambaryan, P. P. (1974). How mammals run. New upside down horizontal quadrupedalism in York: Wiley. the slow loris (Nycticebus coucang). In: F. K.Gatesy, S. M., & Biewener, A. A. (1991). Bipedal Jouffroy, M. H. Stack, & C. Niemitz (Eds.), locomotion: Effects of speed, size, and limb Gravity, posture and locomotion in primates posture in birds and humans. Journal of (pp. 209–220). Florence: Il Sedicesimo. Zoology of London, 224, 127–147. Ishida, H., Kumakura, H., & Kondo, S. (1985).Griffin, T. M., & Kram, R. (2000) Penguin Primate bipedalism and quadrupedalism: waddling is not wasteful. Nature, 408, 929. Comparative electromyography. In: S. Kondo,Griffin, T. M., Main, R. P., & Farley, C. T. (2004). H. Ishida, & T. Kimura (Eds.), Primate Biomechanics of quadrupedal walking: How do morphophysiology, Locomotor analyses and four-legged animals achieve inverted human bipedalism (pp. 59–80). Tokyo: pendulum-like movements? Journal of University of Tokyo Press. Experimental Biology, 207, 3545–3558. Jenkins, F. A., Jr. (1972). ChimpanzeeGunther, M. M. (1991). The jump as a fast mode of bipedalism: Cineradiographic analysis and locomotion in arboreal and terrestrial biotopes. implications for the evolution of gait. ¨ Zeitschrift fur Morphologie und Anthropologie, Science, 178, 877–879. 78, 341–372. Jenkins, F. A., Jr. (1974). Tree shrew locomotionHeglund, N. C., Cavagna, G. A., & Taylor, C. R. and the origins of primate arborealism. In: (1982). Energetics and mechanics of terrestrial F. A. Jenkins, Jr. (Ed.), Primate locomotion locomotion. III. Energy changes of the center of (pp. 85–115). New York: Academic Press. mass as a function of speed and body size in Jenkins, F. A., Jr., Dombrowski, P. J., & Gordon, birds and mammals. Journal of Experimental E. P. (1978). Analysis of the shoulder in Biology, 97, 41–56. brachiating spider monkeys. American JournalHildebrand, M. (1967). Symmetrical gaits of of Physical Anthropology, 48, 65–76. primates. American Journal of Physical Jones, F. W. (1916). Arboreal man. London: Anthropology, 26, 119–130. Edward Arnold.Hildebrand, M. (1985). Walking and running. In: Jouffroy, F. K. (1983). Etude cineradiographique M. Hildebrand, D. M. Bramble, K. F. Liem, & des deplacements du membre anterieur du D. B. Wake (Eds.), Functional vertebrate Potto de Bosman (Perodicticus potto, P.L.S. morphology (pp. 38–57). Cambridge: Harvard Muller, 1766) au cours de la marche University Press. quadrupede sur une branche horizontale.Hirasaki, E., Kumakura, H., & Matano, S. (1993). Annales des Sciences Naturelles, Zoologie, Paris, Kinesiological characteristics of vertical 5, 75–87. climbing in Ateles geoffroyi and Macaca Jouffroy, F. K. (1989). Quantitative and fuscata. Folia Primatologica, 61, 148–156. experimental approaches to primateHirasaki, E., Kumakura, H., & Matano, S. (1995). locomotion: A review of recent advances. In: Electromyography of 15 limb muscles in P. Seth & S. Seth (Eds.), Perspective in primate Japanese macaques (Macaca fuscata) during biology. (pp. 47–108). New Delhi: Today and vertical climbing. Folia Primatologica, 64, Tomorrow’s Printers and Publishers. 218–224. Jouffroy, F. K., & Gasc, J. P. (1974). A cine-Hirasaki, E., Kumakura, H., & Matano, S. (2000). radiographic analysis of leaping in an African Biomechanical analysis of vertical climbing in prosimian (Galago alleni). In: F. A. Jenkins

58 PRIMATE NEUROETHOLOGY (Ed.), Primate locomotion (pp. 117–142). New Kimura, T. (1991). Body center of gravity and York: Academic Press. energy expenditure during bipedal locomotionJouffroy, F. K., Gasc, J. P., Decombas, M., & Oblin, in humans, chimpanzees and macaques. S. (1974). Biomechanics of vertical leaping from Primate Reports, 31, 19–20. the ground in Galago alleni: a cineradiographic Kimura, T. (1992). Hindlimb dominance during analysis. In: R. D. Martin, A. G. Doyle, & A. C. primate high-speed locomotion. Primates, 33, Walker (Eds.), Prosimian biology. Liverpool: 465–474. Duckworth and Co. Ltd. Kimura, T. (1996). Center of gravity of the bodyJouffroy, F. K., & Petter, A. (1990). Gravity-related during the ontogeny of chimpanzee bipedal kinematic changes in lorisine horizontal walking. Folia Primatologica, 66, 126–136. locomotion in relation to position of the body. Kimura, T., Okada, M., & Ishida, H. (1979). In: F. K. Jouffroy, M. H. Stack, & C. Niemitz Kinesiological characteristics of primate (Eds.), Gravity, posture and locomotion in walking: Its significance in human walking. primates (pp. 199–207). Florence: Il Sedicesimo. In: M. Morbeck, H. Preuschoft, & N.Jouffroy, F. K., & Stern, J. T., Jr. (1990). Gomberg (Eds.), Environment, behavior, and Telemetered EMG study of the antigravity morphology: Dynamic interactions in primates versus propulsive actions of knee and elbow (pp. 297–312). New York: Gustav Fischer. muscles in the Slow loris (Nycticebus Kimura, T, Okada, M., Yamazaki, N., & Ishida, H. coucang). In: F. K. Jouffroy, M. H. Stack, & C. (1983). Speed of the bipedal gaits of man and Niemitz (Eds.), Gravity, posture and locomotion nonhuman primates. Annals of Scientific in primates (pp. 221–236). Florence: Il National Zoology of Paris, 5, 145–158. Sedicesimo. Larney, E., & Larson, S. G. (2004). LimbJungers, W., & Anapol, F. (1985). Interlimb compliance during walking: Comparisons of coordination and gait in the Brown Lemur elbow and knee yield across quadrupedal (Lemur fulvus) and the Talapoin Monkey primates and in other mammals. American (Miopithecus talapoin). American Journal of Journal of Physical Anthropology, 125, 42–50. Physical Anthropology, 67, 89–97. Larson, S. G. (1988). Subscapularis function inJungers, W., & Stern, J. (1980). Telemetered gibbons and chimpanzees: Implications for electromyography of forelimb muscle chains in interpretation of humeral head torsion in gibbons. Science, 208, 617–619. hominoids. American Journal of PhysicalJungers, W., & Stern, J. (1981). Preliminary Anthropology, 76, 449–462. electromyography of brachiation in gibbons Larson, S. G. (1989). Role of supraspinatus in and spider monkeys. International Journal of quadrupedal locomotion of vervets Primatology, 2, 19–30. (Cercopithecus aethiops): Implications forJungers, W. L., & Stern, J. T. (1984). interpretation of humeral morphology. Kinesiological aspects of brachiation in lar American Journal of Physical Anthropology, 79, gibbons. In: H. Preuschoft, D. J. Chivers, W. 369–377. Y. Brockelman, & N. Creel (Eds.), The lesser Larson, S. G. (1998). Unique aspects of apes: Evolutionary and behavioural biology (pp. quadrupedal locomotion in nonhuman 119–134). Edinburgh: Edinburgh University primates. In: E. Strasser, J. G. Fleagle, A. L. Press. Rosenberger, & H. M. McHenry (Eds.),Kimura, T. (1985). Bipedal and quadrupedal Primate locomotion: Recent advances (pp. 157– walking of primates, comparative dynamics. 173). New York: Plenum. In: S. Kondo, H. Ishida, & T. Kimura (Eds.), Larson, S. G., & Stern, J. T., Jr. (1986). EMG of Primate morphophysiology, Locomotor analyses scapulohumeral muscles in the chimpanzee and human bipedalism (pp. 81–104). Tokyo: during reaching and ‘‘arboreal’’ locomotion. University of Tokyo Press. American Journal of Anatomy, 176, 171–190.Kimura, T. (1990). Voluntary bipedal walking of Larson, S. G., & Stern, J. T., Jr. (1987). EMG of infant chimpanzees. In: F. K. Jouffroy, M. H. chimpanzee shoulder muscles during knuckle- Stack, & C. Niemitz (Eds.), Gravity, posture and walking: problems of terrestrial locomotion in a locomotion in primates (pp. 237–251). Florence: suspensory adapted primates. Journal of Il Sedicesimo. Zoology, London, 212, 629–655.

PRIMATE LOCOMOTOR EVOLUTION 59Larson, S. G., & Stern, J. T. (1989). The role of Lemelin, P., Schmitt, D., MacKenzie, A., George, propulsive muscles of the shoulder during G., & Cartmill, M. (2008). The effects of quadrupedalism in vervet monkeys substrate type and size on the locomotion of (Cercopithecus aethiops): Implications for kinkajous. Integrated and Comprehensive neural control of locomotion in primates. Biology, 47, Suppl. 1, e-71. Journal of Motor Behavior, 21, 457–472. Li, Y., Crompton, R. H., Alexander, R. McN.,Larson, S. G., & Stern, J. T., Jr. (1992). Further Gunther, M. M., & Wang, W. J. (1996). evidence for the role of supraspinatus in Characteristics of ground reaction forces in quadrupedal monkeys. American Journal of normal and chimpanzee-like bipedal Physical Anthropology, 87, 359–363. walking by humans. Folia Primatologica,chimpanzee: Scapular rotators revisited. 66, 137–159. American Journal of Physical Anthropology, McClearn, D. (1992). Locomotion, posture, 85, 71–84. and feeding behavior of kinkajous, coatis,Larson, S. G., Schmitt, D., Lemelin, P., & Hamrick, M. and raccoons. Journal of Mammals, 73, W. (1999). The uniqueness of primate forelimb 245–261. posture during quadrupedal locomotion. McMahon, T. A. (1985). The role of compliance in American Journal of Physical Anthropology, 112, mammalian running. Journal of Experimental 87–101. Biology, 115, 263–282.Larson, S. G., Schmitt, D., Lemelin, P., & Hamrick, McMahon, T. A., Valiant, G., & Frederick, E. C. M. W. (2001). Limb excursion during (1987). Groucho running. Journal of Applied quadrupedal walking: How do primates Physiology, 62, 2326–2337. compare to other mammals. Journal of Zoology Meldrum, D. J. (1991). Kinematics of the of London, 255, 353–365. Cercopithecine foot on arboreal and terrestrialLatimer, B., Ohman, J. C., & Lovejoy, C. O. (1987). substrates with implications for the Talocrural joint in African hominoids: interpretation of hominid terrestrial adaptations. Implications for Australopithecus afarensis. American Journal of Physical Anthropology, 84, American Journal of Physical Anthropology, 74, 273–290. 155–175. Muybridge, E. (1887). Animals in Motion [1957Lee, C. R., & Farley, C. T. (1998). Determinants of reprint]. New York: Dover. the center of mass trajectory in human walking Okada, M. (1985). Primate bipedal walking: and running. Journal of Experimental Biology, Comparative Kinematics. In: S. Kondo, H. 201, 2935–2944. Ishida, & T. Kimura (Eds.), PrimateLemelin, P. (1999). Morphological correlates of morphophysiology, Locomotor analyses and substrate use in didelphid marsupials: human bipedalism (pp. 47–58). Tokyo: implications for primate origins. Journal of University of Tokyo Press. Zoology, London, 247, 165–175. Okada, M., & Kondo, S. (1982). Gait and EMGsLemelin, P., & Schmitt, D. (1998). Relation during bipedal walk of a gibbon (Hylobates between hand morphology and agilis) on flat surface. Journal of the quadrupedalism in primates. American Journal Anthropological Society of Nippon, 90, of Physical Anthropology, 105, 185–197. 325–330.Lemelin, P., & Schmitt, D. (2007). The origins of Polk, J. D. (2001). The influence of body size and grasping and locomotor adaptations in body proportions on primate quadrupedal primates: Comparative and experimental locomotion. PhD. Dissertation, State University approaches using an opossum model. In: M. of New York at Stony Brook. Dagosto & M. Ravosa (Eds.), Primate origins Polk, J. D. (2002). Adaptive and phylogenetic (pp. 329–380). New York: Kluwer. influences on musculoskeletal design inLemelin, P., Schmitt, D., & Cartmill, M. (2003). Cercopithecine primates. Journal of Footfall patterns and interlimb coordination in Experimental Biology, 205, 3399–3412. opossums: Evidence for the evolution of Pridmore, P. A. (1994). Locomotion in Dromiciops diagonal-sequence walking gaits in primates. australis (Marsupialia: Microbiotheriidae). Journal of Zoology, 260, 423–429. Australian Journal of Zoology, 42, 679–699.

60 PRIMATE NEUROETHOLOGYProst, J. H. (1965). The methodology of gait World monkeys. In: E. Strasser, J. G. Fleagle, A. analysis and the gaits of monkeys. American L. Rosenberger, & H. M. McHenry (Eds.), Journal of Physical Anthropology, 23, Primate locomotion: Recent advances (pp. 215–240. 175–200). New York: Plenum Press.Prost, J. H. (1967). Bipedalism of man and Gibbon Schmitt, D. (1999). Compliant walking in compared using estimates of joint motion. primates. Journal of Zoology, 247, 149–160. American Journal of Physical Anthropology, 26, Schmitt, D. (2003a). Evolutionary implications of 135–148. the unusual walking mechanics of the commonProst, J. H. (1969). Gaits of monkeys and horses: marmoset (Callithrix jacchus). American A methodological critique. American Journal of Journal of Physical Anthropology. 122, 28–37 Physical Anthropology, 332, 121–128. Schmitt, D. (2003b). The relationship betweenProst, J. (1980). Origin of bipedalism. American forelimb anatomy and mediolateral forces in Journal of Physical Anthropology, 52, 175–189. primate quadrupeds: Implications forProst, J. H., & Sussman, R. W. (1969). Monkey interpretation of locomotor behavior in locomotion on inclined surfaces. American extinct primates. Journal of Human Evolution. Journal of Physical Anthropology, 31, 53–58. 44, 49–60Raichlen, D., & Shapiro, L. (2007). The evolution of Schmitt, D., & Larson, S. G. (1995). Heel contact as mammalian biomechanics: Adaptations or a function of substrate type and speed in spandrels. Journal of Morphology, 268, 1122. primates. American Journal of PhysicalReynolds, T. R. (1985). Stresses on the limbs of Anthropology, 96, 39–50. quadrupedal primates. American Journal of Schmitt, D., & Lemelin, P. (2002). The origins of Physical Anthropology, 67, 351–362. primate locomotion: gait mechanics of theReynolds, T. R. (1987). Stride length and its woolly opossum. American Journal of Physical determinants in humans, early hominids, Anthropology, 118, 231–238. primates, and mammals. American Journal of Schmitt, D., Lemelin, P., & Trueblood, A. (1999). Physical Anthropology, 72, 101–115. Shock wave transmission through the humanRichmond, B. G., Begun, D. R., & Strait, D. S. body during normal and compliant walking. (2002). Origin of human bipedalism: The American Journal of Physical Anthropology, knuckle-walking hypothesis revisited. Yearbook Suppl. 28, 243–244. of Physical Anthropology, Suppl. 33, 70–105 Schmitt, D., Stern, J. T., Jr., & Larson, S. G. (1996).Rollinson, J., & Martin, R. D. (1981). Comparative Compliant gait in humans: Implications for aspects of primate locomotion with special substrate reaction forces during reference to arboreal cercopithecines. australopithecine bipedalism. American Symposium of the Zoological Society of London, Journal of Physical Anthropology, Suppl. 22, 209. 48, 377–427. Shapiro, L. J., Anapol, F. C., & Jungers, W. L.Rose, M. D. (1973). Quadrupedalism in primates. (1997). Interlimb coordination, gait, and neural Primates 14: 337–358. control of quadrupedalism in chimpanzees.Rowe, N. (1996) The pictorial guide to living American Journal of Physical Anthropology, 102, primates. New York: Pogonias Press. 177–186.Schmidt, M., & Fischer, M. S. (2000). Shapiro, L. J., Demes, A. B., & Cooper, J. (2001). Cineradiographic study of forelimb movements Lateral bending of the lumbar spine during during quadrupedal walking in the brown lemur quadrupedalism in strepsirhines. Journal of (Eulemur fulvus, primates: Lemuridae). Human Evolution, 40, 231–259. American Journal of Physical Anthropology, 111, Shapiro, L. J., & Jungers, W. L. (1988). Back muscle 245–262 function during bipedal walking in chimpanzeeSchmitt, D. (1994). Forelimb mechanics as a function and gibbon: Implications for the evolution of of substrate type during quadrupedalism in human locomotion. American Journal of two anthropoid primates. Journal of Human Physical Anthropology, 77, 201–212. Evolution, 26, 441–458. Shapiro, L. J., & Jungers, W. L. (1994).Schmitt, D. (1998). Forelimb mechanics during Electromyography of back muscles during arboreal and terrestrial quadrupedalism in Old quadrupedal and bipedal walking in primates.

PRIMATE LOCOMOTOR EVOLUTION 61 American Journal of Physical Anthropology, 93, Swartz, S. M., Bertram, J. E., & Biewener, A. A. 491–504. (1989). Telemetered in vivo strain analysis ofSmith, G. E. (1912). The evolution of man. locomotor mechanics of brachiating gibbons. Smithsonian Institution Annual Reports, 1912, Nature, 342, 270–272. 553–572. Szalay, E. S., and E. Delson, E. (1979). EvolutionaryStern, J. T., Jr. (1976). Before bipedality. Yearbook history of the primates. Academic Press, of Physical Anthropology, 20, 59–68. New York.Stern, J. T., Jr. (2000). Climbing to the top: a Szalay, F. S., & Dagosto, M. (1988). Evolution of personal memoir of Australopithecus afarensis. hallucial grasping in the primates. Journal of Evolutionary Anthropology, 9, 113–133. Human Evolution, 17, 1–33.Stern, J. T., Jr., & Larson, S. G. (2001). Telemetered Szalay, F. S., Rosenberger, A. L., & Dagosto, M. electromyography of the supinators and (1987). Diagnosis and differentiation of the pronators of the forearm in gibbons and Order Primates. Yearbook of Physical chimpanzees: Implications for the funda- Anthropology, 30, 75–105. mental positional adaptation of hominoids. Tardieu, C., Aurengo, A., & Tardieu, B. (1993). American Journal of Physical Anthropology, 115, New method of three-dimensional analysis of 253–268. bipedal locomotion for the study ofStern, J. T., Jr., & Susman, R. L. (1981). displacements of the body and body-parts Electromyography of the gluteal muscles in centers of mass in man and non-human Hylobates, Pongo and Pan: Implications for primates: Evolutionary framework. American the evolution of hominid bipedality. Journal of Physical Anthropology, 90, 455–476. American Journal of Physical Anthropology, 55, Taylor, C. R., & Rowntree, V. (1973). Running on 153–166. two or four legs: Which consumes more energy.Stern, J. T., Jr., Wells, J. P., Vangor, A. K., & Science, 179, 597–601. Fleagle, J. G. (1977). Electromyography of Turnquist, J. E., Schmitt, D., Rose, M. D., & Cant, some muscles of the upper limb in Ateles and J. G. (1999). Pendular motion in the brachiation Lagothrix. Yearbook of Physical Anthropology, of captive Lagothrix and Ateles. American 20, 498–507. Journal of Primatology, 48, 263–281.Stern, J. T., Jr., Wells, J. P., Vangor, A. K., & Fleagle, J. Tuttle, R. H., & Basmajian, J. V. (1974a). Electro- G. (1980). An electromyographic study of the myography of Pan gorilla: an experimental pectoralis major in atelines and Hylobates with approach to the problem of hominization. special reference to the evolution pars Symposium of the 5th Congress of the clavicularis. American Journal of Physical International Primate Society, 303–312. Anthropology, 52, 13–25. Tuttle, R., & Basmajian, J. (1974b).Stevens, N. J. (2001). Effects of substrate Electromyography of the brachial muscles in orientation on quadrupedal walking in Loris Pan gorilla and hominoid evolution. American tardigradus. Journal of Morphology, 248, 288. Journal of Physical Anthropology, 41, 71–90.Stevens, N. J., Demes, A. B., & Larson, S. G. Tuttle, R., & Basmajian, J. (1974c). Electro- (2001). Effects of branch compliance on myography of the forelimb musculature in quadrupedal walking in Loris tardigradus. Gorillas and problems related to knuckle American Journal of Physical Anthropology, walking. In: F. Jenkins (Ed.), Primate 113, Suppl. 32, 142. locomotion (pp. 293–383). New York:Susman, R. L. (1983). Evolution of the human foot: Academic. Evidence from Plio-Pleistocene hominids. Foot Tuttle, R. H., & Basmajian, J. V. (1977). and Ankle, 3, 365–376. Electromyography of pongid shoulder musclesSusman, R. J., Stern, J. T., Jr., & Jungers, W. L. and hominoid evolution. I. Retractors of the (1984). Arboreality and bipedality in the humerus and ‘‘rotators’’ of the scapula. Hadar hominids. Folia Primatologica, 43, Yearbook of Physical Anthropology, 20, 491–497. 113–156. Tuttle, R. H., & Basmajian, J. V. (1978a).Sussman, R. W. (1991). Primate origins and the Electromyography of pongid shoulder muscles. evolution of angiosperms. American Journal of III. Quadrupedal positional behavior. American Primatology, 23, 209–223. Journal of Physical Anthropology, 49, 57–70.

62 PRIMATE NEUROETHOLOGYTuttle, R. H., & Basmajian, J. V. (1978b). aethiops). American Journal of Physical Electromyography of pongid shoulder muscles II. Anthropology, 83, 203–210. deltoid, rhomboid and ‘‘rotator cuff’’. American Vilensky, J. A., Gankiewicz, E., & Townsend, D. Journal of Physical Anthropology, 49, 47–56. (1986). Effects of size on Vervet (CercopithecusTuttle, R. H., Hollowed, J. R., & Basmajian, J. V. aethiops) gait parameters: A cross sectional (1992). Electromyography of pronators and approach. American Journal of Physical supinators in Great Apes. American Journal of Anthropology, 76, 463–480. Physical Anthropology, 87, 215–226. Vilensky, J. A., & Gehlsen, G. (1984). Temporal gaitTuttle, R. H., Velte, M. J., & Basmajian, J. V. (1983). parameters in humans and quadrupeds: how do Electromyography of brachial muscles in Pan they change with speed? Journal of Human troglodytes and Pongo pygmaeus. American Movement Study, 10, 175–188. Journal of Physical Anthropology, 61, 75–83. Vilensky, J. A., & Larson, S. G. (1989). PrimateVangor, A. K., & Wells, J. P. (1983). Muscle locomotion: Utilization and control of recruitment and the evolution of bipedality: symmetrical gaits. Annual Review of Evidence from telemetered electromyography of Anthropology, 18, 17–35. spider, woolly and patas monkeys. Annales des Vilensky, J. A., Libii, J. N., & Moore, A. M. (1991). Sciences naturelles, zoologie, Paris, 5, 125–135. Trot-gallop transition in quadrupeds. ˆVereecke E, D’ Aout K, De Clercq D, Van Elsacker L, Physiology and Behavior, 50, 835–842. Aerts P (2003) Dynamic plantar pressure Vilensky, J. A., & Patrick, M. (1985). Gait distribution during terrestrial locomotion of characteristics of two squirrel monkeys with bonobos (Pan paniscus). Am. J. Phys. emphasis on the relationship with speed and Anthropol. 119, 37–51. neural control. American Journal of Physical ˆVereecke, E. E., DAut, K. and Aerts, P. (2006). The Anthropology, 68, 429–444. dynamics of hylobatid bipedalism: evidence for Vilensky, J. A., Moore, A. M., & Libii, J. N. (1994). an energy-saving mechanism? J. Exp. Biol. 209, Squirrel monkey locomotion on an inclined 2829-2838. treadmill: Implications for the evolution of gaits.Vilensky, J. A. (1980). Trot-gallop transition in a Journal of Human Evolution, 26, 375–386. macaque. American Journal of Physical Vilensky, J. A., Moore-Kuhns, M., & Moore, A. M. Anthropology, 53, 347–348. (1990). Angular displacement patterns ofVilensky, J. A. (1983). Gait characteristics of two leading and trailing limb joints during macaques with emphasis on the relationship galloping in monkeys. American Journal of with speed. American Journal of Physical Primatology, 22, 227–239. Anthropology, 61, 255–265. Ward, C. V. (2002). Interpreting posture andVilensky, J. A. (1987). Locomotor behavior locomotion of Australopithecus afarensis: and control in humans and non-human Where do we stand? Yearbook of Physical primates: Comparisons with cats and dogs. Anthropology, 45, 185–215. Neuroscience and Biobehavioral Review, 11, Wells, J. P., & Wood, G. A. (1975). The application 263–274. of biomechanical motion analysis to aspects ofVilensky, J. A. (1988). Effects of size on Vervet green monkey (Cercopithecus a. sabaeus) (Cercopithecus aethiops) gait parameters: locomotion. American Journal of Physical A Cross sectional Approach. American Journal Anthropology, 43, 217–226. of Physical Anthropology, 76, 463–480. White, T. (1990). Gait selection in the brush-tailVilensky, J. A. (1989). Primate quadrupedalism: possum (Trichosurus vulpecula), the northern How and why does it differ from that of quoll (Dasyurus hallucatus), and the Virginia typical quadrupeds? Brain Behavioral opossum (Didelphis virginiana). Journal of Evolution, 34, 357–364. Mammals, 71, 79–84.Vilensky, J. A., & Gankiewicz, E. (1986). Effects of Whitehead, P. F., & Larson, S. G. (1994). Shoulder size on Vervet (Cercopithecus aethiops) gait motion during quadrupedal walking in parameters: A preliminary analysis. Folia Cercopithecus aethiops: Integration of Primatologica, 46, 104–117. cineradiographic and electromyographic data.Vilensky, J. A., & Gankiewicz, E. (1990). Effects of Journal of Human Evolution, 26, 525–544. speed on forelimb joint angular displacement Wunderlich, R. E., & Ford, K. R. (2000). Plantar patterns in vervet monkeys (Cercopithecus pressure distribution during bipedal and

PRIMATE LOCOMOTOR EVOLUTION 63 quadrupedal walking in the chimpanzee (Pan plantigrady in hominoids. American Journal of troglodytes). EMED Scientific Meeting, August, Physical Anthropology, Suppl. 30, 328. 2000, Munich, Germany. Yaguramaki, N. (1995). The relationship betweenWunderlich, RE and Jungers, WJ (2009). Manual posture and external force in walking. digital pressures during knuckle-walking in Anthropological Science, 103, 117–140. chimpanzees (Pan troglodytes) American Jou- Yamazaki, N., & Ishida, H. (1984). rnal of Physical Anthropology 139, 394–403. A biomechanical study of vertical climbingWunderlich, R. W., & Schmitt, D. (2000). and bipedal walking in gibbons. Journal of Hindlimb adaptations associated with heel-strike Human Evolution, 13, 563–571.

CHAPTER 4Foraging Cognition in Nonhuman Primates ¨Klaus Zuberbuhler and Karline JanmaatFORESTS AS PRIMATE HABITATS: opposable thumbs and toes, allowing them toCOEVOLUTION OF PRIMATES grasp and reach fruit at the terminal treeAND FRUIT branches, which are inaccessible to many other animals. Hindlimb dominance and graspingIn terms of total biomass, primates are very ability enable many primates to leap betweensuccessful vertebrates in most undisturbed tro- trees in an energetically efficient way, in contrastpical forests (Chapman et al., 1999a; Fleagle & other arboreal mammals such as most tree squir-Reed, 1996). Many primate species are forest rels (Gebo, 2004; Sussman, 1991; Taylor et al.,dwellers, and the forest habitat is likely to have 1972). Other adaptations concern forward-had a major impact on primate evolution. This is facing eyes and stereotypic vision, which facil-especially true for the great apes, whose changes itates hand–eye coordination and foraging atin diversity have followed climate-related retrac- high speed (Cartmill, 1972; Gebo, 2004).tions and expansions of wooded habitats since Similarly, diurnal activity, high visual acuity,the late Miocene (Potts, 2004). Most primates, and color vision enable spotting of fruit andincluding typical leaf-eaters, consume consider- their nutritional value from large distancesable amounts of fruits as part of their daily diets (Barton, 2000; Polyak, 1957; Riba-Hernandez ´(e.g., Korstjens, 2001). Fleshy fruits and the et al., 2005; Sumner & Mollon, 2000). Diurnalarthropods that associate with them are highly foraging is also beneficial because ripening ratesnutritious, which provide arboreal animals, such of fruits tend to be highest in the early afternoonas primates, birds, and bats, with a stationary following high midday incident radiation andand relatively reliable source of energy (Janmaat ambient temperature (Diaz-Perez et al., 2002;et al., 2006a). Primates and fruiting trees have Graham et al., 2003; Houle, 2004; Spayd et al.,shared a long evolutionary history, and the 2002).arrival of angiosperm fruits and flowers mayhave been of particular importance in primateevolution (Soligo & Martin, 2006; Sussman, EVOLUTIONARY THEORIES OF1991, 2004). About 85 million years ago, a PRIMATE COGNITIONtrend toward increased fruit size can be found(Eriksson et al., 2000), roughly coinciding with Primates, and especially humans, have relativelythe radiation of early ancestors of today’s pri- larger brains than other groups of mammals ´mates, about 82 million years ago (Tavare et al., (Harvey & Krebs, 1990; Jerison, 1973). It has2002). also been noted that a variety of brain size vari- Compared to other groups of animals, pri- ables in primates correlate positively with mea-mates possess a number of adaptations that sures of social complexity, such as group size,make them particularly suited for arboreal fora- deceptive behaviour, or strength of social bondsging on fruit. Many primate species have (Barton, 1996, 1999; Byrne & Corp, 2004; 64

FORAGING COGNITION IN NONHUMAN PRIMATES 65Dunbar, 1998; Dunbar & Shultz, 2007). This has geoffroyi) with less encephalized folivorousbeen taken to suggest that large groups, and howler monkeys (Alouatta palliata). It is inter-social complexity that emerges from them, esting that in diurnal frugivorous primates, rela-have acted as a primary selection force favouring tive brain enlargements are primarily foundthe evolution of increased brain size. This is within the visual system, while in nocturnal spe-because high social intelligence is likely to pro- cies enlargements are in the olfactory structuresvide individuals with a competitive and repro- (Barton et al., 1995), suggesting that the brainductive advantage over their less socially skilled has directly responded to the demands of fora-conspecifics. ging. In addition, increases in the degree of As appealing as it is, the social intelligence orbital convergence (associated with stereotypichypothesis has a number of problems. Large vision) correlate with expansion of visual brainpromiscuous multimale/multifemale groups, structures and, as a consequence, with overallthe presumed breeding grounds for high social size of the brain (Barton, 2004).intelligence, are the exception in primate socie- Compared to other body tissues, brains areties (Smuts et al., 1986) and it is often not spe- metabolically expensive organs, requiring acified how group size relates to social continuous and reliable flow of nutrientscomplexity. Moreover, although food competi- (Armstrong, 1983; Mink et al., 1981). Accor-tion is likely to increase with group size, larger ding to recent analyses, relative brain size isgroups also benefit from increased search swath positively correlated with basal metabolic rate,and accumulated knowledge of individuals to indicating that larger brains may be a reflectionlocate food sources, avoid predators, and deal of being able to sustain higher basal energy costswith neighboring groups (Garber & Boinski, (Isler & van Schaik, 2006a). Any increase in2000; Janson & Di Bitetti, 1997). Individuals relative brain size, therefore, may only be pos-are especially likely to benefit from older and sible in populations that have managed to eithermore knowledgeable group members during improve their access to nutrition or decreaseperiods of food scarcity when long-term experi- other existing energy demands. Energy can beence is more crucial (Byrne, 1995; Chauvin & saved, for example, by reducing an organism’sThierry, 2005; van Roosmalen, 1988). The social locomotor costs (Isler & van Schaik, 2006b) orintelligence hypothesis also struggles to explain reducing the metabolic requirements of otherhow exactly primates were able to grow expen- expensive tissues, such as the digestive systemsive large brains in the first place. Why did pri- (Aiello & Wheeler, 1995). Higher-quality foods,mates benefit more than other social animals such as fruit and animal matter, are easier tofrom increased encephalization? The relation- digest than other material, allowing theship between neocortex and group size is cer- organism to reduce the size of its digestivetainly real, but the causal arrow could also point tract. This hypothesis is supported by the find-the other way: Primates have evolved large ings that frugivorous primates usually have rela-brains for nonsocial reasons, which enables tively larger brains and smaller digestive systemsthem to live in larger groups, form more com- than folivorous primates (Barton, 2000;plex social systems, and maintain more complex Clutton-Brock & Harvey, 1980; Hladik, 1967).social relations than other smaller-brained spe- The various special adaptations for har- ¨cies (Muller & Soligo, 2005). vesting the fruits discussed in the previous sec- A main contender of social intelligence is the tion enabled primates to monopolize one of the‘‘ecological intelligence’’ hypothesis developed most nutritious food sources in these forests.by Milton (1981). Large brains, according to This may have allowed primates, especially hap-this idea, are the evolutionary products of exten- lorhines (see Chapter 1) that live in areas withsive mental mapping requirements faced by fru- relatively high fruit production, to afford largergivorous species, a hypothesis that emerged brains than other groups of animalsfrom empirical work comparing highly encepha- (Cunningham & Janson, 2007; Fish &lized and frugivorous spider monkeys (Ateles Lockwood, 2003). What benefits they gain

66 PRIMATE NEUROETHOLOGYfrom this relatively costly trait and what selec-tion pressures have favored its evolution is sub-ject of an ongoing debate. In sum, a more complete understanding forwhy primates have relatively bigger brains thanother groups of animals requires evidence at theultimate and proximate level (Tinbergen, 1963).The current literature favors social explanations,mainly because of what is available in terms ofempirical studies, but we have outlined anumber of reasons for caution. By contrast, wediscuss recent empirical progress on under-standing the impact of foraging problems oncognition. The studies we review all have beenconducted with the intent to investigate the cog-nitive capacities employed by nonhuman pri-mates in relation to finding food in theirnatural habitats, and we contend that some ofthese findings are of direct relevance to the eco-logical intelligence hypothesis.HOW DO FOREST PRIMATES Figure 4.1 A gray-cheeked mangabey (LophocebusKNOW WHERE TO FIND FRUIT? albigena johnstonii) feeding on purple flowers of Milettia dura. Picture by Rebecca Chancellor.A large-bodied monkey group’s home rangecan contain as many as 100,000 trees (e.g.,Lophocebus albigena johnstonii; Waser, 1974),yet only a small fraction of these trees will carryripe fruit at any given time. Estimates for someforests vary anywhere from 50 to 4,000 trees per Janmaat et al., 2006b; Figs. 4.3 and 4.4). Whenaverage home range (Janmaat et al., submitted). we measured the number of trees that wereAre primates able to find these trees, and how encountered while following individual mon-efficient are they at doing so? A number of stu- keys, we found that they encountered ordies found that wild primates were more effi- approached significantly more fruit-bearingcient in finding food than predicted by random trees than during control transects (i.e., whensearch models, suggesting that individuals use the observer walked a path parallel to the mon-some mental heuristics to locate food (e.g., key’s own route) (Janmaat, 2006; Janmaat et al.,Cunningham, 2003; Garber & Hannon, 1993; submitted).Janson, 2000; Milton, 2000; Valero & Byrne, It has been argued that the most efficient way2007). In our own studies on gray-cheeked and to optimize foraging success is to mentally repre-sooty mangabeys (Lophocebus albigena john- sent the location of all fruit trees in a home rangestonii; Cercocebus atys atys), we found that mon- as well as their fruiting state and overall temporalkeys were more likely to approach and search for patterning, that is, to maintain a cognitive mapfruit under or in trees that had produced fruits (Milton, 1981, 2000). According to most defini-than empty trees of the same species (Janmaat tions, cognitive maps are mental representationset al., 2006b; Figs. 4.1 and 4.2). of the real world, as if viewed from above, a We also found that target trees with fruit were Euclidian representation of landmarks withapproached significantly faster and with sharper vector distance and angular relationships betweenangles than trees without fruit (Janmaat, 2006; them (O’Keefe & Nadel, 1978; Tolman, 1948).

FORAGING COGNITION IN NONHUMAN PRIMATES 67Figure 4.2 Terrestrial sootymangabeys (Cercocebus atys atys)foraging for insects in dead wood.Picture by Karline Janmaat. However, current evidence suggests that it is something like a cognitive map comes from anot very likely that nonhuman primates, or even captive study with a young bonobo (Panhumans, represent their home ranges in such a paniscus) that was tested with an artificial lexi-way (Byrne, 1979; Janson & Byrne, 2007). One gram system, but it is unclear to what degreecrucial empirical test for the cognitive map primates use this capacity in the wild (Menzelhypothesis concerns the responses of individuals et al., 2002).to obstacles on a foraging route (Bennett, 1996),that is, whether they are capable of finding an SPATIO-TEMPORAL MENTALefficient detour. So far, wild primates have failed REPRESENTATIONS IN THEthis test. In one study, the behavior of wild NATURAL HABITATChacma baboons (Papio ursinus) was studiedwhen encountering neighboring groups on Apart from the general difficulties of providingtheir habitual foraging routes. Individuals did empirical evidence for a cognitive map, no onenot take detours around such groups to get to seriously doubts that primates are able to men-their foraging goal, but either waited for them to tally represent space in some way, although inpass by or simply abandoned their goal comple- many cases it is not clear what exactly thesetely (Noser & Byrne, 2007a). Some support for representations consist of, especially in the

68 PRIMATE NEUROETHOLOGY Route group Component 1 Component 2 Within 100/150 m. Group out of sight Group into sight T Target tree TFigure 4.3 Illustration of the method used to measure speed toward a target tree. The observer follows thegroup while staying within a 5-m distance of the individual that is closest to the target tree. Following tookplace in two components of direction, either along (a) component 1 (arrow) that is directed toward the treetrunk or (b) component 2 (thick dotted line), which is directed along the imaginary circle around the treetrunk. Speed was determined by counting steps per minute when walking in the direction of thetree (component 1) only. The observer was updated on the direction of the tree trunk by the calling orclicking sounds produced by a second observer, who was waiting under the tree trunk. The outer circle has aradius of 100 or 150 meters dependent on the species. The shaded area represents the area in which the groupcomes into sight of the second observer waiting under the tree.natural habitat. A major challenge in field studies Hannon, 1993; Janson, 1998; Milton, 2000;is to determine if primates reach a resource by Noser & Byrne, 2007a; Valero & Byrne, 2007).goal-directed travel, an indicator of mental repre- In one experiment with Argentinean capu-sentations of space, or by chance. It is important chin monkeys (Cebus apella nigritus), threeto consider that the shortest route is not always feeding sites were arranged in a triangle andthe most efficient one, and that animals could provisioned once per day. Once a monkeycombine different goals in one single route, and group had chosen a site, its next choice wasthat they could monitor food without exploiting between the two remaining sites, a close oneit (Sigg & Stolba, 1981). Some researchers have with less food and a far away one with moregenerated geometric or step models combined food. The surprising finding was that capuchinswith sophisticated statistics to determine the like- generally chose the closer feeding site, even whenlihood of whether spatial representations are the more distant site offered up to 12 times asinvolved in travel decisions (Bates, 2005; much food (Janson, 2007). Should we concludeCunningham & Janson, 2007; Garber & that the monkeys did not possess a mental

FORAGING COGNITION IN NONHUMAN PRIMATES 69 3500 No Fruit Approach Speed (cm/min) 3000 Fruit 2500 N = 17 2000 N = 89 1500 1000 N = 37 N = 12 500 0 Sooty Grey-cheekedFigure 4.4 Speed of approach to trees with and without fruits. For both mangabey groups, bars representthe median speeds, while the top and bottom of the boxes represent the percentiles. The highest and lowestwhiskers represent the highest and smallest values, which are not outliers. Circles and stars represent outliersand extreme values.representation of the locations and value of the The Role of Secondary Cuesprovided food? Alternatively, did they simply One problem with field studies is that it is oftenweigh up travel distance and likelihood of difficult to make reliable assumptions about howarriving at the food in time in the highly compe-titive situation of a rainforest? Further experi- far an individual can detect, using both visualments will be required to determine what exactly and olfactory sensory information, a targetinfluenced the monkeys’ foraging decisions. resource. Moreover, travel decisions may be Similarly, a recent study on Chacma baboons influenced by other secondary cues, such as(Papio ursinus) showed that the sleeping cliff, a food calls of other species. The availability ofpresumably important goal, was not always visual cues is particularly difficult to assess in aapproached fast and in a straight line, because rainforest where fruit trees are sometimes visiblethe group was regularly foraging for seeds close over considerable distances, even from theto the sleeping site (Noser & Byrne, 2007b). ground. Humans are capable of spotting fruitsSpider and woolly monkeys (Ateles belzebuth; in emergent trees from a distance of 150 m if theLagothrix poeppigii) travel through their home view is unobstructed, suggesting that other pri-ranges along repeatedly used paths, which has mates may possess comparable abilities (Gollabeen taken as evidence that spatial mental repre- et al., 2004; Janmaat, unpublished data).sentations are in the form of route-based or Only a small number of field studies have beennetwork maps (De Fiore & Suarez, 2007). able to convincingly reject the use of such sensoryNonrandom foraging patterns have also been cues to find resources (Garber & Paciulli, 1997;reported from tamarins (Saguinus mystax, S. fus- Janson, 1998; Janson & Di Bitetti, 1997; Sigg &cicollis) in the Amazon of northeastern Peru Stolba, 1981). For example, departure latency in(Garber, 1989). For great apes, the empirical evi- Chacma baboons was significantly shorter beforedence for spatial cognition is surprisingly weak. traveling to scarce mountain figs compared withOne study on tool-transporting behavior in wild traveling to other more abundant fruit sources.chimpanzees (Pan troglodytis) concluded that Because the fig trees were approximately 700 msubjects remembered distances between different from the sleeping site, visible to human observersnut-cracking sites and different stone hammers, only from short distances, it was unlikely that theas if using Euclidian space, but this interpretation monkeys were guided by any secondary cuesis controversial (Boesch & Boesch, 1984). (Noser & Byrne, 2007b). In another study, the

70 PRIMATE NEUROETHOLOGYranging behavior of sooty mangabeys was studied EVIDENCE FOR FRUITin relation to Anthonota trees with empty crowns LOCALIZATION STRATEGIES(Janmaat et al., 2006a; Fig. 4.5). Monkeys Another characteristic of forest fruits concernsapproaching within 150 m of empty trees were their ephemeral nature. Temporal patterns ofmore likely to approach if the tree was sur- emergence can be complex, and fruits are oftenrounded by fruits that had fallen to the forest present for short periods only (Chapman et al.,floor than if the tree had not produced any fruit. 1999b; 2004; Janmaat et al., submitted; Milton,The authors were able to rule out the possibility 1981, 1988). Many fruit tree species rely on ani-that the monkeys had seen any of the inconspic- mals for seed dispersal and have evolved featuresuous fallen fruits in the leafy substrate, indicating that make their fruits appeal to a large number ofthat the monkeys used spatial knowledge species, leading to high levels of inter- andacquired during previous feeding experiences to intraspecies competition (e.g., Hauser &relocate trees with fruit (Janmaat et al., 2006b). Wrangham, 1990; Houle et al., 2006; McGraw In a similar way, gray-cheeked mangabeys ¨ & Zuberbuhler, 2007; Sterck, 1995). Early arrivalthat came within 100 m of an empty Ficus sansi- is therefore advantageous, and natural selectionbarica tree were less likely to enter if the tree had is likely to favor any cognitive strategy thatrecently been depleted than if the tree had not makes this behavior possible. In the followingproduced any fruits so far (Fig. 4.6). Since both final section, we discuss a number of behavioraltree types had empty crowns, with no differences strategies, and their potential underlying cogni-in overall appearance, the visiting pattern was tive processes, that enable free-ranging primatesbest explained by memories of previous visits to deal with these temporal constraints.(Janmaat et al., 2006b). In sum, it seems safe to assume that navigation Monitoring Individual Treesof primates in their natural habitats involves somekind of mental representations of space, but it is Gray-cheeked mangabeys have been observed tooften unclear how enduring and rich these mem- bypass about a third of all available fruit-bearingories really are. Memories of spatial locations could fig species (Janmaat et al., 2006b). The monkeysbe relatively short lived (a few days), and there is were more likely to revisit trees in which they hadno good evidence for a geometric representation of good feeding experiences before, compared tospace (Byrne, 2000; Janson, 2000). trees in which they were less successful. SimilarFigure 4.5 A sooty mangabey eating Anthonota fragans fruit (left). Picture by Karline Janmaat. RipeAnthonota fragans fruits are harvested by the monkeys after they have fallen into the leaf litter underneath ¨the tree (right). Picture by Ralph Bergmuller. Used with permission.

FORAGING COGNITION IN NONHUMAN PRIMATES 71Figure 4.6 A gray-cheeked mangabey inspecting the ripening state of Ficus sansibarica fruit (left),F. sansibarica fruit (right). Pictures by Karline Janmaat. Used with permission.patterns have been reported from a study on wild Monitoring Meteorological Cuestamarins (Garber, 1989). For the mangabeys, Work on captive primates has shown that theyintriguingly, this was also the case for fig trees can readily learn to anticipate delayed foodwith unripe fruits. Unripe figs are attractive to rewards (Dufour et al., 2007; Ramseyer et al.,these monkeys because some of them contain 2005) and that they can trade off rewardweevil larvae or edible seeds. Monkeys have toinspect each fig individually, an interesting factfor the purpose of cognitive studies, because iteffectively rules out the possibility that the mon- Log10 average quality (#ind. * min) 2keys responded to long-distance visual or olfac- 3.5 R = 0.33tory cues (Janmaat et al., 2006a). In addition, the 3authors also found that the average speed with 2.5which the group approached such trees was sig-nificantly correlated with their average prior 2feeding experience in that tree (Fig. 4.7). Similar 1.5results have been reported from free-ranging sakis 1(Pithecia pithecia) in Venezuela. These primatesbypassed a majority of fruit-bearing trees without 1.8 2.3 2.8 3.3 Log10 average approach speedfeeding because they preferentially revisited spe-cific trees, which they already knew as highly Figure 4.7 Approach speed to trees of differentproductive from prior visits (Cunningham & quality. The sum of quality values per tree is plottedJanson, 2007). in relation to the average of the total speeds with which In sum, there is good evidence that primates are the gray-cheeked group traveled toward that tree.able to distinguish between individual trees, which Each dot represents the values of a target tree thatthey assess in terms of quality, and that they use carried unripe fruits. Values are based on an averagesuch memories in their daily foraging decisions. number of three visits (Nmin = 1, Nmax = 11).

72 PRIMATE NEUROETHOLOGYamount versus time delay (Stevens et al., 2005). between temperature and solar radiation andSimilar suggestions for a rudimentary ability to the maturation rate of fruit and insect larvaeanticipate future events have also been made for (Fig. 4.9). How exactly the monkeys managedwild primates, but only a few good empirical to register the relatively subtle differences instudies are available (Janmaat et al., 2006a; average temperature values was not addressed,Janson, 2007; Noser & Byrne, 2007b; Sigg & a topic for further research.Stolba, 1981; Wrangham, 1977). Temperature and solar radiation influence Monitoring Competitor Behaviorripening rates of fruits as well as the maturationof insect larvae inside them (e.g., Adams et al., The presence of other fruit-eating individuals may ´2001; Diaz-Perez et al., 2002; Houle, 2004; also serve as a reliable indicator of the presence ofMazzei et al., 1999; Morrison & Noble, 1990), edible fruits, especially for tree species that do notmaking the emergence of edible fruits somewhat have predictable patterns of fruit emergence andpredictable. A recent study on free-ranging man- that do not offer conspicuous secondary cues ofgabeys investigated whether these primates were edibility. In free-ranging tamarins (Saguinusable to take previous weather conditions into imperator and S. fuscicollis), high-ranking indivi-account when deciding to revisit particular duals tended to monitor the activities of otherfruit trees (Janmaat et al., 2006b). For this pur- group members, rather than to initiate their ownpose, a study group was followed from dawn to food searches, providing evidence that these pri-dusk for three continuous long observation per- mates were able to associate social cues with theiods totalling 210 days, yielding an almost com- presence of foods (Bicca-Marques & Garber,plete record of all revisit decisions toward 80 2005). Similarly, Tonkean macaques, Macaca ton-preselected fruit trees (Fig. 4.8). keana, kept in a large outdoor enclosure used food The results were consistent with the idea that odor cues, acquired by smelling the mouths ofthese monkeys made foraging decisions based on other group members, to guide their own searchepisodic-like memories of whether or not a tree for food (Chauvin & Thierry, 2005).previously carried fruit, combined with a more Primates also use auditory cues, such asgeneralized understanding of the relationship feeding calls of group members, to find fruitFigure 4.8 Measuring revisiting Behavior. The diagram illustrates an example of part of the study group’sdaily route (arrows) among target trees, each surrounded by an imaginary 100-m radius circle (dotted line).Once the group entered the circle, one observer rushed to the tree to determine the fruiting state andwhether the group came into sight and entered the tree. In this example, the group visited one tree with fruitand bypassed one without fruit.

FORAGING COGNITION IN NONHUMAN PRIMATES 73 Revisit Bypass 34 * * ns 32 Mean maximum daily temperature 30 28 of intervening days (°C) 26 24 49 143 71 90 91 22 92 20 18 Cues No cues No fruit Fruit previous visit previous visitFigure 4.9 The influence of temperature on revisiting behavior. Average daily maximum temperaturedetermined for the intervening period between the time the group entered the 100-m-radius circle and thetime the group last visited the same tree. Shaded boxes represent average temperature values for revisits;white boxes represent bypasses. Different clusters refer to trees that (1) did not carry fruit at the previousvisit, (2) carried fruit at the previous visit, and (3) carried fruit at the previous visit but no longer offered anysensory cues. Bars represent the median values of the average temperatures; top and bottom of the boxesrepresent the 75 and 25 percentiles. Whiskers represent highest and lowest values; circles represent outliers.Results showed that average daily maximum temperature was significantly higher for days preceding revisitsthan bypasses. These effects were found only for trees that carried fruit at the previous visit but not for treesthat had carried none, providing empirical evidence that these primates were capable of taking into accountpast weather conditions when searching for food.(e.g., tamarins, Saguinus labiatus: Caine et al., carried none, suggesting that primates could use1995; macaques, Macaca sinica: Dittus, 1984). the sound of sympatric foragers as an indicatorRed-tailed monkeys (Cercopithecus ascanius), for fruit availability (Janmaat, 2006). We thusblue monkeys (C. mitis), and gray-cheeked man- analyzed the behavior of our mangabey studygabeys have been suggested to recognize the group on 10 different occasions when they dis-food-arrival calls of sympatric frugivores covered newly emerged or newly ripened fruits(Hauser & Wrangham, 1990). (Janmaat, 2006). In 2 out of 10 encounters, the In Kibale National Park, Uganda, fig trees that tree was already occupied by a chimpanzee orcarried fruit contained a significantly larger hornbills feeding inside the tree, and interestinglynumber of noisy frugivorous animals, such as in these cases the speed of approach was muchchimpanzees or hornbills, than fig trees that higher than in the other eight cases (Fig. 4.10).

74 PRIMATE NEUROETHOLOGY 1800 Approach speed (cm/min) 1600 1400 1200 1000 first feeding visit 800 subsequent visit 600 400 200 0 1a 2a 3a 4b 5b 6c 7d 8d 9d Tree individualsFigure 4.10 Speed of approach at discoveries of new edible figs. Closed circles represent the speed withwhich a mangabey group approached a fig tree in which the group was thought to discover newly emergededible fruits. Open circles represent the speed with which the group approached the same tree at thesubsequent visit. Trees 1 and 2 were occupied by other frugivores before the group’s arrival. a, b, c, and drepresent the type of discovery visit: type (a) a feeding visit that succeeds a visit in which the group enteredbut did not eat; (b) a feeding visit in which the tree had grown new fruits during the observation period(50, 60, or 100 days); (c) a feeding visit in which the tree had grown new fruits during the observationperiod and that succeeds a visit in which the group entered but did not eat; and (d) the first time thatfeeding was observed in a tree after a period of at least 40 days in which the group did not come within100 m of the same tree. Olupot and colleagues (1998) found that Our observations suggested that the presencemangabeys were more likely to travel in the of chimpanzees did not stop the mangabeysdirection of areas from which hornbills from approaching fruit trees, despite the fact(Bycanistes subcylindricus) were calling earlier that chimpanzees are notorious monkey preda-in the day compared to other areas. Of course, tors. During continuous observation periodsit is possible that the monkeys already knew that totaling 210 days, we observed seven times thatthe targeted area contained fruit from previous the study group was feeding in a fig tree whenvisits, regardless of the hornbills’ behavior. Apart chimpanzees arrived. Three times the manga-from these and other anecdotes (e.g., Kinnaird & beys left the tree after being chased by maleO’Brien, 2000), little systematic experimental chimpanzees, which were in groups of moreresearch on the use of auditory cues in fruit than four. Four out of seven times, however,finding has been conducted. the monkeys continued feeding together with To address the issue, we conducted a series of the chimpanzees, but these were usually singleplayback experiments in which we played dif- individuals or small groups. Twice, one of theferent animals’ sounds from fig trees that either mangabey males even chased a female chim-carried no fruit or only unripe, inedible fruits. panzee out of the tree. On a further six occa-We used calls produced by hornbills or chim- sions, the mangabeys encountered chimpanzeespanzees, which were recorded while individuals that were already feeding inside a fig tree. In allwere feeding inside fig trees (Fig. 4.11). As a cases, the study group eventually entered thecontrol, we used the territorial calls of a local tree, five times within 100 minutes after waitingbird species, the yellow-rumped tinkerbird at the same spot, and one time only after 6 hours,(Pogoniulus bilineatus). KJ carried out all experi- after some additional traveling.ments with the help of field assistants. For each To investigate systematically whether thesetrial the speaker was positioned at an elevation of monkeys took the presence of other frugivorousat least 12 m within a fig tree. species into account when trying to locate food

FORAGING COGNITION IN NONHUMAN PRIMATES 75Figure 4.11 Chimpanzees oftenproduce pant-hoot and roughgrunts before feeding on Ficuscapensis fruits. Picture by KarlineJanmaat. Used with permission.trees, we conducted two playback experiments. chimpanzee feeding grunts, played from withinThe first consisted of a small number of calls a fig tree, with chimpanzee pant hoot vocaliza-(hornbill and chimpanzee), played from inside tions, played from the forest floor. Finally, thefig trees, with variable fruiting states, at a dis- monkeys did not respond differently to chim-tance of 50 m to 200 m from five different panzee feeding grunts played from empty treesmangabey groups. The second type consisted of or trees with potentially ripe fruits (Janmaata large number of hornbill or tinkerbird calls, 2006). An alternative explanation is that theplayed from inside fig trees, regardless of mangabeys had previously visited the experi-whether we knew of the presence of any mon- mental trees and already knew that the tree didkeys nearby. not carry any edible fruits. After playing back hornbill vocalizations, the Monitoring Synchronicityexperimental tree was reliably approached byother hornbills, suggesting that the playback sti- Most rainforest trees produce fruits synchro-muli were effective. However, we never managed nously with fruit production peaking someto attract any mangabeys in response to playback time of the year (Chapman et al., 1999b; vanof chimpanyee or hornbill calls compared to Schaik et al., 1993). In these species, findingtinkerbird control calls. When comparing fruit in one tree can be a reliable indicator forlatency and duration of looking towards the the presence of fruit in other trees of the samespeaker, we did not find any differences between species, potentially allowing primates to makethe hornbill calls and tinkerbird control calls. predictions about where to find fruit withoutWe also failed to detect any differences in these having to remember the fruiting states of indi-measures when comparing playbacks of vidual trees. Japanese macaques (Macaca

76 PRIMATE NEUROETHOLOGYfuscata), artificially provisioned with fruits of the findings that primates generally forage on a rela-Akebia trifoliate vines prior to fruiting season, tively small number of commonly distributedwere more likely to inspect other Akebia trifoliate species per time period (Eckardt &vines than if they were provided with other food ¨ Zuberbuhler, 2004; Janson et al., 1986) anditems (Menzel, 1991). Intriguingly, the monkeys ‘‘trap-line’’ trees of species that have a high den-manipulated both Akebia trifoliata and Akebia sity and high fruit production (Janson et al.,quinatavines, although the leaves and fruits of 1986; Milton, 2000; Terborgh & Stern, 1987).this vine species look very different. Both Akebia At a proximate level, it is possible that monkeysspecies fruit simultaneously, suggesting that the develop a ‘‘search image,’’ originally proposed tomonkeys were not simply searching for the ori- explain the behavior of predators (Tinbergen,ginal source of the presented fruit, but used the 1960). Identifying trees by their visual featuresdiscovery of a fruit as an indicator for the pre- is not a trivial task, as trees of different fruitsence of fruit in vines of the same or other species can resemble each other to a highsimultaneous fruiting vine species. degree (Janmaat, 2006; Fig. 4.12). During the Compared to temperate zones of the Japanese entire study period the mangabeys fed on 28woodlands, seasonality is much less pronounced different fruit tree species, suggesting that thein African rainforests (Walter, 1984; Worman & monkeys must be able to retain a long-termChapman, 2005). In Kibale forest, Uganda, a memory of the specific visual characteristics ofmajority of tree species fruit synchronously a large number of fruit tree species, which could(64%), but the percentage of trees that carry require substantial processing power (Barton,fruit during fruiting peaks differs substantially 2000; Fagot & Cook, 2006).between species, and within species betweenyears and areas (Chapman et al., 1999b; CONCLUSIONSJanmaat et al., submitted). For example,Strombosia scheffleri produced fruits only four Several decades ago, Eisenberg (1973) and Napiertimes within 12 years (Chapman et al., 2004), (1970) described what sets primate societies apartwith variable peaks from 5% to 50%. In May from those of other long-lived animals. The sug-1996, 60% of the Uvariopsis congensis population gestion was that primates were equipped withat the Kanyawara research site carried fruit, brains able to store and retrieve a great deal ofwhile none did at three other research sites, all independently acquired information about thewithin a 12-km distance (Chapman et al., 2004). environment and able to apply considerable In a recent study, Janmaat and colleagues degrees of behavioral plasticity in responding to(submitted) investigated the foraging behavior specific situations. In searching for food, primatesof gray-cheeked mangabeys in relation to dif- could be more skilled than other groups of animalsferent levels of synchronicity in rainforest fruit in their abilities to combine and integrate differentspecies. Results showed that active searching was types of information. For example, in order to useonly triggered if the monkeys encountered high synchronicity in fruit emergence, monkeys need tofrequencies of trees with ripe fruits in the same keep track of the local density of several fruitarea. Thresholds for switching to an ‘‘inspect- species, recognize the visual characteristics ofall’’ strategy appear to vary between different fruits and trees that show local abundance, and/tree species, perhaps influenced by the nutrition or remember the location of the trees or patches ofvalue and productivity of the trees. Such a trees that have started fruiting (Janmaat et al.,strategy is likely to be adaptive for the monkeys, submitted). There is currently no strong evidencebecause it allows them to flexibly respond to that primates use olfactory cues or the soundsfrequent and irregular fluctuations in fruit pro- made by other animal species to locate fruit trees,duction and differences in nutritional value but it is likely that such cues are integrated withbetween species (Chapman et al., 1999b, 2003, spatial knowledge of likely food sources (Janmaat,2004; Janson et al., 1986; Worman & Chapman, 2006). Before revisiting particular trees, primates2005). The results were also consistent with the appear to combine weather conditions with

FORAGING COGNITION IN NONHUMAN PRIMATES 77 T. nobilis U. congensis Trunks of U. congensis vs T. nobilisFigure 4.12 Illustration of the similarity in appearance of U. congensis and T. nobilis trees. Leafshape, color, and configuration of U. congensis (top left) and T. nobilis (top right) and a close-upof the (yellow marked) trunks of both tree species (bottom). Despite the similarity, mangabeys didnot enter more T. nobilis trees in U. congensis season than out of season, or in areas with higherripe U. congensis fruit densities, suggesting that the monkeys use a memory of the visualcharacteristics of the trees or the locations of the tree patches of each species when searching forfruit (Janmaat 2006). Used with permission.memories of previous fruiting states of trees, and How are these foraging abilities relevant to thethey may even have some rudimentary under- more general question of why primates have rela-standing that high temperature and radiation tively larger brains than other animals? Fieldworkaccelerates fruit ripening (Janmaat et al., 2006b). in different parts of the world has shown thatOne emerging point from the studies reviewed is primates have been exceptionally successfully inthat successful foraging depends on various cogni- monopolizing the arboreal space of most tropicaltive skills, much beyond simply remembering the forests, much more so than other groups of ani-spatial location of a number of food trees mals. Moreover, primates possess a number ofthroughout a home range. The degree to which morphological adaptations that make them espe-nonprimate species possess comparable abilities is cially well suited for arboreal foraging. The brainan important question, but unfortunately the is an expensive organ, and primates’ reliableanswer is largely unknown. access to the highest-quality nutrition available

78 PRIMATE NEUROETHOLOGYin this habitat may have enabled them to afford ACKNOWLEDGMENTSunusually large brains. Fruit-eating bats and birds also feed on KJ’s fieldwork was funded by the followingarboreal fruit, but these competitors are perhaps organizations: the Wenner-Gren Foundation,more constrained by their specialized locomotor the Leakey Foundation, the University of Stapparatus, which also prevents them from Andrews’ School of Psychology, Primatemanipulating and harvesting difficult-to-open Conservation International Inc., Schure-fruits (Isler & van Schaik 2007b; Ross, 1996). Bijerinck-Popping fonds of the KNAW,Parrots, often noted for their exceptional cogni- Stichting Kronendak, Dobberke Stichting voortive abilities, are somewhat of an exception as Vergelijkende Psychology, Lucie Burgertheir tongue, feet, and toes are highly mobile, Stichting, and Fonds Doctor Catharine vanallowing them to manipulate and discard lower- Tussenbroek. The Tai Monkey Project obtainedquality portions of food items (Milton 2001). core funding by the ESF’s ‘‘Origins of Man,Isler and van Schaik (2007b) report a negative Language and Languages’’ program, as well asrelationship between brain size and the relative the EC’s FP6 ‘‘What it means to be human.’’ In ` Ivory Coast, we thank the Ministere de lamass of pectoral muscle in birds, which are cru- ` Recherche Scientifique and the Ministere decial for taking in air. It is interesting that someforest birds appear to minimize these costs by l’Agriculture et des Ressources Animales of ˆ Cote d’Ivoire, the Centre Suisse de Recherchesclimbing up trees with their hooked claws Scientifiques (CSRS), PACPNT, Eaux et Foret, ˆ(Hoatzin bird; Opisthocomus hoazin). Primates ¨ and the Tai Monkey Project (R. Noe in parti-are less constrained in these ways, which mayhave allowed them to evolve larger bodies and cular) for logistic support and permission tobrains while accessing the most nutritious foods, conduct research in the Tai National Park.including young leaves, fleshy fruits, and the In Uganda, we thank the Office of thearthropods associated with them (Kay, 1984; President, Uganda National Council forMartin, 1990). As a result, primates have been Science and Technology, Uganda Wildlifeable to evolve more complex behavioral strate- Authority, Makerere University Biologicalgies and mental capacities when dealing with Field Station, and Kibale Fish and Monkeyboth environmental and social problems. Project (C. A. Chapman in particular) forVarious studies reviewed in this chapter indicate logistic support and permission to conductthat primates engage in a number of complex research in the Kibale National Park. Forcognitive foraging strategies, which gives them invaluable assistance in the field we arean advantage in competing over food with other indebted to R. Meijer, D. J. van der Post, L. B.species, reinforcing their chosen strategy to Prevot, D. C. M. Wright, R. Samuel, J. Rusoke,invest in brain size. Although not very popular and P. Irumba. We thank Gabriel Ramos-at the moment, the ecological intelligence Fernandez, Malcolm MacLeod, and Dickhypothesis appears to be more parsimonious Byrne for their comments and suggestions.than its rivals in the evolutionary scenario itpresupposes: Overall, primates have been more REFERENCESsuccessful in exploiting sustained high-qualitynutrition from their habitats compared to Adams, S. R., Cockshull, K. E., & Cave, C. R. J.their competitors, which has allowed them to (2001). Effect of temperature on growth and development of tomato fruits. Annals ofevolve an unusually large brain. The sophistica- Botany, 88, 869–877.tion seen in primates’ social behavior as well as Aiello, L. C., & Wheeler, P. (1995). The expensiveother aspects of their cognitive sophistication tissue hypothesis. Current Anthropology, 36,may be a by-product of their highly encephalized 184–193.neural system, afforded by their special adapta- Armstrong, E. (1983). Relative brain size andtions to the ecological conditions of the forest metabolism in mammals. Science, 220, 1302–habitat. 1304.

FORAGING COGNITION IN NONHUMAN PRIMATES 79Barton, R. A., Purvis, A., & Harvey, P. H. (1995). Byrne, R. W. (1995). The thinking ape. Evolutionary Evolutionary radiation of visual and olfactory origins of intelligence. Oxford: Oxford University brain systems in primates, bats and insectivores. Press. Philosophical Transactions of the Royal Society B, Caine, N. G., Addington, R. L., & Windfelder, T. L. 348, 381–392. (1995). Factors affecting the rates of food callsBarton, R. A. (1996). Neo-cortex size and behavioral given by red-bellied tamarins. Animal ecology in primates. Proceedings of the Royal Behaviour, 50, 53–60. Society of London, Series B, 263, 173–177. Cartmill, M. (1972). Arboreal adaptations and theBarton, R. A. (1999). The evolutionary ecology origin of the order Primates. In: Tuttle R. H. of the primate brain. In: P. C. Lee (Ed.), (Ed.), The functional and evolutionary biology of Comparative primate socioecology (pp. 167– primates (pp. 97–122). Chicago: Aldine- 194). Cambridge: Cambridge University Atherton. Press. Chapman, C. A., Chapman, L. J., Struhsaker, T. T.,Barton, R. A. (2000). Primate brain evolution: Zanne, A. E., Clark, C. J., & Poulsen, J. R. Cognitive demands of foraging or of social (2004). A long-term evaluation of fruit life? In: S. Boinski & P. A. Garber (Eds.), On phenology: Importance of climate change. the move. How and why animals travel in groups Journal of Tropical Ecology, 21, 1–14. (pp. 204–238). Chicago, London: University of Chapman, C. A., Chapman, L. J., Rode, K. D., Chicago Press. Hauck, E. M., & McDowell, L. R. (2003).Barton, R. A. (2004). Binocularity and brain Variation in the nutritional value of primate evolution in primates. Proceedings of the foods: Among trees, time periods, and areas. National Academy of Sciences of the United International Journal of Primatology, 24, 317–333. States of America, 101, 10113–10115. Chapman, C. A., Gautier-Hion, A., Oates, J. F., &Bates, L. (2005). Cognitive aspects of travel and food Onderdonk, D. A. (1999a). African primate location by chimpanzees (Pan troglodytes communities: Determinant of structure and schweinfurthii) of the Budongo Forest Reserve, threats to survival. In: J. G. Fleagle, C. H. Uganda. Ph.D. thesis. University of St. Andrews, Janson, & K. E. Reed (Eds.), Primate St. Andrews. communities (pp. 1–37). Cambridge:Bennett, A. T. C. (1996). Do animals have cognitive Cambridge University Press. maps? Journal of Experimental Biology, 199, Chapman, C. A., Wrangham, R. W., Chapman, 219–224. L. J., Kennard, D. K., & Zanne, A. E. (1999b).Bicca-Marques, J. C., & Garber, P. A. (2004). Fruit and flower phenology at two sites in Use of spatial, visual, and olfactory Kibale National Park, Uganda. Journal of information during foraging in wild Tropical Ecology, 15, 189–211. nocturnal and diurnal anthropoids: A field Chauvin, C., & Thierry, B. (2005). Tonkean experiment comparing Aotus, Callicebus, macaques orient their food search from and Saguinus. American Journal of Prima- olfactory cues conveyed by group mates. tology, 62, 171–187. Ethology, 111, 301–310.Boesch, C., & Boesch, H. (1984). Mental map in wild Clutton-Brock, T. H., & Harvey, P. H. (1980). chimpanzees: An analysis of hammer transports Primate brains and ecology. Journal of Zoology for nut cracking. Primates, 25, 160–170. London, 190, 309–323.Byrne, R. W. (2000). How monkeys find their way. Cunningham, E., & Janson, C. (2007). A Leadership, coordination, and cognitive maps socioecological perspective on primate of African baboons. In: S. Boinski & P. A. cognition, past and present. Animal Cognition, Garber (Eds.), On the move: How and why 10, 273–281. animals travel in groups (pp. 491–518). Cunningham, E. P. (2003). The use of memory in Chicago: University of Chicago Press. Pithecia Pithecia’s foraging strategy. Ph.D.Byrne, R. W., & Corp, N. (2004). Neocortex size thesis. The City University of New York, predicts deception in primates. Proceedings of New York. the Royal Society B, 271, 1693–1699. Di Fiore, A., & Suarez, S. A. (2007). Route-basedByrne, R. W. (1979). Memory for urban geography. travel and shared routes in sympatric spider and Quarterly Journal of Experimental Psychology, woolly monkeys: Cognitive and evolutionary 31, 147–154. implications. Animal Cognition, 10, 317–329.

80 PRIMATE NEUROETHOLOGYDiaz-Perez, J. C., Bautista, S., Villanueva, R., & Garber, P. A. (1989). Role of spatial memory in Lopez-Gomez, R. (2002). Modelling the primate foraging patterns Saguinus mystax and ripening of sapote mamey (Pouteria sapota Saguinus fuscicollis. American Journal of (Jacq.) H. E. Moore and Stearn) fruit at Primatology, 19, 203–216. various temperatures. Postharvest Biology and Garber, P. A., & Boinski, S. (2000). On the move. Technology, 28, 199–102. How and why animals travel in groups. Chicago,Dittus, W. P. J. (1984). Toque macaque food calls: London: University of Chicago Press. semantic communication concerning food Gebo, D. L. (2004). A shrew-size origin for distribution in the environment. Animal primates. Yearbook of Physical Anthropology, Behaviour, 32, 470–477. 47, 40–62.Dufour, V., Pele, M., Sterck, E. H. M., & Thierry, B. Golla, H., Ignashchenkova, A., Haarmeier, T., & (2007). Chimpanzee anticipation of food Thrier P. (2004). Improvement of visual acuity return: coping with waiting time in an by spatial cueing: a comparative study in human exchange task. Journal of Comprehensive and non-human primates. Vision Research, 44, Psychology, 121(2), 145–155. 1589–1600.Dunbar, R. I. M. (1998). The social brain Graham, E. A., Mulkey, S. S., Kitajima, K., Philips hypothesis. Evolutionary Anthropology, 6, 178– N. G., & Wright, S. J. (2003). Cloud cover limits 190. net CO2 uptake and growth of a rainforest treeDunbar, R. I. M., & Shultz, S. (2007). Evolution of during tropical rainy seasons. Proceedings of the the social brain. Science, 317, 1344–1347. National Academy of Sciences, 100, 572–576. ¨Eckardt, W., & Zuberbuhler, K. (2004). Coope- Harvey, P. H., & Krebs, J. R. (1990). Comparing ration and competition in two forest monkeys. brains. Science, 249, 140–146. Behavioral Ecology, 15, 400–411. Hauser, M. D., & Wrangham, R. W. (1990).Eisenberg, J. F. (1973). Mammalian social systems: Recognition of predator and competitor calls Are primate social systems unique? In: E. W. in nonhuman primates and birds: A Menzel, Jr. (Ed.), Precultural primate behavior preliminary report. Ethology, 86, 116–130. (pp. 232–249). Basel, Switzerland: S. Karger. Hladik, C. M. (1967). Surface relative du tractus ¨Eriksson, O., Friis, E. M., & Lofgren, P. (2000). Seed digestif de quelques primates: Morphologies size, fruit size and dispersal systems in villosites intestinales et correlations avec le angiosperms form the early Cretaceous to the regime alimentaire. Mammalia, 31, 120–147. late Tertiary. American Natute, 156, 47–58. Houle, A., Vickery, W. L., & Chapman, C. A.Fagot, J., & Cook, R. G. (2006). Evidence for large (2006). Mechanisms of coexistence among two long-term memory capacities in baboons and species of frugivorous primates. Journal of pigeons and its implications for learning and Animal Ecology, 75, 1034–1044. the evolution of cognition. Proceedings of the ´ Houle, A. (2004). Mecanismes de coexistence chez National Academy of Sciences, 103, 17564–17567. les primates frugivores du Parc National deFish, J. L., & Lockwood, C. A. (2003). Dietary Kibale en Ouganda. Ph.D. thesis. University of constraints on encephalization in primates. Quebec, Quebec. American Journal of Physical Anthropology, 120, Isler, K., & van Schaik, C. P. (2006a). Metabolic 171–181. costs of brain size evolution. Biology Letters, 2,Fleagle, J. G., & Reed, K. E. (1996). Comparing 557–560. primate communities: A multivariate approach. Isler, K., & van Schaik, C. P. (2006b). Costs of Journal of Human Evolution, 30, 489–510. encephalization: The energy trade-offGarber, P. A., & Hannon, B. (1993). Modeling hypothesis tested on birds. Journal of Human monkeys: A comparison of computer- Evolution, 51(3), 228–243. generated and naturally occurring foraging Janmaat, K. R. L. (2006). Fruits of enlightenment. patterns in two species of neotropical Fruit localization strategies in wild mangabey primates. International Journal of Primatology, monkeys. Ph.D. thesis, University of St. Andrews, 14, 827–852. St. Andrews.Garber, P. A., & Paciulli, L. M. (1997). Experimental ¨ Janmaat, K. R. L., Byrne, R. W., & Zuberbuhler, K. field study of spatial memory and learning in (2006a). Evidence for spatial memory of fruiting wild capuchin monkeys (Cebus capucinus). states of rain forest fruit in wild ranging Folia Primatologica, 68(3–5), 236–253. mangabeys. Animal Behavior, 71, 797–807.

FORAGING COGNITION IN NONHUMAN PRIMATES 81 ¨Janmaat, K. R. L., Byrne, R. W., & Zuberbuhler, K. socio-ecological strategies of the three colobines (2006b). Primates take weather into account ¨ of Taı. In: Behavioural Biology Group: Utrecht. when searching for fruit. Current Biology, 16, Martin, R. D. (1990). Primate origins and evolution: 1232–1237. A phylogenetic reconstruction. London: ¨Janmaat, K. R. L., Byrne, R. W., & Zuberbuhler, K. Chapman and Hall. (Submitted). The use of fruiting synchronicity Mazzei, K. C., Newman, R. M., Loos, A., & Ragsdale, by foraging Mangabeys (Lophocebus albigena D. W. (1999). Developmental rates of the native johnstonii). Animal Cognition. milfoil weevil, euhrychiopsis lecontei andJanson, C., & Byrne, R. W. (2007). Resource damage to Eurasian water milfoil at constant cognition in wild primates – opening up the temperature. Biological Control, 16, 139–143. black box. Animal Cognition, 10, 357–367. ¨ ¨ McGraw, S. W., Zuberbuhler, K. & Noe, R. (2007).Janson, C. (2007). Experimental evidence for ¨ Monkeys of the Taı Forest: An African Monkey route integration and strategic planning in Community. In: Cambridge Studies in Biological wild capuchin monkeys. Animal Cognition, and Evolutionary Anthropology (No. 51). 10, 341–356. Cambridge: Cambridge Univ. Press.Janson, C. H., & Di Bitetti, M. S. (1997). Menzel, C. R. (1991). Cognitive aspects of foraging Experimental analysis of food detection in in Japanese monkeys. Animal Behaviour, 41, capuchin monkeys: Effects of distance, travel 397–402. speed, and resource size. Behavioral Ecology Menzel, C. R., Savage-Rumbaugh, E. S., & Menzel, and Sociobiology, 41, 17–24. E. W., Jr. (2002). Bonobo (Pan paniscus) spatialJanson, C. H. (1998). Experimental evidence for memory and communication in a 20-hectare spatial memory in foraging wild capuchin forest. Journal of International Journal of monkeys Cebus apella. Animal Behaviour, 55, Primatology, 23, 601–619. 1229–1243. Milton, K. (1981). Distribution pattern of tropicalJanson, C. H. (2000). Spatial movement strategies: plant foods as an evolutionary stimulus to Theory, evidence and challenges. In: S. Boinski primate mental development. American & P. A. Garber (Eds.), On the move. How and Anthropologist, 83, 534–548. why animals travel in groups (pp. 165–204). Milton, K. (1988). Foraging behaviour and the Chicago, London: University of Chicago Press. evolution of primate intelligence. In: R.W.Janson, C. H., Stiles, E. W., & White, D. W. (1986). Byrne & A. Whiten (Eds.), Machiavellian Selection on plant fruiting traits by brown intelligence: Social expertise and the evolution of capuchin monkeys: a multivariate approach. intellect in monkeys, apes and humans In: A. Estrada & T. Fleming (Eds.), Frugivores (pp. 285–305). Oxford: Clarendon Press. and seed dispersal (pp. 83–92). The Hague: Junk Milton, K. (2000). Quo Vadis? Tactics of food Publishers. search and group movement in primates andJerison, H. (1973). Evolution of the brain and other animals. In: S. Boinski & P. A. Garber intelligence. New York: Academy Press. (Eds.), On the move. How and why animalsKay, R. F. (1984). On the use of anatomical features travel in groups (pp. 375–417). Chicago, to infer foraging behavior in extinct primates. London: University of Chicago Press. In: P. A. Rodman & J. G. H. Cant (Eds.), Mink, J. W., Blumenschine, R. J., & Adams, D. B. Adaptations for foraging in nonhuman primates (1981). Ratio of central nervous system to body (pp. 21–53). New York: Columbia University metabolism in vertebrates – its constancy and Press. functional basis. American Journal of Physiology,Kinnaird, M. F., & O’Brien, T. G. (2000). The Tana 241, 203–212. River Crested mangabey and the sulawesi crested Morrison, J. F., & Noble, A. C. (1990). The black macaque. Comparative movement patterns effects of leaf and cluster shading on the of two semi terrestrial cercopithecine primates. composition of cabernet sauvignon grapes In: S. Boinski & P. A. Garber (Eds.), On the and on fruit and wine sensory properties. move. How and why animals travel in groups American Journal of Enology and Viticulture, (pp. 327–350). Chicago, London: University of 41(3), 193–199. Chicago Press. ¨ Muller, A. E., & Soligo, C. (2005). Primate socialityKorstjens, A. H. (2001). The mob, the secret in evolutionary context. American Journal of sorority, and the phantoms: An analysis of the Physical Anthropology, 128, 399–441.

82 PRIMATE NEUROETHOLOGYNapier, J. (1970). The roots of mankind. Thomas langur and long-tailed macaque. Washington, D. C.: Smithsonian Press. Ph.D. thesis. Utrecht University, Utrecht.Noser, R., & Byrne, R. W. (2007a). Investigating the Stevens, J., Rosati, A., Ross, K., & Hauser, M. mental maps of chacma baboons (Papio (2005). Will travel for food: Spatial ursinus), using intergroup encounters. Animal discounting in New World monkeys. Current Cognition, 10, 331–340. Biology, 15(20), 1855–1860.Noser, R., & Byrne, R. W. (2007b). Travel routes Sumner, P., & Mollon, J. D. (2000). Chromaticity as and planning of visits to out-of-sight resources a signal of ripeness in fruits taken by primates. in wild chacma baboons (Papio ursinus). Journal of Experimental Biology, 203, 1987–2000. Animal Behaviour, 73, 257–266. Sussman, R. W. (1991). Primate origins and theO’Keefe, J., & Nadel, L. (1978). The hippocampus as evolution of angiosperms. American Journal of a cognitive map. Oxford: Oxford University Primatology, 23, 209–223. Press. Sussman, R. W. (2004). Flowering plants and theOlupot W., Waser, P. N., & Chapman, C. A. (1998). origins of primates: a new theory. In: M. Bekoff Fruit finding by mangabeys (Lophocebus (Ed.), Encyclopedia of animal behavior (pp. albigena): Are monitoring of fig trees and use 967–969). Portsmouth: Greenwood Press. of sympatric frugivore calls possible strategies? ´ Tavare, S., Marshall, C. R., Will, O., Soligo, C., & International Journal of Primatology, 19(2), Martin, R. D. (2002). Using the fossil record to 339–353. estimate the age of the last common ancestor ofPolyak, S. L. (1957). The vertebrate visual system. extant primates. Nature, 416, 726–729. Chicago: Chicago University Press. Taylor, C. R., Caldwell, S. L., & Rowntree, V. J.Potts, R. (2004). Paleo environmental basis of (1972). Running up and down hills: Some cognitive evolution in great apes. American consequences of size. Science, 178, 1096–1097. Journal of Primatology, 62, 209–228. Terborgh, J., & Stern, M. (1987). The surreptitious ´Ramseyer, A., Pele, M., Dufour, V., & Chauvin, C. life of the saddle-backed tamarin. American (2005). Accepting loss: The temporal limits of Scientist, 75, 260–269. reciprocity in brown capuchin monkeys. Tinbergen, N. (1960). The natural control of insects Proceedings of the Royal Society B, 273, 179–184. in pine woods: Vol. I. Factors influencing the ´Riba-Hernandez, P., Stoner, K. E., & Lucas, P. W. intensity of predation by songbirds. Archives (2005). Sugar concentration of fruits and their Neelandaises de Zoologie, 13, 265–343. detection via color in the Central American Tinbergen, N. (1963). On the aims of methods of spider monkey (Ateles geoffroyi). American ethology. Zeitschrift fur Tierpsychologie, 20, Journal of Primatology, 67, 411–123. 410–433.Ross, C. (1996). Adaptive explanation for the Tolman, E. C. (1948). Cognitive maps in rats and origins of the Anthropoidea (primates). men. Psychological Review, 55, 189–208. American Journal of Primatology, 40, 205–230. Valero, A., & Byrne R. W. (2007). Spider monkeySigg, J., & Stolba, A. (1981). Home range and daily ranging patterns in Mexican subtropical forest: march in a hamadryas baboon troop. Folia Do travel routes reflect planning? Animal Primatologica, 36, 40–75. Cognition, 10, 305–315.Smuts, B. B., Cheney, D. L., Seyfarth, R. M., Van Roosmalen, M. G. M. (1988). Diet, feeding Wrangham, R. W., & Struhsaker, T. T. (1986). behaviour and social organization of the Primate societies. Chicago: University of Guianan black spider monkey (Ateles paniscus Chicago Press. paniscus). 12th Congress of the InternationalSoligo, C., & Martin, R. D. (2006). Adaptive origins Primatological Society, July 24–29, Brasilia, of primates revisited. Journal of Human Brazil. Evolution, 50, 414–430. Van Schaik, C. P., Terborgh, J. W., & Wright, S. J.Spayd, S. E., Tarara, J. M., Mee, D. L., & Ferguson, (1993). The phenology of tropical forests: J. C. (2002). Separation of sunlight and Adaptive significance and consequences for temperature effects on the composition of vitis primary consumers. Annual Review of the vinifera cv. merlot berries. American Journal of Ecological System, 24, 353–377. Enology and Viticulture, 53(3), 171–182. Walter, H. (1984) Zonobiome of the equatorialSterck, E. H. M. (1995). Females, food and fights. A humid diurnal climate with evergreen tropical socioecological comparison of the sympatric rain forest. In: H. Walter (Ed.), Vegetation of the

FORAGING COGNITION IN NONHUMAN PRIMATES 83 earth and ecological systems of the geo biosphere and the response of three frugivores. Journal of (pp. 46–71). Berlin: Springer-Verlag. Tropical Ecology, 21, 689–697.Waser, P. M. (1974). Intergroup interacton in Wrangham, R. W. (1977). Feeding behaviour of a forest monkey: The mangabey Cercocebus chimpanzees in Gombe National Park, albigena. Ph.D. thesis. The Rockefeller Tanzania. In: T. H. Clutton-Brock (Ed.), University, New York. Primate ecology: Studies of feeding and rangingWorman, C., & Chapman, C. A. (2005). Seasonal behaviour in lemurs, monkeys and apes variation in the quality of a tropical ripe fruit (pp. 503–556). London: Academic Press.

CHAPTER 5Primate Vocal CommunicationRobert M. Seyfarth and Dorothy L. CheneyIn 1871, Charles Darwin drew attention to a can deduce an almost limitless number of mean-dichotomy in the vocal communication of ani- ings from the calls of others be able to producemals that had perplexed philosophers and nat- only a limited number of calls of his or her own?uralists for at least 2,000 years. In marked The difference between production and percep-contrast to human language, he wrote, animal tion is also puzzling because it constitutes avocalizations appeared to be involuntary expres- crucial distinction between human and non-sions of emotion and movement: ‘‘When the human primates. Why should monkeys andsensorium is strongly excited, the muscles of apes—so similar to humans in so many otherthe body are generally thrown into violent respects—be so different when it comes to vocalaction; and as a consequence, loud sounds are production?uttered, . . . although the sounds may be of no The contrast between vocal production anduse’’ (1871/1981, p. 83). However, two pages perception constitutes the starting point for thislater Darwin wrote: ‘‘That which distinguishes chapter. We wish to make three points. First, it isman from the lower animals is not the under- important to be clear exactly what we meanstanding of articulate sounds, for, as every one when we say that primate vocal production isknows, dogs understand many words and ‘‘sharply constrained.’’ Many scientists havesentences. . . . Nor is it the mere capacity of con- taken this to mean that primate call productionnecting definite sounds with definite ideas; for it is fixed, uncontrollable, and involuntary. Thisis certain that some parrots, which have been conclusion is too extreme. In fact, both fieldtaught to speak, connect unerringly words with and laboratory studies paint a more complexthings, and persons with events’’ (1871/1981, picture. Monkeys and apes can call or remainp. 85). silent, modify the timing and duration of calling, The vocal communication of monkeys and and make subtle acoustic modifications to theapes appears to be no different from that of other calls they give in specific social contexts.animals. Production is highly constrained. However, although they can modify call produc-Nonhuman primates have a relatively small tion in many ways, they rarely create entirelyrepertoire of calls, each of which is closely new calls or call combinations, or sever the linklinked to particular social circumstances and between a particular call type and the circum-shows little modification during development stances in which it is normally given.(see Hammerschmidt & Fischer, 2008, for a Second, the dichotomy between productionreview). Perception, by contrast, is more flexible, and perception has important theoretical impli-open ended, and modifiable as a result of experi- cations, because it draws our attention to theence. The difference between production and very different mechanisms that underlie theperception is puzzling because producers are behavior of speakers and listeners, even whenalso perceivers: Why should an individual who these individuals are involved in the same84

PRIMATE VOCAL COMMUNICATION 85communicative event. Nonhuman primates pre- A1 > A2 > A3 > B1 > B2 > C1, etc.) (Cheney &sent us with a communicative system in which a Seyfarth, 2007).small repertoire of relatively fixed, inflexiblecalls, each linked to a particular social context, Listeners Extract Rich ‘‘Narratives’’ fromnonetheless gives rise to an open-ended, highly Simple Call Sequencesmodifiable, and cognitively rich set of meanings. Third, the contrast between production and Baboon vocalizations, like those of many otherperception in primates and many other mammals primates, are individually distinctive (e.g.,cries out for an evolutionary explanation. What Owren et al., 1997; Rendall, 2003), and listenersselective pressures caused our human ancestors— recognize the voices of others (reviewed inand they alone among the primates—to develop Cheney & Seyfarth, 2007). Baboons’ vocal reper-flexible vocal production? Unconstrained by any toire contains a number of acoustically gradeddefinitive data that might help resolve the issue, signals, each of which is relatively context spe-we offer some speculations. cific. The alarm ‘‘wahoos’’ given by adult males to predators, for example, are acoustically similar to the wahoos that males give duringVOCAL PERCEPTION aggressive contests (Fischer et al., 2002).As Darwin noted, primates—like other mam- Nonetheless, listeners respond to the two callmals—produce a small repertoire of acoustically types as if they convey qualitatively differentfixed, species-specific calls that are closely tied to information (Kitchen et al., 2003).particular contexts and show little modification Grunts, the most common call given byduring development. By contrast, when it comes baboons, are given in a variety of social interac-to perception and comprehension, primates and tions and also differ acoustically according toother animals display an almost open-ended context. Move grunts are given in bouts of oneability to learn new sound-meaning pairs. They or two calls while the group is on a move oralso appear to ascribe intentions and motives to when one or more individuals attempt to initiatesignalers when assessing whether or not to a group move, and they often elicit answeringrespond to a given individual’s calls. Consider grunts from nearby listeners. Slightly acousti-baboons, for example. cally different infant grunts are given in a variety Baboons are Old World monkeys that shared of affiliative contexts and function to facilitatea common ancestor with humans roughly 30 social interactions (Cheney et al., 1995; Rendallmillion years ago (Steiper et al., 2004). They et al., 1999). If a high-ranking female grunts aslive throughout the savannah woodlands of she approaches a lower-ranking female, theAfrica in groups of 50 to 150 individuals. lower-ranking female is less likely to moveAlthough most males emigrate to other groups away than if the approaching female remainsas young adults, females remain in their natal silent. Grunts also function to reconcile oppo-groups throughout their lives, maintaining close nents after a dispute, increasing the likelihoodsocial bonds with their matrilineal kin (Silk that former opponents will tolerate each other’set al., 1999, 2006a,b). Females can be ranked in close proximity and reducing the probability ofa stable, linear dominance hierarchy that renewed aggression (Cheney & Seyfarth, 1997;determines priority of access to resources, and Cheney et al., 1995).daughters acquire ranks similar to those of their Because calls are individually distinctive andmothers. Baboon social structure can therefore each call type is predictably linked to abe described as a hierarchy of matrilines, in particular social context, baboon listeners canwhich all members of one matriline (e.g., potentially acquire quite specific informationmatriline B) outrank or are outranked by all from the calls that they hear. This applies notmembers of another (e.g., matrilines C and A, only to calls of a single type, like predator alarmrespectively). Ranks within matrilines are calls (Fischer et al., 2000, 2001a,b), but also toas stable as those between matrilines (e.g., the sequences of different call types that arise

86 PRIMATE NEUROETHOLOGYwhen two or more individuals are interacting grunts and C3 screams’’ is different from ‘‘C3with each other. threat-grunts and B2 screams’’ can make the Throughout the day, baboons hear other same judgment for all possible pairs of groupgroup members giving vocalizations to each members as well as any new individuals whoother. Some interactions involve aggressive may join (Cheney & Seyfarth, 2007, Chapterscompetition, for example, when a higher- 10 and 11).ranking animal gives a series of threat-grunts to Underlying the baboons’ sophisticated sociala lower-ranking animal and the latter screams. cognition is an almost open-ended ability toThreat-grunts are aggressive vocalizations given learn new sound-meaning pairs. This open-by higher-ranking to lower-ranking individuals, ended learning is found in many nonhumanwhereas screams are submissive signals, given primates, as well as other animals. Baboons andprimarily by lower- to higher-ranking indivi- other primates learn to recognize the voices ofduals. A threat-grunt-scream sequence, there- new individuals as they are born or join thefore, provides information not only about the group from elsewhere, just as they learn toidentities of the opponents involved but also distinguish their own species’ alarm callsabout who is threatening whom. Baboons are (Fischer et al., 2000; Seyfarth & Cheney, 1986;very sensitive to both types of information. In ¨ Zuberbuhler, 2000) and the different alarm callsplayback experiments, listeners respond with of sympatric birds and mammals (Hauser, 1988;apparent surprise to sequences of calls that Hauser & Wrangham, 1990; Seyfarth & Cheney,appear to violate the existing dominance ¨ 1990; Zuberbuhler, 2001). Primates in labora-hierarchy. Whereas they show little response tories readily learn to recognize the voices ofupon hearing the sequence ‘‘B2 threat-grunts their different caretakers and to associate dif-and C3 screams,’’ they respond strongly—by ferent sounds, like the rattling of keys or thelooking toward the source of the call—when beep of a card swipe, with impending eventsthey hear ‘‘C3 threat-grunts and B2 screams.’’ that may be good (feeding) or bad (the visit ofBetween-family rank reversals (C3 threat-grunts a veterinarian). In cross-fostering experiments,and B2 screams) elicit a stronger violation of infant rhesus (Macaca mulatta) and Japaneseexpectation response than do within-family macaques (M. fuscata) raised among the mem-rank reversals (C3 threat-grunts and C1 bers of another species learned to recognize theirscreams) (Bergman et al., 2003). foster mothers’ calls—and the foster mothers A baboon who ignores the sequence ‘‘B2 learned to recognize theirs—even in contexts inthreat-grunts and C3 screams’’ but responds which the two species used acoustically differentstrongly when she hears ‘‘C3 threat-grunts and vocalizations (Seyfarth & Cheney, 1997). TakenB2 screams’’ reveals, by his or her responses, that together, these results suggest that nonhumanhe or she recognizes the identities of both parti- primates are both innately predisposed tocipants, their relative ranks, and their family ascribe meaning to different sounds and alwaysmembership. He or she also acts as if he or she ready to learn new information from novel audi-assumes that the threat-grunt and scream have tory stimuli.occurred together not by chance, but because These generalizations apply with equal forceone vocalization caused the other to occur. to other mammals. Consider Rico, for example,Without this assumption of causality there a border collie who learned the names of morewould be no violation of expectation when B2’s than 200 different toys (Kaminski et al., 2004).scream and C3’s threat-grunt occurred together. Rico was able to learn and remember the names Baboons’ ability to deduce a social narrative of new toys by process of exclusion, or ‘‘fastfrom a sequence of sounds reveals a rich cogni- mapping,’’ and—like small children—usedtive system in which listeners extract a large gaze and attention to guide word learning. Butnumber of complex, nuanced messages from a of course Rico never learned to say any of therelatively small, finite number of signals. words he learned. In this respect, his vocalA baboon who understands that ‘‘B2 threat- perception and production were similar to

PRIMATE VOCAL COMMUNICATION 87those of language-trained apes (Savage- To do so, the listeners must be able to recognizeRumbaugh, 1986; Terrace, 1979), sea lions that a grunt from a particular female is causally(Schusterman et al., 2002), and dolphins related to a previous fight even though she has(Herman et al., 1993). not interacted recently with the signaler, but with the signaler’s relative.Listeners Ascribe Intentions to Signalers There are intriguing parallels between these results and recent neurophysiological research.In addition to making judgments based on social In primates, faces and vocalizations are thecausation, baboons appear to recognize other primary means of transmitting social signals,individuals’ intentions and motives. Baboon and monkeys recognize the correspondencegroups are noisy, tumultuous societies, and between facial and vocal expressionsbaboons would not be able to feed, rest, or (Ghazanfar & Logothetis, 2003). When rhesusengage in social interactions if they responded macaques hear another monkey’s calls, theyto every call as if it were directed at them. In fact, exhibit neural activity not only in areas asso-baboons appear to use a variety of behavioral ciated with auditory processing but also incues, including gaze direction, learned contin- higher-order visual areas (Gil da Costa et al.,gencies, and the memory of recent interactions 2004). Ghazanfar and colleagues explored thewith specific individuals when making infer- neural basis of sensory integration usingences about the target of a vocalization. For the coos and grunts of rhesus macaques andexample, when a female hears a recent oppo- found that cells in the auditory cortex werenent’s threat-grunts soon after fighting with more responsive to bimodal (visual and audi-her, she responds as if she assumes that the tory) presentation of these calls than to unim-threat-grunt is directed at her, and she avoids odal presentation (Ghazanfar et al., 2005; seethe signaler. However, when she hears the same Romanski & Ghazanfar, this volume).female’s threat-grunts soon after grooming with Intriguingly, the effect of cross-modal presenta-her, she ignores the calls and acts as if the calls tion was greater with grunts than with coos. Theare directed at someone else (Engh et al., 2006). authors speculate that this may have occurredConversely, when a female hears her opponent’s because grunts are usually directed toward afriendly infant grunt soon after a fight, she acts as specific individual, whereas coos are moreif she assumes that the call is directed at her and often broadcast to the group at large. The greateris intended as a reconciliatory gesture. She cross-modal integration in the processing ofapproaches her recent opponent and tolerates grunts may arise because, in contrast to a coo,her opponent’s approaches at a rate that is even listeners who hear a grunt must immediatelyhigher than baseline rates (Cheney & Seyfarth, determine whether or not the call is directed1997). In contrast, hearing the grunt of an unin- at them.volved dominant female unrelated to her oppo-nent has no effect on the female’s behavior. Inthis latter case, she acts as if she is not the PRIMATE VOCAL PRODUCTIONintended target of the call and treats the call as Monkeys and apes have a relatively small reper-irrelevant. toire of context-specific calls that show relatively In some cases, inferences about the intended little modification in their acoustic propertiestarget of a call seem to involve rather complex during development (Janik & Slater, 1997;and indirect causal reasoning about, among McComb & Semple, 2005; Seyfarth & Cheney,other things, the kinship bonds that exist 1997). Cross-fostering experiments with maca-among others. Playback experiments, for ques suggest that the link between particular callexample, have shown that baboons will accept types and the contexts in which they are given isthe ‘‘reconciliatory’’ grunt by a close relative of a difficult to break. For example, normally raisedrecent opponent as a proxy for direct reconcilia- rhesus and Japanese macaques differ in their usetion by the opponent herself (Wittig et al., 2007). of calls in several social contexts: Rhesus

88 PRIMATE NEUROETHOLOGYmacaques use a mixture of coos and grunts, forces gave rise to learned, flexible vocal produc-whereas Japanese macaques use coos almost tion in our hominid ancestors? Below we offer aexclusively. In a 2-year experiment involving speculative answer to this question, but first wefour individuals who were raised in groups of consider more closely the ‘‘fixed’’ nature ofthe other species, infant rhesus and Japanese primate vocal production and explore themacaques adhered to their species-typical theoretical implications of communicationpattern of calling even though, in every other between relatively constrained vocal producersrespect, they were fully integrated into their and flexible, open-ended receivers.adopted social groups (Owren et al., 1993). There is also little evidence that nonhuman VOCAL PRODUCTION, THOUGHprimates adapt calls to different contexts or CONSTRAINED, IS NOT FIXEDcreate new calls to deal with novel situations. AND INVOLUNTARYAnd although they routinely hear differentcall combinations—combinations, like those Compared to the large, learned repertoires ofdescribed previously, whose meaning is more many songbirds and the imitative skills of ceta-than the sum of their constituent elements— ceans and pinnipeds, the vocal repertoires ofthese combinations are created when two nonhuman primates are small (McComb &baboons are vocalizing to each other. With a Semple, 20051). Nonhuman primates also usefew possible exceptions (see later), signalers their vocalizations in highly predictable socialnever combine different vocalizations to create circumstances. These two observations, togethernew messages. Thus, primate vocal repertoires with early neurophysiological studies showingare far from open ended. Production is very that seemingly normal calls could be elicited bydifferent from perception. electrical stimulation of subcortical areas in the brain (e.g., Jurgens & Ploog, 1970; Ploog, 1981), have led many anthropologists (Washburn,PRIMATES ARE TYPICAL OF MOST 1982; Gardenfors, 2003), ethologists (Goodall,MAMMALS 1986), linguists (Bickerton, 1990), psychologistsIn their highly constrained vocal production (Terrace, 1983), and neuroscientists (Arbib,combined with flexible perception and cogni- 2005) to conclude that primate vocalizationstion, nonhuman primates are typical of most are reflexive, involuntary signals—or, inmammals. Indeed, the ability to modify the Bickerton’s words, ‘‘quite automatic and impos-acoustic features of calls depending on experi- sible to suppress’’ (1990:142). This characteriza-ence seems comparatively rare in the animal tion is misleading.kingdom. As of 1997, when Janik and Slater In both the field and the laboratory, non-published their review of the topic, vocal human primates appear to be able to controllearning had been documented in only three whether they produce a vocalization or remainorders of birds, cetaceans, harbor seals, and silent. Baboons, as already noted, may follow anhumans. True, we know much more about aggressive interaction with a reconciliatoryvocal communication in monkeys than in non- grunt or they may not, and like other primatesprimate mammals or even the great apes, and we they vocalize more to some individuals than tomay yet be surprised by novel evidence of vocal others (e.g., Smith et al., 1982). Even in highlyimitation (e.g., Poole et al., 2005) or creative call emotional circumstances like encounters with ¨combinations (Arnold & Zuberbuhler, 2006; predators, some individuals call at high rates, ¨Crockford & Boesch, 2003; Zuberbuhler, 2002). others call less often, and still others remainFor the moment, however, there is no reason to silent (Cheney & Seyfarth, 1990).believe that mammals in general—including The ‘‘decision’’ to call or remain silent canapes—differ from the baboons and other have significant behavioral consequences. Inprimates described previously. The question for experiments conducted on wild capuchinprimatologists then becomes: What selective monkeys (Cebus capucinus) in Costa Rica,

PRIMATE VOCAL COMMUNICATION 89Gros-Louis (2004) found that individuals who acoustic similarity of their calls, Crockford anddiscovered food were more likely to give ‘‘food’’ colleagues could rule out an explanation of callcalls when other group members were nearby convergence based on shared genes. Instead,than when they were alone. Furthermore, they they propose that ‘‘chimpanzees may activelywere more likely to call if a higher-ranking, as modify pant hoots to be different from theiropposed to a lower-ranking, bystander was neighbors’’ (2004, p. 221). Such differencesnearby. Individuals who called when approached have functional consequences: Playbackby a high-ranking animal were less likely to experiments conducted by Herbinger (2003) onreceive aggression than those who remained individuals in the same West Africansilent. Gros-Louis (2004) concluded that capu- community found that chimpanzees recognizechin food calls function to announce both the pant-hoots of other individuals, associateownership and the signaler’s willingness to them with particular areas, and distinguish thedefend his or her possession. As a result, unless calls of neighbors from strangers.they were strongly motivated to take the food, Like rhesus macaques (Gouzoules et al., 1984),listeners refrained from harassing the signaler. wild chimpanzees who are receiving aggression In more controlled laboratory settings, the produce acoustically different screams dependingtiming, duration, and rate of calling by monkeys on the severity of the attack (Slocombe &can be brought under operant control (Pierce, ¨ Zuberbuhler, 2005). Intriguingly, Slocombe and1985). In a recent series of experiments, ¨ Zuberbuhler (2007) also found that chimpanzeeEgnor et al. (2007) exposed cotton-top tamarins victims produced screams that appeared to exag-(Saguinus oedipus) to intermittent bursts of gerate the severity of the attack, but they did sowhite noise and found that subjects quickly only when there was at least one individuallearned to restrict their calling to the silent nearby whose dominance rank was equal to orintervals. Clearly, then, primates can control higher than that of the aggressor. These resultswhether they vocalize or not depending on suggest that chimpanzees both have a limitedvariations in the ecological, social, and acoustic ability to modify the acoustic structure of theirenvironments. vocalizations and can recognize the dominance Within a given context, nonhuman primates relations that exist among others.can also make subtle modifications in the Laboratory experiments confirm that pri-acoustic structure of their calls (reviewed by mates can make subtle modifications to theHammerschmidt & Fischer, 2008). Wild chim- acoustic features of their calls depending onpanzees (Pan troglodytes) in Uganda, for experience. Elowson and Snowdon (1994) docu-example, give long, elaborate pant-hoots either mented acoustic differences between the calls ofalone or in ‘‘choruses’’ with others. When two pygmy marmosets (Cebuella pygmaea) housedindividuals have called together several times, in Washington, DC, and Madison, Wisconsin.the acoustic features of their pant-hoots begin When a group of marmosets was moved fromto converge (Mitani & Brandt, 1994; Mitani & Washington to Madison, the calls of the trans-Gros-Louis, 1998). Apparently, they modify the planted individuals changed to become moreacoustic structure of their calls depending on like their hosts’. In another experiment, Egnorauditory experience. Crockford et al. (2004) et al. (2007) exposed cotton-top tamarins totested this hypothesis on four communities of bursts of white noise just as they producedchimpanzees in the Tai Forest, Ivory Coast. They their ‘‘contact loud call.’’ The tamarinsfound that males in three contiguous commu- responded by producing calls that were shorter,nities had developed distinctive, community- with fewer pulses. Calls given immediatelyspecific pant-hoots, whereas males in a fourth before or after white noise were also louder andcommunity 70 km away showed only minor had longer interpulse intervals. Egnor andacoustic differences from males in the other Hauser (2004) review several other cases inthree communities. By comparing the genetic which nonhuman primates make subtle modifi-relatedness between pairs of males with the cations in the acoustic structure of their calls.

90 PRIMATE NEUROETHOLOGY In sum, it is misleading and overly simplistic The sharp distinction between signaler andto describe primate vocal production as ‘‘fixed’’ recipient helps to clarify a theoretical issue thatand ‘‘involuntary.’’ A more accurate conclusion has deviled studies of primates’—and otheris that the basic structure of nonhuman primate animals’—vocalizations since Darwin first dis-vocal signals appears to be innately determined, cussed them in The Expression of the Emotions inwhereas the fine spectrotemporal features can Animal and Man. Following Darwin, modernbe modified based on auditory experience ethologists have typically assumed that vocaland social context (Egnor & Hauser, 2004; communication in animals differs from humanHammerschmidt & Fischer, 2008). The distinc- language largely because the former is an ‘‘affec-tion between relatively innate and more modifi- tive’’ system based on emotion, whereas theable components of phonation is important, latter is a ‘‘referential’’ system based on the rela-because it has significant implications for tion between words and the objects or eventsfuture research on the neurobiology of primate they represent (see, for example, Hauser, 1996;communication (see Egnor & Hauser, 2004; Marler et al., 1992; Owings & Morton, 1998;Hammerschmidt & Fischer, 2008). For example, Owren & Rendall, 1997; Seyfarth et al., 1980).what brain areas are responsible for the innate But this dichotomy is logically false.and the modifiable components of primate call A call’s potential to serve as a referentialproduction, and how are these two aspects signal depends on how tightly linked the call isintegrated at the neural level? Second, what to a particular social or ecological context. Thecontextual factors are most important in mechanisms that underlie this specificity aremodifying primate vocal production: Age? irrelevant. A tone that informs a rat about theCaller identity? The history of interaction imminence of a shock, an alarm call that informsbetween participants? And what neural path- a vervet monkey about the presence of a leopard,ways are responsible for this modulation? One or a sequence of threat-grunts and screams thatrecent study found differences between the informs a baboon that B3 and D2 are involved inneural mechanisms involved in spontaneous a fight all have the potential to provide a listenervocalizations and those involved in the produc- with precise information because of their pre-tion of calls that were elicited by calls from dictable association with a narrow range ofanother individual (Gemba et al., 1999). Given events. The widely different mechanisms thatthe flexibility of human phonation, those inter- lead to this association have no effect on theested in the evolution of language will be curious signal’s potential to inform (Seyfarth &to know which social situations and areas of the Cheney, 2003). Put slightly differently, there isbrain are responsible for the limited flexibility no obligatory relation between ‘‘referential’’ andthat occurs in the phonation of monkeys’ and ‘‘affective’’ signaling. Knowing that a call isapes’ calls. referential (i.e., has the potential to convey highly specific information) tells us nothing about whether its underlying cause is affective‘‘AFFECTIVE’’ AND ‘‘SYMBOLIC’’ or not. Conversely, knowing that a call’s produc-SIGNALS: A FALSE DICHOTOMY tion is due entirely to the caller’s affect tells usNonhuman primates present us with a commu- nothing about the call’s potential to serve as anicative system in which a small repertoire of referential signal.relatively fixed and inflexible calls, each linked to It is therefore wrong, on theoretical grounds,a particular social context, nonetheless gives rise to treat animal signals as either referential orto an open-ended, highly modifiable, and affective, because the two properties of a com-cognitively rich set of meanings. Listeners municative event are distinct and independentextract rich, semantic, and even propositional dimensions. Highly referential signals could, ininformation from signalers who did not, in the principle, be caused entirely by a signaler’shuman sense, intend to provide it (Cheney & emotions, or their production could be relativelySeyfarth, 1998). independent of measures of arousal. Highly

PRIMATE VOCAL COMMUNICATION 91affective signals could be elicited by very specific (2003) conclusion that different levels of arousalstimuli and thus function as referential calls, or play an important role in causing baboons tothey could be elicited by so many different give acoustically different grunts in the infantstimuli that they provide listeners with only and move contexts. This conclusion, however,general information. In principle, any combina- tells us nothing about the grunts’ potential to acttion of results is possible. The affective and refer- as referential signals that inform nearby listenersential properties of signals are also logically about social or ecological events taking place atdistinct, at least in animal communication, the time. As already noted, move grunts are givenbecause the former depends on mechanisms of in a restricted set of circumstances, when thecall production in the signaler, whereas the latter group is about to initiate, or has alreadydepends on the listener’s ability to extract initiated, a move. As a result, they have theinformation from events in its environment. potential to convey quite specific informationSignalers and recipients, though linked in a com- to listeners. When one baboon hears anothermunicative event, are nonetheless separate and give a move grunt, he or she learns with somedistinct, because the mechanisms that cause a accuracy what is happening at that moment. Bysignaler to vocalize do not in any way constrain comparison, infant grunts are not as tightlya listener’s ability to extract information from linked to a particular type of social interaction.the call. They may be given as the caller approaches a Baboon grunts offer a good example. Rendall mother with infant, in answer to another(2003) used behavioral data to code a social animal’s grunt, or as a reconciliatory signal fol-interaction involving move or infant grunts as lowing aggression. As a result, their meaning ishaving high or low arousal. He then examined less precise. When one baboon hears another’scalls given in these two circumstances and found infant grunt, he or she learns only that the callerthat in each context certain acoustic features or is involved in some sort of friendly interaction,modes of delivery were correlated with apparent but the precise nature of the interaction isarousal. Bouts of grunting given when arousal unknown.was apparently high had more calls, a higher rate In sum, far from being a communicativeof calling, and calls with a higher fundamental system that is either affective or symbolic, vocalfrequency than bouts given when arousal was communication in nonhuman primates (andapparently low. Further analysis revealed signif- many other animals) contains elements oficant variation between contexts in the same both. In their production, monkeys and apesthree acoustic features that varied within con- use a small repertoire of relatively stereotypedtext. By all three measures (call number, call rate, calls, each closely linked to a particular context.and fundamental frequency), infant grunts were This predictable association between call andcorrelated with higher arousal than were move context creates, for listeners, a world in whichgrunts. Infant grunts also exhibited greater pitch there are statistical regularities—regularities thatmodulation and more vocal ‘‘jitter,’’ a measure allow them to ascribe meaning to vocalizationsof vocal instability (Rendall, 2003). In human and to organize their knowledge into a richspeech, variations in pitch, tempo, vocal conceptual structure (Cheney & Seyfarth, 2007,modulation, and jitter are known to provide Chapters 10 and 11).listeners with cues about the speaker’s affect orarousal (e.g., Bachorowski & Owren, 1995; THE EVOLUTION OF FLEXIBLEScherer, 1989). VOCAL PRODUCTION It is, of course, difficult to obtain indepen-dent measures of a caller’s arousal in the field. At some point in our evolutionary history—However, similarities between human and probably after the divergence of the evolutionarynonhuman primates in the mechanisms of lines leading to chimpanzees and bonobos onphonation (Fitch & Hauser, 1995; Fitch et al., the one hand and humans on the other (Enard ¨2002; Schon Ybarra, 1995) support Rendall’s et al., 2002)—our ancestors developed much

92 PRIMATE NEUROETHOLOGYgreater control over the physiology of vocal call combination in chimpanzees may carryproduction. As a result, vocal output became new meaning that goes beyond the meaning ofboth more flexible and considerably more the individual calls themselves, but these raredependent on auditory experience and imitation exceptions meet few of the definitions of(Fitch, 2007; Lieberman, 1991). What selective human syntax. By contrast, there is growingpressures might have given rise to these evidence that both Old World monkeysphysiological changes? (Cheney & Seyfarth, 2007; Engh et al., 2006; Vocal communication in nonhuman pri- Flombaum & Santos, 2005) and apesmates lacks three features that are abundantly (Buttelmann et al., 2007; Hare et al., 2001;present in human language: the ability to gen- Tomasello et al., 2005) may possess rudimentaryerate new words, lexical syntax, and a theory of abilities to attribute motives or knowledge tomind. By the latter we mean the ability of both others, and engage in simple forms of sharedspeakers and listeners to make attributions attention and social referencing.about each other’s beliefs, knowledge, and More theoretically, we suggest that theother mental states (Grice, 1957). These are the evolution of a theory of mind acted as a primesimplest, most basic features that distinguish mover in the evolution of language because,human and nonhuman primate vocal produc- while it is easy to imagine a scenario in which ation, and it is with these traits that speculations rudimentary theory of mind came first andabout the evolution of language must start. At provided the impetus for the evolution of largethe earliest stages of language evolution we need vocabularies and syntax, any alternativenot worry about the more complex properties of sequence of events seems less likely.language that probably came later—properties Consider, for example, the course of wordlike case, tense, subject-verb agreement, open- learning in children. Beginning as early as 9 toand closed-class items, recursion, long-distance 12 months, children exhibit a nascent under-dependency, subordinate clauses, and so on. standing of other individuals’ motives, beliefs, How might the ability to generate new and desires, and this skill forms the basis of awords, lexical syntax, and a theory of mind shared attention system that is integral to earlyhave evolved: simultaneously, in response to word learning (Bloom & Markson, 1998;the same selective pressures, or more serially, Tomasello, 2003). One-year-old children seemin some particular order? We propose that the to understand that words can be mapped ontoevolution of a theory of mind preceded lan- objects and actions. Crucially, this understandingguage, creating the selective pressures that gave is accompanied by a kind of ‘‘social referencing’’rise to the ability to generate new words and in which the child uses other people’s direction oflexical syntax, and to the flexibility in vocal gaze, gestures, and emotions to assign labels toproduction that these two traits would have objects (Baldwin, 1991; reviewed in Fisher &required (Cheney & Seyfarth, 2005, 2007). We Gleitman, 2002; Pinker, 1994). Gaze and atten-make this argument on both empirical and tion also facilitate word learning in dogs andtheoretical grounds. other animals. Children, however, rapidly surpass Empirically, there is no evidence in non- the simpler forms of shared attention and wordhuman primates for anything close to the large learning demonstrated by animals. Long beforevocal repertoire we find even in very young they begin to speak in sentences, young childrenchildren. Similarly, nonhuman primates provide develop implicit notions of objects and events,few examples of lexical syntax. Recent work by actors, actions, and those that are acted upon. ¨Zuberbuhler and colleagues on the alarm calls of As Fisher and Gleitman (2002:462) argue, theseforest monkeys provides intriguing evidence ‘‘conceptual primitives’’ provide children with athat the presence of one call type can ‘‘modify’’ kind ‘‘conceptual vocabulary onto which thethe meaning of another (Arnold & Zuberbuhler,¨ basic linguistic elements (words and structures) ¨2006; Zuberbuhler, 2002), and a study by are mapped.’’ Moreover, in contrast to monkeys,Crockford and Boesch (2003) suggests that a apes, and other animals, 1-year-old children

PRIMATE VOCAL COMMUNICATION 93attempt to share what they know with others any stable or predictable way. Because they(Tomasello & Carpenter, 2007). While animals cannot attribute mental states like ignorance toare concerned with their own goals and knowl- others, and are unaware of the causal relationedge, young children are motivated to make their between behavior and beliefs, monkeys and apesthoughts and knowledge publically available. do not actively seek to explain or elaborate upon The acquisition of a theory of mind thus their thoughts. As a result, they are largely incap-creates a cognitive environment that drives the able of inventing new words or of recognizingacquisition of new words and new grammatical when thoughts should be made explicit.skills. Indeed, the data on children’s acquisition We suggest, then, that long before ourof language suggest that they could not increase ancestors spoke in sentences, they had a lan-their vocabularies or learn grammar as rapidly as guage of thought in which they represented thethey do if they did not have some prior notion of world—and the meaning of call sequences—inother individuals’ mental states (Fisher & terms of actors, actions, and those who are actedGleitman, 2002; Pinker, 1994; Tomasello, 2003). upon. The linguistic revolution occurred when By contrast, it is much more difficult to our ancestors began to express this tacit knowl-imagine how our ancestors could have learned edge and to use their cognitive skills in speakingnew words or grammatical rules if they were as well as listening. The prime mover behind thisunable to attribute mental states to others. The revolution was a theory of mind that had evolvedlack of syntax in nonhuman primate vocaliza- to the point where its possessors did not justtions cannot be traced to an inability to recog- recognize other individuals’ goals, intentions,nize argument structure—to understand that an and even knowledge—as monkeys and apesevent can be described as a sequence in which an already do—but were also motivated to shareagent performs some action on an object. their own goals, intentions, and knowledgeBaboons, for example, clearly distinguish with others. Whatever the selective pressuresbetween a sequence of calls indicating that that prompted this change, it led to a mindSylvia is threatening Hannah, as opposed to that was motivated to make public thoughtsHannah is threatening Sylvia. Nor does the lack and knowledge that had previously remainedof syntax arise because of an inability to mentally private. The evolution of a theory of mindrepresent descriptive verbs, modifiers, or prepo- spurred the evolution of words and grammar.sitions. In captivity, a variety of animals, It also provided the selective pressure for theincluding dolphins (Herman et al., 1993), sea evolution of the physiology adaptations thatlions (Schusterman & Krieger, 1986), and enabled vocal modifiability.African gray parrots (Pepperberg, 1992), can betaught to understand and in some cases even toproduce verbs, modifiers, and prepositions.Even in their natural behavior, nonhuman pri- NOTEmates and other animals certainly seem capable 1. One should, however, treat estimates of the sizeof thinking in simple sentences, but the ability to of a species’ vocal repertoire with caution. Oftenthink in sentences does not motivate them to the best predictors of repertoire size are thespeak in sentences. Their knowledge remains length, creativity, and ingenuity with which alargely private. species has been studied. This may occur in large part because primatesand other animals cannot distinguish betweenwhat they know and others know and cannotrecognize, for example, that an ignorant indivi- REFERENCESdual might need to have an event explained to Arbib, M. (2005). From monkey-like action-them. As a result, although they may mentally recognition to human language: Antag events as argument structures, they fail to evolutionary framework for neurolinguistics.map these tags into a communicative system in Behavioral and Brain Science, 28, 105–167.

94 PRIMATE NEUROETHOLOGY ¨Arnold, K., & Zuberbuhler, K. (2006). Language Egnor, S. E. R., & Hauser, M. D. (2004). A paradox evolution: Compositional semantics in in the evolution of primate vocal learning. primate calls. Nature, 441, 303. Trends in Neuroscience, 27, 649–654.Bachorowski, J. A., & Owren, M. J. (1995). Vocal Egnor, S. E. R., Wickelgren, J., & Hauser, M. D. expression of emotion: Acoustic properties of (2007). Tracking silence: Adjusting vocal speech are associated with emotional intensity production to avoid acoustic interference. and context. Psychological Science, 6, 219–224. Journal of Comparative Physiology Series A,Baldwin, D. (1991). Infants’ contribution to the 193, 477–483. achievement of joint reference. Child Elowson, M., & Snowdon, C. T. (1994). Pygmy Development, 92, 875–890. marmosets, Cebuella pygmaea, modify vocalBickerton, D. (1990). Language and species. structure in response to changed social Chicago: University of Chicago Press. environment. Animal Behaviour, 47, 1267–Bloom, P., & Markson, L. (1998). Capacities 1277. underlying word learning. Trends in Cognitive Enard, W., Przeworski, M., Fisher, S. E., Lai, C. S. L., Science, 2, 67–73. Wiebe, V., Kitano, T., et al. (2002). MolecularButtelmann, D., Carpenter, M., Call, J., & Tomasello, evolution of FOXP2: A gene involved in speech M. (2007). Enculturated apes imitate rationally. and language. Nature, 418, 869–872. Developmental Science, 10, 31–38. Engh, A. E., Hoffmeier, R. R., Cheney, D. L., &Cheney, D. L., & Seyfarth, R. M. (1990). How Seyfarth, R. M. (2006). Who, me? Can monkeys see the world. Chicago: University of baboons infer the target of vocalisations? Chicago Press. Animal Behaviour, 71, 381–387.Cheney, D.L., & Seyfarth, R. M. (1997). Fischer, J., Cheney, D. L., & Seyfarth, R. M. (2000). Reconciliatory grunts by dominant female Development of infant baboon responses to baboons influence victims’ behaviour. Animal female graded variants of barks. Proceedings of Behaviour 54, 409–418. the Royal Society of London Series B, 267,Cheney, D. L., & Seyfarth, R. M. (1998). Why 2317–2321. monkeys don’t have language. In: G. Petersen Fisher, C., & Gleitman, L. R. (2002). Language (Ed.), The Tanner lectures on human values (vol acquisition. In: H. F. Pashler & C. R. Gallistel 19, pp. 175–219). Salt Lake City: University of (Eds.), Handbook of experimental Psychology, vol Utah Press. 3: Learning and motivation (pp. 445–496). NewCheney, D. L., & Seyfarth, R. M. (2005). Constraints York: Stevens Wiley. and preadaptations in the earliest stages of Fischer, J., Hammerschmidt, K., Cheney, D. L., & language evolution. Linguistic Review, 22, Seyfarth, R. M. (2002). Acoustic features of 135–159. male baboon loud calls: Influences of context,Cheney, D. L., & Seyfarth, R. M. (2007). Baboon age, and individuality. Journal of the Acoustical metaphysics: The evolution of a social mind. Society of America, 111, 1465–1474. Chicago: University of Chicago Press. Fischer, J., Hammerschmidt, K., Seyfarth, R. M., &Cheney, D. L., Seyfarth, R. M., & Silk, J. B. (1995). Cheney, D. L. (2001a). Acoustic features of The responses of female baboons to anomalous female chacma baboon barks. Ethology, 107, social interactions: Evidence for causal 33–54. reasoning? Journal of Comparative Psychology, Fischer, J., Metz, M., Cheney, D. L., & Seyfarth, 109, 134–141. R. M. (2001b). Baboon responses toCrockford, C., & Boesch, C. (2003). Context- graded bark variants. Animal Behaviour, 61, specific calls in wild chimpanzees (Pan 925–931. troglodytes verus): Analysis of barks. Animal Fitch, W. T. (2007). The evolution of language: A Behaviour, 66, 115–125. comparative perspective. In: G. Gaskell (Ed.),Crockford, C., Herbinger, L., Vigilant, L., & Oxford handbook of psycholinguistics. Oxford: Boesch, C. (2004). Wild chimpanzees have Oxford University Press. group-specific calls: A case for vocal learning? Fitch, W. T., & Hauser, M. D. (1995). Vocal Ethology, 110, 221–243. production in nonhuman primates: Acoustics,Darwin, C. (1871/1981) The descent of man, and physiology, and functional constraints on selection in relation to sex. Princeton: Princeton ‘‘honest’’ advertisement. American Journal of University Press. Primatology, 37, 191–220.

PRIMATE VOCAL COMMUNICATION 95Fitch, W. T., Neubauer, J., & Herzel, H. (2002). Hauser, M. D. (1988). How infant vervet monkeys Calls out of chaos: The adaptive significance of learn to recognize starling alarm calls: The role nonlinear phenomena in mammalian vocal of experience. Behaviour, 105, 187–201. production. Animal Behaviour, 63, 407–418. Hauser, M. D. (1996). The evolution ofFlombaum, J. L., & Santos, L. R. (2005). Rhesus communication. Cambridge, MA: MIT Press. monkeys attribute perceptions to others. Hauser, M. D., & Wrangham, R. W. (1990). Current Biology, 15, 447–452. Recognition of predator and competitor callsGardenfors, P. (2003). How homo became sapiens: in nonhuman primates and birds: A On the evolution of thinking. Oxford: Oxford preliminary report. Ethology, 86, 116–130. University Press. Herbinger, I. (2003). Inter-group aggression inGemba, H., Kyuhou, S., Matsuzaki, R., & Amino, Y. wild West African chimpanzees (Pan (1999). Cortical field potentials associated with troglodytes verus): Mechanisms and function. audio-initiated vocalization in monkeys. Ph.D. dissertation, University of Leipzig. Neuroscience Letters, 272, 49–52. Herman, L. M., Pack, A. A., & Morrel-Samuels, P.Ghazanfar, A. A., & Logothetis, N. K. (2003). Facial (1993). Representational and conceptual skills expressions linked to monkey calls. Nature, 423, of dolphins. In: H. L. Roitblat, L. M. Herman, & 937–938. P. E. Nachtigall (Eds.), Comparative cognitionGhazanfar, A. A., Maier, J. X., Hoffman, K. L., & and neuroscience (pp. 403–442). Hillsdale, NJ: Logothetis, N. K. (2005). Multisensory Lawrence Erlbaum Associates. integration of dynamic faces and voices in Janik, V. W., & Slater, P. J. B. (1997). Vocal learning rhesus monkey auditory cortex. Journal of in mammals. Advances in the Study of Behavior, Neuroscience, 25, 5004–5012. 26, 59–99.Gil da Costa, R., Braun, A., Lopes, M., Hauser, Jurgens, U., & Ploog, D. (1970). Cerebral M. D., Carson, R. E., Herscovitch, P., et al. representation of vocalization in the squirrel (2004). Toward an evolutionary perspective on monkey. Experimental Brain Research, 10, conceptual representation: Species-specific calls 532–554. activate visual and affective processing systems Kaminski, J., Call, J., & Fischer, J. (2004). Word in the macaque. Proceedings of the National learning in a domestic dog: Evidence for ‘‘fast Academy of Sciences, 101, 17516–17521. mapping.’’ Science, 304, 1682–1683.Goodall, J. (1986). The chimpanzees of Gombe: Kitchen, D. M., Cheney, D. L., & Seyfarth, R. M. Patterns of behavior. Cambridge, MA: Harvard (2003). Female baboons’ responses to male loud University Press. calls. Ethology, 109, 401–412.Gouzoules, S., Gouzoules, H., & Marler, P. (1984). Lieberman, P. (1991). Uniquely human. Rhesus monkey (Macaca mulatta) screams: Cambridge, MA: Harvard University Press. Representational signaling in the recruitment Marler, P., Evans, C. S., & Hauser, M. D. (1992). of agonistic aid. Animal Behaviour, 32, 182–193. Animal signals: Motivational, referential, orGrice, H. P. (1957). Meaning. Philosophical Review, both? In: H. Papousek, U. Jurgens, & 66, 377–388. M. Papousek (Eds.), Nonverbal vocal communi-Gros-Louis, J. (2004). The function of food- cation: Comparative and developmental approaches associated calls in white-faced capuchin (pp. 66–86). Cambridge: Cambridge University monkeys, Cebus capucinus, from the perspective Press. of the signaler. Animal Behaviour, 67, 431–440. McComb, K., & Semple, S. (2005). Coevolution ofHammerschmidt, K., & Fischer, J. (2008). vocal communication and sociality in primates. Constraints in primate vocal production. In: Biology Letters, 1, 381–385. U. Griebel & K. Oller (Eds.), The evolution of Mitani, J. C., & Brandt, K. L. (1994). Social factors communicative creativity: From fixed signals to influence the acoustic variability in the long- contextual flexibility (pp. 93–119). Cambridge, distance calls of male chimpanzees. Ethology, MA: MIT Press. 96, 233–252.Hare, B., Call, J., & Tomasello, M. (2001). Do Mitani, J., & Gros-Louis, J. (1998). Chorusing and chimpanzees know what conspecifics know? call convergence in chimpanzees: Tests of three Animal Behaviour, 61, 139–151. hypotheses. Behaviour, 135, 1041–1064.

96 PRIMATE NEUROETHOLOGYOwings, D. H., & Morton, E. S. (1998). Animal ¨ Schon Ybarra, M. (1995). A comparative approach to vocal communication: A new approach. the nonhuman primate vocal tract: Implications Cambridge: Cambridge University Press. for sound production. In: E. Zimmerman, J. D.Owren, M. J., Dieter, J. A., Seyfarth, R. M., & Newman, & U. Jurgens (Eds.), Current topics in Cheney, D. L. (1993). Vocalizations of rhesus primate vocal communication (pp. 185–198). New and Japanese macaques cross-fostered between York: Plenum Press. species show evidence of only limited Schusterman, R .J., & Krieger, K. (1986). Artificial modification. Developmental Psychobiology, 26, language comprehension and size transposition 389–406. by a California sea lion (Zalophus californianus).Owren, M. J., & Rendall, D. (1997). An affect- Journal of Comparative Psychology, 100, conditioning model of nonhuman primate 348–355. vocal signaling. In: M. D. Beecher, Schusterman, R. J., Reichmuth Kastak, C., & D. H. Owings, & N. S. Thompson (Eds.), Kastak, D. (2002). The cognitive sea lions: Perspectives in ethology (vol 12, pp. 299–346). Meaning and memory in the lab and in New York: Plenum Press. nature. In: M. Bekoff, C. Allen, &Owren, M. J., Seyfarth, R. M., & Cheney, D. L. G. Burghardt (Eds.), The cognitive animal: (1997). The acoustic features of vowel-like Empirical and theoretical perspectives on animal grunt calls in chacma baboons (Papio cognition (pp. 217–228). Cambridge, MA: cynocephalus ursinus): Implications for MIT Press. production processes and functions. Journal of Seyfarth, R. M., & Cheney, D. L. (1986). Vocal the Acoustical Society of America, 101, development in vervet monkeys. Animal 2951–2963. Behaviour, 34, 1640–1658.Pepperberg, I. M. (1992). Proficient performance Seyfarth, R. M., & Cheney, D. L. (1990). The of a conjunctive, recursive task by an African assessment by vervet monkeys of their own gray parrot (Psittacus erithacus). Journal of and another species’ alarm calls. Animal Comparative Psychology, 106, 295–305. Behaviour, 40, 754–764.Pierce, J. (1985). A review of attempts to condition Seyfarth, R. M., & Cheney, D. L. (1997). Some operantly alloprimate vocalizations. Primates, general features of vocal development in non- 26, 202–213. human primates. In: M. Husberger &Pinker, S. (1994). The language instinct. New York: C. T. Snowdon (Eds.), Social influences on William Morrow and Sons. vocal development (pp. 249–273). Cambridge:Ploog, D. (1981). Neurobiology of primate audio- Cambridge University Press. visual behavior. Brain Research Review, 3, Seyfarth, R. M., & Cheney, D. L. (2003). Signalers 35–61. and receivers in animal communication.Poole, J., Tyack, P., Stoeger-Horwarth, A., & Annual Review of Psychology, 54, 145–173. Watwood, S. (2005). Elephants are capable of Seyfarth, R. M., Cheney, D. L., & Marler, P. (1980). vocal learning. Nature, 434, 455–456. Vervet monkey alarm calls: SemanticRendall, D. (2003). Acoustic correlates of caller communication in a free-ranging primate. identity and affect intensity in the vowel-like Animal Behaviour, 28, 1070–1094. grunt vocalizations of baboons. Journal of the Silk, J. B., Altmann, J., & Alberts, S. C. (2006a). Acoustical Society of America, 113, 3390–3402. Social relationships among adult femaleRendall, D., Seyfarth, R. M., Cheney, D. L., & baboons (Papio cynocephalus). I. Variation in Owren, M. J. (1999). The meaning and the strength of social bonds. Behavioral Ecology function of grunt variants in baboons. Animal and Sociobiology, 61, 183–195. Behaviour, 57, 583–592. Silk, J. B., Altmann, J., & Alberts, S. C. (2006b).Savage-Rumbaugh, E. S. (1986). Ape language: Social relationships among adult female From conditioned response to symbol. baboons (Papio cynocephalus). II: Variation in New York: Columbia University Press. the quality and stability of social bonds.Scherer, K. R. (1989). Vocal correlates of emotion. Behavioral Ecology and Sociobiology, 61, In: H. Wagner & A. Manstead (Eds.), Handbook 197–204. of psychophysiology: Emotion and social behavior Silk, J. B., Seyfarth, R. M., & Cheney, D. L. (1999). (pp. 167–195). New York: John Wiley and Sons. The structure of social relationships among

PRIMATE VOCAL COMMUNICATION 97 female baboons in the Moremi Reserve, Tomasello, M. (2003). Constructing a language: A Botswana. Behaviour, 136, 679–703. usage-based theory of language acquisition. ¨Slocombe, K., & Zuberbuhler, K. (2005). Agonistic Cambridge, MA: Harvard University Press. screams in wild chimpanzees (Pan troglodytes Tomasello, M., & Carpenter, M. (2007). Shared schweinfurthii) vary as a function of social role. intentionality. Developmental Science, 10, Journal of Comparative Psychology, 119, 67–77. 121–125. ¨Slocombe, K., & Zuberbuhler, K. (2007). Tomasello, M., Carpenter, M., Call, J., Behne, T., & Chimpanzees modify recruitment screams as a Moll, H. (2005). Understanding and sharing function of audience composition. Proceedings of intentions: The origins of cultural cognition. the National Academy of Sciences, 104, 228–233. Behavioral Brain Science, 28, 675–691.Smith, H. J., Newman, J. D., & Symmes, D. (1982). Washburn, S. L. (1982). Language and the fossil Vocal concomitants of affiliative behavior record. Anthropology UCLA, 7, 231–238. in squirrel monkeys. In: C. T. Snowdon, Wittig, R. M., Crockford, C., Seyfarth, R. M., & C. H. Brown, & M. Petersen (Eds.), Primate Cheney, D. L. (2007). Vocal alliances in communication (pp. 30–49). Cambridge: chacma baboons, Papio hamadryas ursinus. Cambridge University Press. Behavioral Ecology and Sociobiology, 61,Steiper, M. E., Young, N. M., & Sukarna, T. Y. 899–909. (2004). Genomic data support the hominoid ¨ Zuberbuhler, K. (2000). Referential labeling in slowdown and an Early Oligocene estimate for Diana monkeys. Animal Behaviour, 59, 917–927. the hominoid–cercopithecoid divergence. ¨ Zuberbuhler, K. (2001). Predator-specific alarm Proceedings of the National Academy of calls in Campbell’s guenons. Behavioral Ecology Sciences, 101, 17021–17026. and Sociobiology, 50, 414–422.Terrace, H. S. (1979). Nim. New York: Knopf. ¨ Zuberbuhler, K. (2002). A syntactic rule in forestTerrace, H. S. (1983). Nonhuman intentional monkey communication. Animal Behaviour, 63, systems. Behavioral and Brain Science, 6, 378–379. 293–299.

CHAPTER 6Rational Decision Making in Primates: The Boundedand the EcologicalJeffrey R. StevensA young female rhesus macaque steals furtive a new home? In all of these cases, primates mustglances at the male off to her right. He just trade off the costs and benefits associated witharrived to the territory and therefore immedi- uncertain and delayed decision outcomes. Theately piques her interest. The alpha male, how- outcomes of these choices influence survival andever, sits a few meters off, basking in the sun. reproduction, and natural selection should favorBeing in estrus, the young female faces a choice: those individuals whose choices lead to the pro-solicit a mating from the alpha male or follow pagation of their genes.the unfamiliar male into the brush to sneak a The vast majority of economic analyses ofmating with him. Mating with the alpha male decision making define good or ‘‘rational’’ deci-almost guarantees ‘‘good genes’’ for her off- sions as those consistent with a set of mathema-spring. But something pushes her toward the tical principles. Yet, this ignores the evolutionaryunfamiliar male. Mating with him reduces the pressures on decision making for the sake ofprobability of inbreeding and adds a bit of mathematical elegance (Kacelnik, 2006; Stevens,genetic diversity to her offspring. Additionally, 2008). Meanwhile, the standard psychologicalmating with the new male could act as an invest- view of decision making seeks to empiricallyment in the future: The current alpha male is undermine the economic theory but cannotgetting old, and befriending a prospective alpha offer an alternative explanatory theory. Here, Imale could yield future benefits. In addition, emphasize an evolutionarily informed frameworkspreading the possibility of paternity may for studying decision making: the bounded andsecure protection for the offspring. Yet, this ecological rationality approaches (Gigerenzermating also involves risks. Males often vocalize et al., 1999). Though these approaches have tra-while mating, which attracts the attention of ditionally focused on human decision making,other males. If the unfamiliar male vocalizes, they are just as relevant for other species,the alpha male may attack the female. Should including other primates. To illustrate the rele-she take the safe option or risk punishment for vance of bounded and ecological rationality to thepossible future payoffs? study of primate decision making, I begin by Primates constantly face decisions that influ- introducing various visions of rationality foundence their survival and reproduction. Continue in the economic and psychological study of deci-foraging in this tree or move on to another? sion making. I then explore how primate studiesExpose oneself to a hidden predator by straying inform three aspects of decision making: utility,from the group or enjoy the safety of having uncertainty, and time. Together, these aspects willother potential victims nearby? Defend one’s guide our understanding of the evolutionary ori-territory from invaders or abandon it and seek gins of primate and human rationality.98

RATIONAL DECISION MAKING IN PRIMATES 99VISIONS OF RATIONALITY preference exists. If an individual prefers bananas over apples and apples over oranges,Rationality means different things to different then he or she must prefer bananas over orangespeople. Kacelnik (2006) proposed that ration- to maintain transitivity. Second, when transitive,ality refers to decisions that are either (1) con- choices are also independent from irrelevantsistent with expected utility maximization for alternatives, meaning the relative preferenceeconomists and psychologists, (2) consistent between options should not be affected by thewithin the self for philosophers, and (3) consis- presence or absence of other options. If bananastent with fitness enhancement for biologists. Of are preferred to apples, the addition of water-particular relevance here are the economic, psy- melons to the choice set should not affect thechological, and biological views. A review of banana/apple preference. Finally, choices mustthese visions of rationality will frame the ques- be invariant, meaning that option A is preferredtion, ‘‘Are primates rational decision makers?’’ to B regardless of presentation format. If endowed with a banana and an apple and askedRational Choice to give up one, preferences should be the same asWhich would you prefer: receiving two bananas if asked to choose freely between a banana andwith certainty or receiving either one or three apple. Given these and a few other principles,bananas with equal probability? Depending on one can show that preferences follow expectedwhether you like bananas and your level of utility calculations.hunger, this may be tricky to answer. Thesequestions of decisions under risk mirror funda- Irrational Choice?mental choices that we and other animals fre-quently face. Very little in life is certain, so all Not long after von Neumann and Morgensternorganisms choose between options without (1947) published the principles of expected uti-knowing the exact consequences. lity theory, cracks began appearing in the math- Economists have approached the question of ematically elegant framework when data showeduncertainty by developing expected utility theory violations of expected utility predictions(reviewed in Wu et al., 2004). In expected utility (Allais, 1953; Ellsberg, 1961). Soon afterward,theory, three features characterize all options: Kahneman and Tversky started a cottagemagnitude, utility, and probability. Magnitude industry of demonstrating violations of the the-(x) refers to the amount of the benefit (or cost) oretical predictions (Kahneman & Tversky,associated with the option. Utility (u) is the 2000; Kahneman et al., 1982; Tversky &mapping of magnitude onto some measure of Kahneman, 1981). In both experimental settingssatisfaction or ‘‘goodness.’’1 Finally, probability and in real-world decisions, laypeople and(p) is the chance that the outcome occurs. Thus, experts made ‘‘irrational’’ choices.2 Subjectsif one faces risky options, the expected utility is showed intransitivity, irrelevant alternativesEU = p Á u(x), where utility is some function of changed preferences, and the framing of deci-magnitude. Von Neumann and Morgenstern sion questions greatly influenced preferences.(1947) formalized expected utility theory to Economists minimized the problem by callingshow that following specific mathematical prin- these findings ‘‘anomalies,’’ whereas psycholo-ciples maximizes expected utility. Therefore, gists emphasized their robustness and labeledexpected utility acts as a normative standard them ‘‘biases.’’ To psychologists, expected utilityfor what decision makers should maximize theory was deeply flawed because it rested purelywhen making risky choices. on mathematically derived principles and not Expected utility maximization assumes con- what we know about human behavior.sistent choice, which requires a number of prin- Kahneman and Tversky (1979) injected a bitciples to hold (reviewed in Luce & Raiffa, 1957; of psychological realism into decision theoryRieskamp et al., 2006b). First, choices must be when they proposed prospect theory. Instead oftransitive, meaning that a fixed order of using a utility function characterized over total

100 PRIMATE NEUROETHOLOGYwealth, prospect theory uses a value function time to decide, full knowledge of the problem,v(x) relative to a reference point. So with 100 and unlimited computational resources to findbananas in the bank, a decision maker would an optimal solution to a decision problem. Thisview a choice between gaining one and losing vision of unbounded rationality contraststhree bananas as a choice between 101 and 97 sharply with what we know about human cogni-bananas from the expected utility perspective tion and decision making, so Simon proposedbut a choice between gaining one and losing the study of bounded rationality—the explora-three from the prospect theory account. In addition of decision making given realistic assump-tion to altering the reference point of the value tions about cognitive abilities. Real-worldfunction, prospect theory integrated data about decision makers lack knowledge and cannot useprobability perception into the equation by optimization processes to make decisions. Thus,adding a decision weight function to the prob- much previous research has ignored cognitiveability. Thus, instead of having a linear relation- processes at work in decision making (but seeship between the objective probability and the Payne et al., 1993). The bounded rationalityperceived probability, prospect theory assumes approach calls for realistic models of the deci-that people tend to overestimate low probabil- sion process based on what we know about cog-ities and underestimate high probabilities. The nition rather than on a set of mathematicalprobability, then, is weighted by the function principles. Knowing the underlying process canp(p). Thus, prospect theory predicts that prefer- help us better understand the decisions. Yet,ences depend on which option has the highest Simon emphasized that studying only the mindvalue V = p(p) Á v(x). As a descriptive theory, it gives you but half of the picture.nicely fits people’s preferences (Kahneman & To fully understand decision making, weTversky, 2000; Wu et al., 2004). However, it must embed the mind in the environment.does not explain why we have reference points Gigerenzer and colleagues (1999) have termedor nonlinear value functions—it takes these as this ecological rationality—the match between agivens and describes how they influence decision mechanism and the environment. Thedecisions. unbounded rationality approach assumes that expected utility works in all decision-making situations—it applies universally. EcologicalBounded and Ecological Rationality rationality, however, appeals to the evolutionaryExpected utility theory is mathematically elegant idea that adaptations match the environment inbut fails to account for many of the data. which they evolved. Therefore, decisionProspect theory fairs much better descriptively mechanisms should not be universal andbut lacks explanatory power. Parameters in the domain general but specifically tailored to themodels are fit to the data, with no a priori pre- environment in which they operate (Barkowdictions about parameter values. Prospect et al., 1992). In fact, ecological rationality sug-theory therefore offers a slight modification of gests that we do not possess a single, complexexpected utility theory by patching a few of the decision-making mechanism used in all con-holes that data have poked into the theory. But texts. Rather, we have an ‘‘adaptive toolbox’’ ofboth theories face a more fundamental problem. mechanisms (including simple heuristics orNamely, neither of these theories adequately rules of thumb) that, when used in the appro-addresses two crucial components of decision priate environment, perform quite wellmaking: the structure of the mind and the struc- (Gigerenzer et al., 1999). This perspective, then,ture of the decision-making environment. offers an explanation for the anomalies and Early in the study of decision making, Simon biases seen by the experimental economists and(1955, 1956) highlighted not only the study of psychologists. Rather than being evidence forthe mind but also the fit between the mind and flawed thinking and irrationality, we are simplythe environment. He criticized the unrealistic putting these decision mechanisms in anassumption that decision makers have infinite unfamiliar and artificial environment—the

RATIONAL DECISION MAKING IN PRIMATES 101experimental laboratory. In general, however, the wrong standard. Do other primates violateour decision mechanisms serve us quite well by these standards, suggesting deep evolutionaryexploiting critical aspects of the environment. roots, or do only humans show these violations? Both bounded and ecological rationality Transitive preferences4 are not well studied inoffer appealing alternatives to the standard primates. Though other species seem to showunbounded approach because they rest on rea- intransitivity (Shafir, 1994; Waite, 2001b), fewlistic evolutionary principles instead of mathe- data exist for primates, so we will not considermatical formalizations. Comparative analyses transitivity here.5provide a unique method for testing questions The principle of independence of irrelevantof bounded and ecological rationality because alternatives implies that previously availablewe have great variability across species in their options should not influence the current prefer-ecological environments. Primates offer an ences (Simonson & Tversky, 1992). Waiteideal group of species for these investigations (2001a), however, found that the backgroundbecause their phylogenetic proximity allows us context does matter for gray jays (Perisoreusto test interesting hypotheses about the evolu- canadensis). When required to pay a lower rela-tion of human decision making. With this fra- tive cost for food in previous choices, the birdsmework in hand, we can now review the reduced their preference for that option laterbounded and ecological rationality of primate when required to pay a higher relative cost.decisions. Note that here I focus on what has Tinklepaugh (1928) found a similar result intraditionally been termed ‘‘individual decision rhesus macaques (Macaca mulatta) and long-making.’’ Though it is likely impossible to com- tailed macaques (Macaca fascicularis). Thesepletely extract an individual from the social monkeys observed an experimenter place aenvironment (Stevens & King, in press), for banana under one of two cups. The monkeysthe purposes of this review I will put aside the then immediately chose the correct cup andexciting and complicated world of social decision received the banana. However, in some trials,making in primates and refer the reader to the experimenter substituted a piece of lettuceMaestripieri (Chapter 19) or Rosati, Santos, and under the cup, unbeknownst to the monkeys.Hare (Chapter 7). Instead, I will focus on three When the monkeys lifted the cups and foundimportant components of individual decision the lettuce, they rejected the less-preferred foodmaking3: utility, uncertainty, and time. item. Though outside of this context the monkeys readily consumed lettuce, they refused to eat it when expecting a banana. Thus, prefer-UTILITY AND PREFERENCE ences are not fixed but depend on previousUtility refers to the mapping of the magnitude of options.a benefit or cost onto some measure of ‘‘good- Tinklepaugh’s data suggest that the monkeysness’’ or goal achievement (Baron, 2000), and a do not have absolute preferences. If they did,utility function describes this mapping. Utility they would always consume the lettuce becauseitself is difficult to assess, so typically we measure it is always better than nothing. However, thechoices to infer preferences. For expected utility preference for consuming lettuce is relative totheory to work, preferences must follow the the previous availability of the highly preferredprinciples mentioned previously: transitivity, banana. The preference is relative to an expecta-independence of irrelevant alternatives, and tion of other options. There are good, adaptiveinvariance. It is well established that humans reasons for avoiding absolute preferences. Forviolate these principles, deviating from the nor- instance, it is well known in foraging theorymative standard (Kahneman & Tversky, 2000). that optimal choices depend on the backgroundThe question remains, ‘‘Is this the appropriate environment. If the environment is rich, animalsstandard?’’ Examining choices in other species should ‘‘skim the cream’’ and choose to investcan address this question, because if other spe- little time in extracting food from patches; if,cies also show the biases, then we likely are using however, the environment is poor, they should

102 PRIMATE NEUROETHOLOGYmore thoroughly exploit the patches (Stephens Utility Building Blocks& Krebs, 1986). Houston (1997) argued that Simon’s (1955, 1956) vision of bounded ration-since the current choice is incorporated into ality emphasized realistic assumptions about thethe estimate of the background environment cognitive abilities of decision makers. Applyingfor the next choice point, preferences can this perspective suggests that we need to con-change for the same set of options depending sider both the cognitive building blocks oron the previous background environment. Thus, evolved capacities needed for exhibiting prefer-relative preferences can result from an ecologi- ences and the limits placed on these capacities.cally rational mechanism of adaptive decisionmaking. Magnitude Perception The classical economic approach to ration-ality also assumes that decision makers have A critical component of exhibiting preferences isstrictly ordered preferences that are invariant the ability to perceive differences in magnitudeto extraneous characteristics of the choice situa- between options. Preferring two bananas to onetion. If an agent prefers A to B, it should always requires discriminating between the amountsprefer A, regardless of whether one is buying or two and one. There are many mechanisms usedselling A, or whether other choices are made to quantify objects in the world, and many stu-before the A/B choice, etc. A common violation dies have explored these mechanisms in pri-of invariance found in behavioral economics is mates (reviewed in Brannon, 2005a,b; Hauserthe endowment effect. This phenomenon & Spelke, 2004). Brannon et al. (Chapter 8)occurs when decision makers have a higher provide a useful overview of primate quantifica-preference for an object when they own it tion in this volume, but it is worthwhile to(Kahneman et al., 1990). In humans, this is briefly describe two of these mechanisms. Thetypically demonstrated by showing that subjects first mechanism discriminates between quanti-require a higher price to sell an object they ties only approximately via the analog magni-possess than to buy the same object. Brosnan tude system. Importantly, the discriminationsand colleagues (2007) tested similar effects in follow Weber’s law: Variance in the representa-chimpanzees (Pan troglodytes) by offering them tion increases with magnitude (Gallistel, 1990).a choice between two different food items and This results in the ratio between quantitiesrecording their preferences. The experimenters rather than the absolute magnitude driving thethen endowed the chimpanzees with one of the discrimination. The approximate numberitems and measured their willingness to trade system yields a limit to the precision withfor the other item. The choice preferences and which individuals can discriminate magnitudes,trading preferences did not match, suggesting with larger magnitudes being more difficult thanan endowment effect. Similar results have smaller ones (Fig. 6.1). In addition to thebeen found for brown capuchin monkeys approximate system, both humans and other(Cebus apella—Lakshminaryanan et al., 2008). primates seem to have a precise system thatThough anomalous to economists, an evolu- tracks individual objects. With this system, indi-tionary perspective provides an explanation for viduals can enumerate discrete quantities butthe endowment effect. The question of why only up to a maximum of three or four objectsowners fight harder than intruders to maintain (Hauser et al., 2000). Therefore, the precisiona resource has a long history in behavioral that primates can exhibit in their preferencesecology (Hammerstein, 1981; Krebs & Davies, depends on the magnitude of the amount:1993; Maynard Smith & Parker, 1976). They can precisely choose between smallOwnership has its privileges, including addi- amounts and approximately choose betweentional knowledge about a resource. Even larger amounts.without direct benefits, as an arbitrary rule, The standard rationality approach wouldrespecting ownership can avoid costly conflicts. assume that if an organism has the more preciseThus, owning a resource can increase its value. system, it should use it when making quantity

RATIONAL DECISION MAKING IN PRIMATES 103 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Figure 6.1 Weber’s law states that the variance around estimates of quantity increases with magnitude.Smaller quantities therefore are easier to discriminate than larger quantities. For instance, there may be littlevariance for estimations of three and five objects, so discriminating between them is easy. However, thevariance dramatically increases for 11 and 13 objects, and estimates greatly overlap for these durations,making them difficult to distinguish. judgments. However, Stevens and colleagues juice rewards against social information in (2007) showed that cotton-top tamarins rhesus macaques. Male monkeys chose between (Saguinus oedipus) used the approximate receiving juice and viewing images of either system in a foraging task, even though this spe- higher-ranking males’ faces, lower-ranking cies can use the precise system (Hauser et al., males’ faces, or female perinea (sexual areas). 2003; Uller et al., 2001). The tamarins used the Interestingly, the valuation functions showed simpler, approximate system as a default that the monkeys would forego juice to view mechanism unless the task demanded the more high-ranking males and female perinea but had precise system. Thus, different aspects of the to be ‘‘paid’’ in juice to view low-ranking males. decision environment trigger different mechan- Comparing these kinds of qualitatively different isms of discrimination. rewards is a critical capacity for decision making, although we do not have good cognitive models Valuation for how these tradeoffs occur. Options differ not only in quantity but also in Inhibitory Control quality. In many cases, decision makers face choices between qualitatively different reward Organisms must not only discriminate the mag- types, both within and between reward domains. nitudes of benefits to establish a preference but To choose between different types of food or also must favor the larger (positive) outcome. even between different types of reward (food, When motivated, this is not a problem for pri- water, sex, social contact), animals must have a mates (except for very large rewards; see mechanism to evaluate the utility of these Silberberg et al., 1998). In fact, primates have a reward types; that is, they must have a valuation very difficult time going against this preference. mechanism that converts different reward types In a task in which chimpanzees had to point to into a common currency. Padoa-Schioppa and the smaller of two rewards to receive the larger colleagues (2006) explored how capuchin mon- reward, they failed miserably (Boysen & keys traded off different amounts of various food Berntson, 1995). In addition to the chimpanzees, types to generate a valuation function (e.g., one bonobos (Pan paniscus), gorillas (Gorilla gor- piece of apple may be worth three pieces of illa), orangutans (Pongo pygmaeus), rhesus and carrot). Deaner and colleagues (2005) pitted Japanese macaques (Macaca fuscata), squirrel

104 PRIMATE NEUROETHOLOGYmonkeys (Saimiri sciureus), tamarins, and which payoff will be realized). Though we andlemurs (Eulemur fulvus and E. macaco) all fail, other animals frequently face uncertainty, this isat least initially, on this task (Anderson et al., difficult to study in the laboratory, so we will2000; Genty et al., 2004; Kralik et al., 2002; focus on risk.Murray et al., 2005; Silberberg & Fujita, 1996;Vlamings et al., 2006). Clearly, an unboundedlyrational agent would adapt quickly to the con- Risky Gamblestingencies of this task, but the preference for a Much of the work undermining classicallarge reward is so powerful that primates cannot expected utility theory involved asking humaninhibit their propensity to choose this. Of subjects about their preferences in risky gambles.course, evolutionarily, it makes sense to Rather than the banana examples suggested pre-employ the simple heuristic ‘‘choose the viously, subjects chose between risky rewardlarger.’’ When in an organism’s ecology would amounts. Would you prefer a 50% chance ofthey opt for a small reward when a larger is receiving $100 (and a 50% chance of receivingpresent? This must occur only rarely. nothing) or a 100% chance of receiving $50? Though these two options have equal expected values, most people have a strong preference forUNCERTAINTY AND RISK the sure thing—they avoid risk. Risk-averse pre-In a letter to Jean-Baptiste Leroy, Benjamin ferences arise with nonlinear utility functions,Franklin stated that ‘‘in this world nothing can specifically when utility increases at a slowerbe said to be certain, except death and taxes’’ (13 rate than the magnitude of the benefitNovember 1789). That leaves a lot of uncertainty (Fig. 6.2). In general, this seems to be the casein the world. As agents navigating in this world, for human risk preferences for intermediate towe must deal with this uncertainty in an adaptive large gains (Tversky & Kahneman, 1992).manner. Knight (1921) posited a useful distinc- Interestingly, nonhuman animals seem totion between uncertainty (not knowing the dis- show a similar pattern. Animals typically aretribution of possible payoffs) and risk (knowing tested by repeatedly experiencing choices similarthe distribution of payoffs but not knowing to the banana examples mentioned previously. Utility Magnitude Figure 6.2 If utility is a linear function of magnitude (red line), then decision makers should be neutral to risk; they should be indifferent between a guaranteed banana and a 50/50 chance of no bananas or two bananas. If utility increases at a decelerating rate with magnitude (blue line), increments of utility are less valuable at larger magnitudes than they are at smaller magnitudes. Satiation offers an important biological example of diminishing utility because limited gut capacity constrains the utility of excess amounts of food. The additional utility of receiving three over two bananas is high, but the additional utility of receiving 103 over 102 is not as great, primarily because no one can consume 102 bananas. Diminishing utility implies risk aversion because the additional gain of the risky option is valued less than the loss.

RATIONAL DECISION MAKING IN PRIMATES 105In a review of risk sensitivity in over 25 species, vegetation, an abundant and reliable foodKacelnik and Bateson (1996) found that most source, and chimpanzees rely more on fruit, aspecies were either risk averse or risk neutral. more temporally and spatially variable foodWork on primates, however, has provided a source (Wrangham & Peterson, 1996).more mixed result. Early tests of risk sensitivity Moreover, chimpanzees face risks when theyin rhesus monkeys showed risk aversion (Behar, hunt monkeys and other small mammals;1961). Yet, more recent studies have shown a bonobos rarely hunt. Interestingly, wild chim-preference for risky rewards in these macaques panzees engage in this risky activity more often(Hayden & Platt, 2007; McCoy & Platt, 2005). when fruit is abundant rather than scarce, aMeanwhile, cotton-top tamarins and bonobos direct contrast to the energy budget hypothesis(Pan paniscus) seem to avoid risk, while for risk-seeking behavior (Gilby & Wrangham,common marmosets (Callithrix jacchus) ignore 2007). Given the generally higher level of riskyrisk and chimpanzees prefer risk (Heilbronner choice in chimpanzees, Heilbronner and collea-et al., 2008, unpublished data). Why does such gues predicted that this would select for risk-variation exist? taking decision mechanisms. As predicted, chimpanzees preferred the risky choice in aVariation in Risk Preferences laboratory experiment, whereas bonobos pre-One of the first hypotheses proposed to account ferred the safe option. Therefore, to exploitfor differences in risk sensitivity was the ‘‘energy risky options in their natural environment, nat-budget rule’’ (Caraco et al., 1980; Stephens, ural selection has likely endowed chimpanzees1981). This rule suggests that hungry individuals with ecologically rational decision mechanisms,should prefer risks because the safe option will yielding preferences for risky outcomes even innot allow them to survive. Though this seems to captive laboratory situations.work in some situations, there is no evidence for The Framing of Riskthis rule in primates. Hayden and Platt (2007)tested an alternative idea proposed by Rachlin Though risk aversion and risk-seeking prefer-and colleagues (1986). Instead of preferring ences do not pose a great challenge to expectedrisky options because of hunger, animals may utility theory, framing effects do challenge theprefer risky options when they require low theory. Prospect theory highlights two types ofcosts—specifically, when another choice will framing effects: reference dependence andarise soon. Repeatedly choosing the risky option loss aversion (Tversky & Kahneman, 1991).guarantees receiving the large payoff at some Reference dependence refers to viewing choicespoint. With short time delays between choices, as gains or losses relative to a reference pointwaiting a few more seconds for this jackpot is not rather than as absolute increases or decreases inthat costly. Rhesus macaques seem to follow this utility. Thus, a set of outcomes could result inrule. The macaques preferred the risky option the exact same levels of wealth but be framed asmore when facing shorter delays between choices a gain or loss. In the classic Asian disease pro-than with larger delays (Hayden & Platt, 2007). blem, a medical treatment has a particular The ecological rationality approach may also effectiveness in combating a disease (Tverskyaccount for some of the patterns of risk prefer- & Kahneman, 1981). However, when the out-ences seen in primates. In particular, when spe- come of a treatment is framed as number ofcies experience risk in their natural ecology, they people saved (a gain), subjects prefer the riskymay have decision rules that bias them toward option more than when framed as number ofrisky options. With this hypothesis in mind, people that die (a loss). Thus, framing the exactHeilbronner and colleagues (2008) predicted a same outcome as either saving or losing livesspecies difference in risk preferences between greatly influences risk preferences. Yet, thischimpanzees and bonobos. Although their diets reference dependence is not symmetric. Peopleoverlap quite a bit in their natural habitat, will try to avoid losses more than they will try tobonobos feed primarily on terrestrial herbaceous obtain gains—a phenomenon termed loss

106 PRIMATE NEUROETHOLOGYaversion (Tversky & Kahneman, 1981). We descriptions of probabilities (e.g., a 50%have already shown that, in risky gambles, chance) and/or on visual displays (e.g., a piepeople typically avoid risk over gains. When chart with half of the circle colored in). Theseexperiencing a loss (a sure loss of $50 or a techniques allow fairly accurate discrimina-50% chance of losing $100), however, people tions between probabilities. In the animal riskprefer risks to avoid the guaranteed loss. literature (and in some human experiments; see Hundreds of studies have documented the Hertwig et al., 2004), the subjects repeatedlyeffects of reference dependence and loss aversion experience the outcomes to gauge the level ofin humans (Kahneman & Tversky, 2000). If this risk. Therefore, to choose between gambles,is truly a bias, then we might expect to find it animals must discriminate probabilities basedonly in humans. But if framing effects offer an on experienced outcomes. Though few studiesadaptive, ecologically rational advantage, other have explicitly tested this, Weber’s law mayanimal species may exhibit them. Though first describe probability discrimination. Bothdemonstrated in European starlings (Marsh & Herrnstein and Loveland (1975) and BaileyKacelnik, 2002), Chen and colleagues (2006) and Mazur (1990) showed that pigeons’ choicesexplored reference dependence and loss aversion for the less risky option increased as the ratioin capuchin monkeys. To test reference depen- between the small to large probability decreaseddence, the monkeys chose between two risky (probabilities became less similar). In addition,options. In one option, subjects saw one food Bailey and Mazur and Krebs et al. (1978)reward and either received one or two rewards showed that pigeons and starlings (respec-with equal probability. In the other option, they tively) took more time to stabilize their prefer-saw two rewards and received either one or two ences when the ratio between probabilitiesrewards with equal probability. Though iden- increased, further suggesting difficulty in dis-tical in outcome, the reference point (number criminating similar probabilities. Thus, likeof initial rewards) varied, resulting in a perceived numerical magnitude, probability discrimina-gain or loss. In this condition, subjects strongly tion likely follows Weber’s law: Individuals canpreferred the gain option, showing clear refer- discriminate a 10% from a 20% chance betterence dependence. Another experiment tested than an 80% from a 90% chance. This hasloss aversion. Here, one option consisted of important implications for how animals dealseeing and then receiving a single reward, and with risk. When facing unlikely events, animalsthe other option consisted of seeing two rewards may discriminate probabilities well and there-but always receiving one. Again, the monkeys fore respond appropriately to risk. For morefaced identical outcomes—a guaranteed one likely events, however, animals may ignore thereward—but receiving that one reward could probabilities and simply focus on the payoffs tohave been neutral or perceived as a loss. Again, determine choice.the monkeys avoided the loss option, revealingthe precursors to loss aversion in nonhuman TIMEprimates. Thus, we share framing effects withother primates, suggesting deep evolutionary All decisions have a temporal component, fromroots for this phenomenon. choosing to search for predators instead of searching for prey to delaying reproduction until the next breeding season. Delayed payoffsUncertainty Building Blocks often have both benefits and costs (Stevens &Which evolved capacities does an organism Stephens, 2009). They may be beneficial whenneed to cope with uncertainty and risk? When investing time in obtaining resources allows forgiven a choice between risky gambles, a deci- the extraction of more resources. For instance,sion maker must compare the probabilities of the more time chimpanzees spend fishing foreach outcome. In the human risk literature, termites, the more termites they will extract.subjects typically choose based on written Yet, delayed rewards often come with a cost.

RATIONAL DECISION MAKING IN PRIMATES 107 Rate maximization Reward value Exponential Hyperbolic Time delayFigure 6.3 Models of intertemporal choice differ in their predictions about how the value of a rewarddecreases with the time delay to receiving the reward. Exponential discounting (red line) predicts a constantrate of decrease over time. Hyperbolic discounting (green line) predicts a decreasing rate of decrease overtime, such that decision makers exhibit high discount rates at short-term delays but lower discount rates atlonger delays. Rate maximization (blue line) predicts similar patterns as hyperbolic, albeit with strangebehavior at very small time delays. The advantages of rate maximization models are that they includerepeated choices and have biologically relevant parameters.Temporal Discounting V = A Á e-lt. Because the value of a reward decays exponentially with time, this is called the expo-When facing options with smaller, sooner payoffs nential model of discounting (Fig. 6.3).and larger, later payoffs, animals must make an Though intuitively appealing, the data do notintertemporal choice; that is, they must trade off support the exponential model. Humans,the magnitude of rewards with the delay to pigeons, and rats violate predictions of thisreceiving them (reviewed in Read, 2004). In model in self-control experiments (Fig. 6.4) bysome cases, ignoring the temporal component choosing between a smaller, immediate rewardand choosing based on magnitude is best, but in and a larger, delayed reward (Ainslie &other cases, a long delay may prove too costly. Herrnstein, 1981; Frederick et al., 2002; Mazur,How should animals deal with this tradeoff? They 1987; Richards et al., 1997). In fact, whenmay discount or devalue delayed rewards because choosing between immediately receiving twothe future is uncertain. The risk of not collecting a pieces of food and waiting for six, rats andreward grows with delay because some event may pigeons only wait a few seconds for three timesinterrupt its collection. For instance, a predator as much food. Animals would have to face extra-may interrupt an extended courtship or a bank ordinarily (and unrealistically) high interrup-may collapse before an investment matures. tion rates for discounting by interruptions toEconomists have modified the expected utility account for this level of impulsivity.models to create a discounted utility model of Psychologists proposed an alternative modeldelayed benefits (Samuelson, 1937). This model that captures the data much better: the hyperbolicreplaces p from the expected utility model with a discounting model (Ainslie, 1975; Mazur, 1987).discounting function that includes a constant rate In the hyperbolic model,6 V = A / (1 + kt),of interruption l per unit time. Thus, for a where k represents a fitted parameter thatreward amount A delayed for t time units, describes the steepness of discounting. RatherDU = e-lt Á u(A). Again, utility is difficult to than predict a constant rate of discounting overassess, so most versions of this model drop utility time, this model predicts a decelerating rate overand just discount the absolute reward amount:

108 PRIMATE NEUROETHOLOGY As ts Figure 6.4 In self-control experiments, animals face a choice between a small reward (AS) after a short Intertrial interval delay (tS) and a large reward (AL) after a long delay (tL). The animals wait for an intertrial interval, choose AL one option, wait the specified delay, consume their tL food, and then begin another intertrial interval.time—decision makers steeply discount at short amount of food gained per unit time (Stephenstime delays, and the rate of discounting declines & Krebs, 1986). In the classic patch-choice sce-at longer delays (Fig. 6.3). This model nicely nario, an animal travels among the many patchesmatches the data in humans, rats, and pigeons of food in his or her environment and extracts(Bickel & Johnson, 2003; Green et al., 2004). resources from each patch. The question is: AtA number of studies have tested intertemporal what point should the animal stop extractingchoice in primate species, including cotton-top resources from the current patch and move ontamarins, common marmosets, brown capu- to search for a new patch? This scenario soundschins, long-tailed and rhesus macaques, quite similar to the self-control situation testedchimpanzees, and bonobos (Dufour et al., 2007; in the laboratory. Animals must choose betweenRamseyer et al., 2006; Rosati et al., 2007; Stevens staying in a patch for a short time to extract aet al., 2005; Szalda-Petree et al., 2004; Tobin et al., small gain and staying for a longer time to1996). Unfortunately, we have too few quantita- extract a larger gain. Optimal foraging theorytive data on primates to distinguish between these makes predictions about how long to stay in atwo models. Nevertheless, in cotton-top tamarins patch: stay until the intake rate drops below theand common marmosets, the rate of discounting background intake rate in the environmentslows with time, contradicting predictions of the (Stephens & Krebs, 1986).exponential model (Hallinan et al., unpublished The rate-based approach to intertemporalmanuscript). choice has two key advantages over the standard Though the hyperbolic model has achieved hyperbolic approach. First, it is by definition agreat empirical success, it suffers from a critical model of repeated choice. Each decision faced bydisadvantage: Like prospect theory, it is a purely animals is embedded in a series of decisions.descriptive model. It cannot make a priori predic- Thus, the rate-based approach models choicestions about intertemporal choices and thus lacks in situations very similar to the self-controlexplanatory power (Stevens & Stephens, 2009). In experiments. Second, rate models provide anaddition, both exponential and hyperbolic models explanation of the pattern of temporal prefer-of discounting rest on the assumption of one-shot ences seen in animals rather than just a descrip-choice. In self-control experiments, however, ani- tion. Animals that maximize their intake ratesmals face repeated choices between the same should survive and reproduce better than theiroptions, and the repeated nature of these experi- counterparts that do not—rate models are builtments has important implications for models of on a foundation of evolutionary fitness.intertemporal choice (Kacelnik, 2003; Stevens & We have sound theoretical reasons to favorStephens, 2009). rate models, but how do they fare empirically? Actually, quite well. The short-term rate model makes similar predictions as the hyperbolicRate model (Fig. 6.3). Bateson and Kacelnik (1996)The evolutionary approach to optimal foraging first demonstrated the effectiveness of the short-theory takes as its major assumption the fact that term rate model in describing choices byanimals maximize their intake rate, that is, the European starlings in a self-control experiment.7

RATIONAL DECISION MAKING IN PRIMATES 109Stevens and colleagues (2005) then tested a var- Interestingly, a decision rule that assesses short-iant of this model: A / (t + h), where h represents term intake rates in a patch situation will auto-the time required to handle the food, an impor- matically result in long-term rate maximizationtant component of the foraging timeline (Rosati because they are logically equivalent (Stephens &et al., 2006). Cotton-top tamarins chose the Anderson, 2001).option that maximized this intake rate. Like If short- and long-term rules are logicallyrats and pigeons, the tamarins waited only 6 to equivalent, this then may explain why bonobos10 seconds for three times as much food. differ from the tamarins and marmosets. BothHowever, the results do not appear so ‘‘impul- represent ‘‘adaptive peaks,’’ so it does not mattersive’’ because we have an explanation for waiting on which peak a particular species rests. Eachsuch short delays that does not require unrea- rule, however, has advantages and disadvan-listic rates of interruption (Stevens & Stephens, tages. The short-term rule has the advantage of2009). simplicity and possibly higher accuracy. It is Data on bonobos also agree with rate max- simpler because it ignores information, namely,imization predictions, but this time long-term the intertrial interval. This may also increaserather than short-term rate predictions (Rosati accuracy because, by ignoring this time interval,et al., 2007). The long-term rate A / ( + t + h) animals can estimate much smaller intervals,includes , the time between trials (or between which results in more accurate estimatespatches in the natural scenario). The long-term (Stephens et al., 2004). Despite the benefits ofrate includes all of the relevant time intervals, the short-term rule, using the long-term ruleand therefore this is the standard model used in provides advantages as well, primarily becauseoptimal foraging theory. Why would some spe- it applies broadly and should result in the ‘‘cor-cies ignore the intertrial interval while other rect’’ outcome in most cases. Unfortunately, thisspecies include it? An ecological rationality is not true for the short-term rule. Though theapproach may help answer this question. short-term rule works well in patch situations, this does not directly carry over to the self-control situation (Stephens & Anderson, 2001). InEcological Rationality of Intertemporal fact, as shown by the tamarins, the short-termChoice rule results in ‘‘impulsive’’ choice in self-controlTamarins and bonobos match predictions of situations, so the animals do not achieve therate-based models of intertemporal choice. Yet, optimal long-term rate of gain. Thus, tamarinsbonobos wait over a minute for three times as may use the short-term rule because it worksmuch food, whereas tamarins wait less than 10 well in a natural foraging task—patch exploita-seconds. Why might the tamarins may use short- tion. When placed in an artificial situation, theterm rates when making these types of choices? rule ‘‘misfires.’’ Bonobos, in contrast, may pos-An answer lies at the heart of ecological ration- sess more acute estimation abilities and there-ality. Namely, the natural decision environment fore use the more powerful long-term rule.strongly shapes the decision mechanisms. Specialized diets also raise interesting ques-Though the self-control experiments appear tions about ecologically rational intertemporalsimilar to natural patch-foraging scenarios, a choice. Common marmosets offer an intriguingkey difference arises: Animals rarely face simul- case because they specialize on a rather uniquetaneous choices in nature but often face sequen- food source: tree exudates. Marmosets have spe-tial choices in patch situations. Rather than cialized teeth, as well as digestive morphologyfacing two binary options, animals regularly and physiology, adapted to gouging holes in treechoose to stay or leave a patch. Stephens and bark and eating the sap and gum that exude fromAnderson (2001) argued that this represents the the holes (Rylands & de Faria, 1993; Stevenson &natural foraging decision that animals typically Rylands, 1988). Because this foraging strategyconfront, and therefore this should be the situa- requires waiting for the sap to exude, Stevenstion for which decision mechanisms evolved. and colleagues (2005) proposed an ecologically

110 PRIMATE NEUROETHOLOGYrational response to this type of foraging Delayed Gratificationstrategy: A decision mechanism that is biased In addition to perceiving time, delayed rewardstoward waiting for longer delays. Compared to require a motivational ability to wait—decisionthe more insectivorous tamarins, marmosets makers must delay gratification. Mischel andshould show stronger preferences for delayed colleagues pioneered the study of delayed grati-rewards. As predicted, the marmosets waited fication by measuring how long children wouldlonger than tamarins in a self-control situation, wait for delayed rewards (Mischel & Ebbesen,suggesting that the natural foraging ecology 1970; Mischel et al., 1989). They recorded theshapes the decision mechanisms (Stevens ability to wait at different ages and showedet al., 2005). that delayed gratification at a young age strongly predicted intelligence, academic suc-Intertemporal Building Blocks cess, standardized test scores, and drug use much later in life.Time poses unique cognitive challenges for Beran and colleagues have tested primates inorganisms. Intertemporal choice combines estab- a similar paradigm by offering chimpanzees andlishing preferences over rewards with tracking rhesus macaques a stream of food rewardsthese payoffs over time. Now, we explore what (Beran et al., 1999; Evans & Beran, 2007b). Thekind of cognitive building blocks might be experimenter placed rewards in front of the sub-required to make intertemporal choices. jects one by one at a particular rate, say one every 10 seconds. If the subject began consuming theTime Perception food, however, the experimenter stopped the flow of food. Chimpanzees performed quiteTo cope with delayed rewards, an organism must well on this task and some waited for over 10perceive the delay. Unfortunately, we know little minutes for the stream of food to be completedabout primate time perception. We do, however, (Beran & Evans, 2006). This corroborates find-know a lot about time perception in rats, which ings in the self-control task in which chimpan-likely applies to primates. Like magnitude judg- zees wait longer than any other species tested soments, time interval judgments seem to follow far, including bonobos (Dufour et al., 2007;Weber’s law. In fact, Gibbon (1977) showed that Rosati et al., 2007). Interestingly, this abilityvariance scaled with magnitude in time percep- can be influenced by attentional factors. Evanstion before applying it to quantification. Given and Beran (2007a) found that offering chimpan-that both primate numerical judgments (see zees a distraction during the waiting periodChapter 8) and human time perception (Allan, could significantly increase their abilities to1998) follow Weber’s law, likely nonhuman pri- delay gratification. Thus, delayed gratificationmate time perception does as well. This has could provide a key building block required forimportant implications for the study of inter- patient choice, but it too is mediated by othertemporal choice. First, logarithmic time percep- factors such as attention.tion (resulting from Weber’s law) may result inthe hyperbolic pattern of intertemporal choicetypically seen in both humans and other animals CONCLUSION(Takahashi, 2005; Takahashi et al., 2008). Thisfinding could rescue the exponential model of Bounded rationality and ecological rationalitydiscounting by overlaying a subjective time per- are both firmly grounded in an evolutionaryception function. Second, as we have already perspective on decision making. Natural selec-seen, long time delays make discriminating the tion places limitations on cognitive capacitiesdelays difficult. If time delays are viewed as and tailors cognitive mechanisms to the envir-equivalent, decision makers should opt for the onment in which they are used (Barkow et al.,larger (and therefore later) reward, resulting in 1992; Stevens, 2008). Darwin (1871, 1872)more patient choice. emphasized a continuity between the ‘‘mental

RATIONAL DECISION MAKING IN PRIMATES 111powers’’ of humans and other animals, and the between the evolutionary and mechanisticanalysis provided here confirms a similar con- approaches to decision making.tinuity for decision making. Nonhuman pri- The study of primate decision making hasmates show many of the so-called biases or much to offer the larger field of decisionanomalies demonstrated by human decision making, and prospects are bright for futuremakers. The human decision-making literature contributions. To advance the field, we musthas highlighted these results as failures to begin exploring primate decisions in the wild orachieve a normative outcome. The evolutionary at least in more naturalistic situations.approach suggests that the bias lies not in the Currently, most studies of primate decisionsbehavior but in the normative criteria used. (and animal decisions more broadly) occur inOrganisms did not evolve to follow a mathema- the laboratory with rather artificial scenarios.tically tractable set of principles—rather, natural Though these studies provide valuable infor-selection favored decision strategies that resulted mation on primate decision making, they mayin greater survival and reproduction. In some also lead us astray. If natural selection tailorscases, the evolutionary and normative perspec- decision mechanisms to the environment intives may overlap. However, the normative per- which they are adapted, then the artificialspective often fails to properly account for the nature of the laboratory might not trigger therole of the environment in decision making, appropriate mechanism. In this case, behaviorwhereas this is a critical part of the evolutionary seen in both humans and other animals mayview. Natural selection shapes decision strategies simply be spurious results elicited by an unna-to match the environment. tural environment (Houston et al., 2007b; This lesson about the limitations of the nor- Stevens & Stephens, 2009). Thus, exploring nat-mative approach is an important one because ural behaviors in natural environments under-many fields, especially neuroscience, use tools lies our understanding of primate decisions.borrowed from economics to greatly expand Another important advance that primatethe study of decision making. Though this researchers may offer is the development andcould prove a fruitful enterprise, the data on testing of process models of decision making.decision making in other animals caution David Marr (1982) introduced the idea of threeagainst relying exclusively on the normative levels of information processing. The computa-models of decision making. As a biological tional level emphasizes the goal of the system,science, neuroscience should heed the warning the algorithmic level emphasizes the processesof Dobzhansky (1964): ‘‘nothing makes sense in used, and the implementational level empha-biology except in light of evolution’’ (p. 449). Of sizes the neural circuitry required to processcourse, the emphasis on evolutionarily plausible information. Most models of decision makingmodels is nothing new to the field of neu- in animals act at the computational level—thatroethology, and this tradition should continue is, level of evolutionary function—in the fieldin the study of decision making. Rather than of behavioral ecology or the implementationallooking for expected or discounted utility in level in the field of neurobiology. The algo-the brain, perhaps we should test hypotheses rithmic level has been greatly neglected as aabout the bounded and ecological rationality of relevant level of analysis. Yet, adding an analysisdecision making. The bounded rationality of the relevant cognitive processes and algo-approach makes clear predictions about what rithms could constrain and improve evolu-cognitive building blocks might be recruited, tionary models of decision making. Only anand neuroscience can help test this. In addition, integrative approach across these levels of ana-neuroscience can allow us to explore what lysis—from the evolutionary to the psycholo-aspects of the environment are relevant gical to the neurobiological—will yieldfor triggering specific decision rules. The neu- satisfying answers to questions of the nature ofroethological approach offers an ideal interface decision making.

112 PRIMATE NEUROETHOLOGYNOTES 6. Though V = A/(1 + kt) is called the hyperbolic model, it represents only a single instance from a1. Utility does not necessarily increase linearly class of hyperbolic models. In fact, most with magnitude. The difference between nonexponential models (including the rate consuming one and three bananas is not the models) are hyperbolic. Nevertheless, the term same as the difference between consuming 11 ‘‘hyperbolic discounting’’ typically refers to and 13 bananas. A difference of two bananas is Mazur’s (1987) version. much more relevant when there are fewer to 7. The key difference between the short-term and begin with. long-term rate models is that the short-term2. Kahneman and Tversky do not use the term model ignores the travel time or intertrial ‘‘irrational.’’ They describe deviations from the interval. Thus, animals using the short-term normative theory as ‘‘biases,’’ ‘‘fallacies,’’ or rate only focus on the times between choice ‘‘cognitive illusions.’’ Nevertheless, the presentation and reward consumption. emphasis on putative errors highlights the irrational nature of human decision making (Rieskamp et al., 2006a).3. Most work on decision making in animals, REFERENCES including primates, involves food as the reward, though there are important exceptions Ainslie, G. (1975). Specious reward: A behavioral (Deaner et al., 2005). Therefore, in this review, I theory of impulsiveness and impulse control. will also focus on food as the reward domain. Psychological Bulletin, 82, 463–496. Nevertheless, many of the principles mentioned Ainslie, G., & Herrnstein, R. J. (1981). Preference here apply to other reward domains, and the reversal and delayed reinforcement. Animal and study of animal decision making needs more Learning Behavior, 9, 476–482. in-depth exploration of other reward types. Allais, P. M. (1953). The behavior of rational man4. Though we have few data on transitive in risk situations: A critique of the axioms preference, transitive inference is well studied in and postulates of the American School. primates. Transitive preference refers to an Econometrica, 21, 503–546. ordered preference over choices. Transitive Allan, L. G. (1998). The influence of the scalar inference refers to the ability to infer a timing model on human timing research. transitive relation between objects Behavioural Processes, 44, 101–117. (Vasconcelos, 2008). For instance, if individual Anderson, J. R., Awazu, S., & Kazuo, F. (2000). Can A is dominant to B and B is dominant to C, can squirrel monkeys (Saimiri sciureus) learn self- an individual infer that A is dominant to C control? A study using food array selection tests without actually seeing A and C interact? and reverse-reward contingency. Journal of Primates are quite good at these tasks, both Experimental Psychology: Animal Behavior with arbitrary objects in artificial laboratory Processes, 26, 87–97. tasks (reviewed in Tomasello & Call, 1997) and Bailey, J. T., & Mazur, J. E. (1990). Choice behavior with social agents in the more naturalistic social in transition: Development of preference for inferences (Cheney & Seyfarth, 1990). the higher probability of reinforcement.5. Despite the sparse data on animal intransitivity, Journal of the Experimental Analysis of Houston and colleagues (Houston, 1997; Behavior, 53, 409–422. Houston et al., 2007a) have developed a series Barkow, J. H., Cosmides, L., & Tooby, J. (1992). of models demonstrating that intransitivity The adapted mind: Evolutionary psychology and might be a perfectly adaptive strategy when a the generation of culture. New York: Oxford decision maker is either updating his or her University Press. assessment of the environment or betting on a Baron, J. (2000). Thinking and deciding. changing environment. Schuck-Paim and Cambridge, UK: Cambridge University Press. Kacelnik (2004) also assert that intransitive Bateson, M., & Kacelnik, A. (1996) Rate currencies choices (particularly when food is the reward) and the foraging starling: The fallacy of the can be perfectly biologically rational (Kacelnik, averages revisited. Behavioral Ecology, 7, 341–352. 2006) when the internal state changes (i.e., Behar, I. (1961). Learned avoidance of nonreward. decision makers become satiated). Psychological Reports, 9, 43–52.

RATIONAL DECISION MAKING IN PRIMATES 113Beran, M. J., & Evans, T. A. (2006). Maintenance of social images by rhesus macaques. Current delay of gratification by four chimpanzees (Pan Biology, 15, 543–548. troglodytes): The effects of delayed reward Dobzhansky, T. (1964). Biology, molecular and visibility, experimenter presence, and extended organismic. American Zoologist, 4, 443–452. delay intervals. Behavioural Processes, 73, 315–324. Dufour, V., Pele, M., Sterck, E. H. M., & Thierry, B.Beran, M. J., Savage-Rumbaugh, E., Pate, J. L., & (2007). Chimpanzee (Pan troglodytes) Rumbaugh, D. M. (1999). Delay of gratification anticipation of food return: Coping with in chimpanzees (Pan troglodytes). Develop- waiting time in an exchange task. Journal of mental Psychobiology, 34, 119–127. Comparative Psychology, 121, 145–155.Bickel, W. K., & Johnson, M. W. (2003). Delay Ellsberg, D. (1961). Risk, ambiguity, and the Savage discounting: A fundamental behavioral process axioms. Quarterly Journal of Economics, 75, of drug dependence. In: G. Loewenstein, D. 643–669. Read, & R. F. Baumeister (Eds.), Time and Evans, T. A., & Beran, M. J. (2007a). Chimpanzees decision: Economic and psychological use self-distraction to cope with impulsivity. perspectives on intertemporal choice (pp. 419– Biology Letters, 3, 599–602. 440). New York: Russell Sage Foundation. Evans, T. A., & Beran, M. J. (2007b). Delay ofBoysen, S. T., & Berntson, G. G. (1995). Responses gratification and delay maintenance by rhesus to quantity: Perceptual versus cognitive macaques (Macaca mulatta). Journal of General mechanisms in chimpanzees (Pan troglodytes). Psychology, 134, 199–216. Journal of Experimental Psychology: Animal Frederick, S., Loewenstein, G., & O’Donoghue, T. Behavior Processes, 21, 82–86. (2002). Time discounting and time preference:Brannon, E. M. (2005a). Quantitative thinking: A critical review. Journal of Economic Literature, From monkey to human and human infant to 40, 351–401. adult. In: S. Dehaene, J-R. Duhamel, M. D. Gallistel, C. R. (1990). The organization of learning. Hauser, & G. Rizzolatti (Eds.), From monkey Cambridge, MA: MIT Press. brain to human brain (pp. 97–116). Genty, E., Palmier, C., & Roeder, J-J. (2004). Cambridge, MA: MIT Press. Learning to suppress responses to the largerBrannon, E. M. (2005b). The numerical ability of of two rewards in two species of lemurs, animals. In: J. D. Campbell (Ed.), Handbook of Eulemur fulvus and E. macaco. Animal mathematical cognition (pp. 85–108). New Behaviour, 67, 925–932. York: Psychology Press. Gibbon, J. (1977). Scalar expectancy theory andBrosnan, S. F., Jones, O. D., Lambeth, S. P., Weber’s law in animal timing. Psychological Mareno, M. C., Richardson, A. S., & Schapiro, Review, 84, 279–325. S. J. (2007). Endowment effects in chimpanzees. Gigerenzer, G., Todd, P. M., & the ABC Research Current Biology, 17, 1704–1707. Group. (1999). Simple heuristics that make usCaraco, T., Martindale, S., & Whittam, T. S. (1980). smart. Oxford: Oxford University Press. An empirical demonstration of risk-sensitive Gilby, I. C., & Wrangham, R. W. (2007). Risk- foraging preferences. Animal Behaviour, 28, prone hunting by chimpanzees (Pan 820–830. troglodytes schweinfurthii) increases duringChen, M. K., Lakshminarayanan, V., & Santos, L. R. periods of high diet quality. Behavioral Ecology (2006). How basic are behavioral biases? and Sociobiology, 61, 1771–1779. Evidence from capuchin monkey trading Green, L., Myerson, J., Holt, D. D., Slevin, J. R., & behavior. Journal of Political Economy, 114, Estle, S. J. (2004). Discounting of delayed food 517–537. rewards in pigeons and rats: Is there aCheney, D. L., & Seyfarth, R. M. (1990). How magnitude effect? Journal of the Experimental monkeys see the world: Inside the mind of another Analysis of Behavior, 81, 39–50. species. Chicago: University of Chicago Press. Hallinan, E. V., Stevens, J. R., & Hauser, M. D.Darwin, C. (1871). The descent of man and selection (Unpublished manuscript). Rate maximization in relation to sex. London: Murray. in common marmosets (Callithrix jacchus) andDarwin, C. (1872). The expression of the emotions in cotton-top tamarins (Saguinus oedipus). man and animals. London: Murray. Hammerstein, P. (1981). The role of asymmetriesDeaner, R. O., Khera, A. V., & Platt, M. L. (2005). in animal contests. Animal Behaviour, 29, Monkeys pay per view: Adaptive valuation of 193–205.

114 PRIMATE NEUROETHOLOGYHauser, M. D., Carey, S., & Hauser, L. B. (2000). Kacelnik, A., & Bateson, M. (1996). Risky theories: Spontaneous number representation in semi- The effects of variance on foraging decisions. free-ranging rhesus monkeys. Proceedings of American Zoologist, 36, 402–434. the Royal Society of London Series B, 267, Kahneman, D., Knetsch, J. L., & Thaler, R. H. 829–833. (1990). Experimental tests of the endowmentHauser, M. D., & Spelke, E. S. (2004). Evolutionary effect and the Coase theorem. Journal of and developmental foundations of human Political Economy, 98, 1325–1348. knowledge: A case study of mathematics. In: Kahneman, D., Slovic, P., & Tversky, A. (1982). M. Gazzaniga (Ed.), The cognitive neurosciences Judgment under uncertainty: Heuristics and III (pp. 853–864). Cambridge, MA: MIT Press. biases. New York: Cambridge University Press.Hauser, M. D., Tsao, F., Garcia, P., & Spelke, E. S. Kahneman, D., & Tversky, A. (1979). Prospect (2003). Evolutionary foundations of number: theory: An analysis of decision under risk. Spontaneous representation of numerical Econometrica, 47, 263–292. magnitudes by cotton-top tamarins. Kahneman, D., & Tversky, A. (2000). Choices, Proceedings of the Royal Society of London values, and frames. New York: Russell Sage Series B, 270, 1441–1446. Foundation; Cambridge University Press.Hayden, B. Y., & Platt, M. L. (2007). Temporal Knight, F. H. (1921). Risk, uncertainty, and profit. discounting predicts risk sensitivity in rhesus Boston: Houghton Mifflin. macaques. Current Biology, 17, 49–53. Kralik, J. D., Hauser, M. D., & Zimlicki, R. (2002).Heilbronner, S. R., Rosati, A. G., Stevens, J. R., The relationship between problem solving and Hare, B., & Hauser, M. D. (2008). A fruit in inhibitory control: Cotton-top tamarin the hand or two in the bush? Divergent risk (Saguinus oedipus) performance on a reversed preferences in chimpanzees and bonobos. contingency task. Journal of Compartive Biology Letters, 4, 246–249. Psychology, 116, 39–50.Herrnstein, R. J., & Loveland, D. H. (1975). Krebs, J. R., & Davies, N. B. (1993). An introduction Maximizing and matching on concurrent ratio to behavioural ecology. Oxford: Blackwell Science. schedules. Journal of the Experimental Analysis Krebs, J. R., Kacelnik, A., & Taylor, P. (1978). Test of Behavior, 24, 107–116. of optimal sampling by foraging great tits.Hertwig, R., Barron, G., Weber, E. U., & Erev, I. Nature, 275, 27–31. (2004). Decisions from experience and the Lakshminaryanan, V., Chen, M. K., & Santos, L. R. effect of rare events in risky choice. (2008). Endowment effect in capuchin monkeys. Psychological Science, 15, 534–539. Philosophical Transactions of the Royal Society ofHouston, A. I. (1997). Natural selection and London, Series B, 363, 3837–3844. context-dependent values. Proceedings of the Luce, R. D., & Raiffa, H. (1957). Games and Royal Society of London Series B, 264, 1539– decisions: Introduction and critical survey. New 1541. York: Wiley.Houston, A. I., McNamara, J. M., & Steer, M. D. Marr, D. (1982). Vision: A computational investi- (2007a). Violations of transitivity under fitness gation into the human representation and maximization. Biology Letters, 3, 365–367. processing of visual information. New York:Houston, A. I., McNamara, J. M., & Steer, M. D. W. H. Freeman and Co. (2007b). Do we expect natural selection to Marsh, B., & Kacelnik, A. (2002). Framing effects produce rational behaviour? Philosophical and risky decisions in starlings. Proceedings of the Transactions of the Royal Society Series B, 362, National Academy of Sciences, 99, 3352–3355. 1531–1543. Maynard Smith, J., & Parker, G. A. (1976). TheKacelnik, A. (2003). The evolution of patience. In: logic of asymmetric contests. Animal G. Loewenstein, D. Read, & R. F. Baumeister Behaviour, 24, 159–175. (Eds.), Time and decision: Economic and Mazur, J. E. (1987). An adjusting procedure psychological perspectives on intertemporal for studying delayed reinforcement. In: choice (pp. 115–138). New York: Russell Sage M. L. Commons, J. E. Mazur, J. A. Nevin, & Foundation. H. Rachlin (Eds.), Quantitative analyses ofKacelnik, A. (2006). Meanings of rationality. In: S. behavior: The effect of delay and of intervening Hurley & M. Nudds (Eds.), Rational animals? events on reinforcement value (pp. 55–73). (pp. 87–106). Oxford: Oxford University Press. Hillsdale, NJ: Lawrence Erlbaum Associates.

RATIONAL DECISION MAKING IN PRIMATES 115McCoy, A. N., & Platt, M. L. (2005). Risk-sensitive temporal discounting in two New World neurons in macaque posterior cingulate cortex. primates. Animal Behaviour, 71, 1379–1387. Nature Neuroscience, 8, 1220–1227. Rylands, A. B., & de Faria, D. S. (1993). Habitats,Mischel, W., & Ebbesen, E. B. (1970). Attention in feeding ecology and range size in the genus delay of gratification. Journal of Personality and Callithrix. In: A. B. Rylands (Ed.), Marmosets Social Psychology, 16, 329–337. and tamarins: Systematics, behaviour, andMischel, W., Shoda, Y., & Rodriguez, M. L. (1989). ecology (pp. 262–272). Oxford: Oxford Delay of gratification in children. Science, 244, University Press. 933–938. Samuelson, P. A. (1937). A note on measurement ofMurray, E. A., Kralik, J. D., & Wise, S. P. (2005). utility. Review of Economic Studies, 4, 155–161. Learning to inhibit prepotent responses: Schuck-Paim, C., Pompilio, L., & Kacelnik, A. Successful performance by rhesus macaques, (2004). State-dependent decisions cause Macaca mulatta, on the reversed-contingency apparent violations of rationality in animal task. Animal Behaviour, 69, 991–998. choice. PLoS Biology, 2, e402.Padoa-Schioppa, C., Jandolo, L., & Visalberghi, E. Shafir, S. (1994). Intransitivity of preferences in (2006). Multi-stage mental process for economic honey-bees: Support for comparative- choice in capuchins. Cognition, 99, B1–B13. evaluation of foraging options. AnimalPayne, J. W., Bettman, J. R., & Johnson, E. J. Behaviour, 48, 55–67. (1993). The adaptive decision maker. Silberberg, A., & Fujita, K. (1996). Pointing at Cambridge: Cambridge University Press. smaller food amounts in an analogue of BoysenRachlin, H., Logue, A. W., Gibbon, J., & Frankel, and Berntson’s (1995) procedure. Journal of the M. (1986). Cognition and behavior in studies of Experimental Analysis of Behavior, 66, 143–147. choice. Psychological Review, 93, 33–45. Silberberg, A., Widholm, J. J., Bresler, D., Fujita, K.,Ramseyer, A., Pele, M., Dufour, V., Chauvin, C., & & Anderson, J. R. (1998). Natural choice in Thierry, B. (2006). Accepting loss: The temporal nonhuman primates. Journal of Experimental limits of reciprocity in brown capuchin Psychology: Animal Behavior Processes, 24, monkeys. Proceedings of the Royal Society of 215–228. London Series B, 273, 179–184. Simon, H. A. (1955). A behavioral model ofRead, D. (2004). Intertemporal choice. In: D. rational choice. Quarterly Journal of Economics, Koehler & N. Harvey (Eds.), Blackwell 69, 99–118. handbook of judgment and decision making (pp. Simon, H. A. (1956). Rational choice and the 424–443). Oxford: Blackwell. structure of the environment. PsychologicalRichards, J. B., Mitchell, S. H., de Wit, H., & Review, 63, 129–138. Seiden, L. S. (1997). Determination of Simonson, I., & Tversky, A. (1992) Choice in discount functions in rats with an adjusting- context: Tradeoff contrast and extremeness amount procedure. Journal of the Experimental aversion. Journal of Marketing Research, 29, Analysis of Behavior, 67, 353–366. 281–295.Rieskamp, J., Hertwig, R., & Todd, P. M. (2006a). Stephens, D. W. (1981). The logic of risk-sensitive Bounded rationality: Two interpretations from foraging preferences. Animal Behaviour, 29, psychology. In: M. Altman (Ed.), Handbook of 628–629. contemporary behavioral economics (pp. 218– Stephens, D. W., & Anderson, D. (2001). The 236). Armonk, NY: M.E. Sharpe. adaptive value of preference for immediacy:Rieskamp, J., Busemeyer, J. R., & Mellers, B. A. When shortsighted rules have farsighted (2006b). Extending the bounds of rationality: consequences. Behavioral Ecology, 12, 330–339. Evidence and theories of preferential choice. Stephens, D. W., Kerr, B., & Fernandez-Juricic, E. Journal of Economic Literature, 44, 631–661. (2004). Impulsiveness without discounting: TheRosati, A. G., Stevens, J. R., Hare, B., & Hauser, M. ecological rationality hypothesis. Proceedings of D. (2007). The evolutionary origins of human the Royal Society of London Series B, 271, patience: Temporal preferences in chimpanzees, 2459–2465. bonobos, and adult humans. Current Biology, Stephens, D. W., & Krebs, J. R. (1986). Foraging 17, 1663–1668. theory. Princeton: Princeton University Press.Rosati, A. G., Stevens, J. R., & Hauser, M. D. Stevens, J. R. (2008). The evolutionary biology of (2006). The effect of handling time on decision making. In: C. Engel & W. Singer

116 PRIMATE NEUROETHOLOGY (Eds.), Better than conscious? Decision making, Tobin, H., Logue, A. W., Chelonis, J. J., & the human mind, and implications for Ackerman, K. T. (1996). Self-control in the institutions (pp. 285–304). Cambridge, MA: monkey Macaca fascicularis. Animal Learning MIT Press. and Behavior, 24, 168–174.Stevens, J. R., Hallinan, E. V., & Hauser, M. D. Tomasello, M., & Call, J. (1997). Primate cognition. (2005). The ecology and evolution of patience Oxford: Oxford University Press. in two New World monkeys. Biological Letters, Tversky, A., & Kahneman, D. (1981). The framing 1, 223–226. of decisions and the psychology of choice.Stevens, J.R., & King, A.J. (in press). The lives of Science, 211, 453–458. others: social rationality in animals. In: Tversky, A., & Kahneman, D. (1991). Loss aversion R. Hertwig, U. Hoffrage, & the ABC Research in riskless choice: A reference-dependent Group (Eds.), Social rationality. Oxford: Oxford model. Quarterly Journal of Economics, 106, University Press. 1039–1061.Stevens, J. R., & Stephens, D. W. (2009) The Tversky, A., & Kahneman, D. (1992). Advances in adaptive nature of impulsivity. In: G. J. prospect theory: Cumulative representation of Madden & W. K. Bickel (Eds.), Impulsivity: uncertainty. Journal of Risk and Uncertainty, 5, Theory, science, and neuroscience of discounting 297–323. (pp. 361–387). Washington, DC: American Uller, C., Hauser, M. D., & Carey, S. (2001). Psychological Association. Spontaneous representation of number inStevens, J. R., Wood, J. N., & Hauser, M. D. (2007). cotton-top tamarins (Saguinus oedipus). Journal When quantity trumps number: of Comparative Psychology, 115, 248–257. Discrimination experiments in cotton-top Vasconcelos, M. (2008). Transitive inference in tamarins (Saguinus oedipus) and common non-human animals: An empirical and marmosets (Callithrix jacchus). Animal theoretical analysis. Behavioural Processes, 78, Cognition, 10, 429–437. 313–334.Stevenson, M. F., & Rylands, A. B. (1988). The Vlamings, P., Uher, J., & Call, J. (2006). How the marmosets, genus Callithrix. In: R. A. great apes (Pan troglodytes, Pongo pygmaeus, Mittermeier, A. B. Rylands, A. F. Coimbra- Pan paniscus, and Gorilla gorilla) perform on Filho, & G. A. B. Fonseca (Eds.), Ecology and the reversed contingency task: The effects of behavior of neotropical primates (pp. 131–222). food quantity and food visibility. Journal of Washington, DC: World Wildlife Fund. Experimental Psychology: Animal BehaviorSzalda-Petree, A. D., Craft, B. B., Martin, L. M., & Processes, 32, 60–70. Deditius-Island, H. K. (2004). Self-control in von Neumann, J., & Morgenstern, O. (1947). rhesus macaques (Macaca mulatta): Theory of games and economic behavior (2nd Controlling for differential stimulus exposure. ed.). Princeton: Princeton University Press. Perceptual and Motor Skills, 98, 141–146. Waite, T. A. (2001a). Background context andTakahashi, T. (2005). Loss of self-control in decision making in hoarding gray jays. intertemporal choice may be attributable to Behavioral Ecology, 12, 318–324. logarithmic time-perception. Medical Waite, T. A. (2001b). Intransitive preferences in Hypotheses, 65, 691–693. hoarding gray jays (Perisoreus canadensis).Takahashi, T., Oono, H., & Radford, M. H. B. Behavioral Ecology and Sociobiology 50, 116–121. (2008). Psychophysics of time perception and Wrangham, R. W., & Peterson, D. (1996). Demonic intertemporal choice models. Physica A: males: Apes and the origins of human violence. Statistical Mechanics and Its Applications, 387, Cambridge, MA: Harvard University Press. 2066–2074. Wu, G., Zhang, J., & Gonzalez, R. (2004). DecisionTinklepaugh, O. L. (1928). An experimental study under risk. In: D. Koehler & N. Harvey (Eds.), of representative factors in monkeys. Journal of Blackwell handbook of judgment and decision Comparative Psychology, 8, 197–236. making (pp. 399–423). Oxford: Blackwell.

CHAPTER 7Primate Social Cognition: Thirty Years AfterPremack and WoodruffAlexandra G. Rosati, Laurie R. Santos and Brian HareIn 1871, Darwin wrote, ‘‘The greatest difficulty following her observations of wild lemurs andwhich presents itself, when we are driven to the sifakas in Madagascar. A decade later, Nicholasabove conclusion on the origin of man (evolu- Humphrey (1976) drew many of the same con-tion through natural selection), is the high stan- clusions from watching captive rhesus monkeydard of intellectual power and moral disposition colonies, noting that it was navigation of thewhich he has attained.’’ Since Darwin declared social world, rather than the physical world,the mind as the province of biology as well as that seemed to require the most complex skills.psychology, the human intellect has been a This basic thesis—that the sophisticated cogni-major challenge for evolutionary biologists, tive abilities of primates have evolved for awith some researchers emphasizing the conti- social function—has since taken severalnuity between humans and other animals and forms. For example, some researchers haveothers emphasizing seemingly unique aspects of emphasized the political maneuvering (deour psychological makeup. Increasing observa- Waal, 1982) or ‘‘Machiavellian intelligence’’tions of nonhuman primate (hereafter, primate) (Byrne, 1988) that primates must use to succeedbehavior in both the wild and in captivity in the in their societies, while others have focused onmid-twentieth century led to a number of pro- the evolutionary arms race between intelligenceposals addressing the question of why primates and increasing social complexity (Dunbar, 2003)seem to be so ‘‘smart.’’ These proposals, and the (for a different perspective on these issues, seecomparative research they have sparked, have Chapter 28).far-reaching implications for how we place The social world has therefore long beenhuman cognition in a broader evolutionary con- thought to be a major force shaping primatetext—both in terms of how or to what degree cognition—but, paradoxically, very little washumans are different from our closest relatives as known about the cognitive abilities primateswell as whether broad taxonomic-level evolu- actually use when interacting with other socialtionary changes in the primate lineage were agents. Most early proposals of the social intelli-necessary precursors to human evolution. gence hypothesis stemmed from observations of The most well-received proposal for the origin complex social behaviors across the primateof primate intelligence argues that the social lives taxon, but the psychological mechanisms under-of primates is sufficiently complex—or predic- lying these behaviors were not well understood.tably unpredictable—to have acted as a driving For example, although human social behavior isforce in primate cognitive evolution. Alison Jolly supported by a rich belief-desire psychology(1966) set forth one of the earliest such proposals, through which we can represent and reasonmusing on the ‘‘social use of intelligence’’ about others’ subjective psychological states, it 117

118 PRIMATE NEUROETHOLOGYwas unknown if primates possessed any similar REASONING ABOUTrepresentational capacities. In fact, when PSYCHOLOGICAL STATESPremack and Woodruff (Premack, 1978) firstasked their big question, ‘‘Does the chimpanzee While studies of primate social cognition havehave a theory of mind?’’ they argued that their until recently made it difficult to characterizesingle test subject had shown the ability to the social skills of primates with confidence,assess the intentions of another. However, studies of human infants and toddlers haveafter two decades of research following this mapped out with ever-increasing resolutionpioneering paper, several major syntheses of the fundamental changes that occur in the wayprimate cognition weighed the evidence and young children come to think about others.concluded that although primates can use This research has pointed to the importanceobservable phenomena to make predictions of social-cognitive skills for the developmentabout the future behaviors of others, there was of normal functioning adult behavior. Forno convincing evidence that any nonhuman example, without the normal development ofprimates represent the underlying, unobser- social-cognitive skills, children cannot partici-vable psychological states of others’ minds pate in all forms of cultural endeavors—(Cheney & Seyfarth, 1990; Heyes, 1998; including language (Tomasello, 1999). StartingTomasello & Call, 1997). Research over the in the first year of life, children begin to treatpast 10 years, however, has drawn this initial other people as intentional agents and come tosweeping conclusion into question, revealing organize other people’s actions in terms ofthat at least some primates have some capability goals and desires (Behne et al., 2005;to assess the psychological states of others— Carpenter et al., 1998; Gergely et al., 1995,while simultaneously showing striking differ- 2002; Meltzoff, 1995; Repacholi & Gopnik,ences between the social-cognitive capacities of 1997; Woodward, 1998; Woodward et al.,humans and other primates (Call & Tomasello, 2001). Secondly, children also come to realize2008; Tomasello et al., 2003). that other agents will behave according to their Here we address two aspects of primate perceptions and knowledge (Brooks & Meltzoff,social cognition—understanding of intentional, 2002; Flavell, 1992; Moll & Tomasello, 2004;goal-directed action and understanding percep- Phillips, 2002). By the time they are aroundtions, knowledge, and beliefs—focusing on the 4 years of age, children begin to expect thatnewest comparative research since the last another person will also act in accord withmajor reviews were written on the topic over their beliefs, even when such beliefs conflicta decade ago. We first review evidence sug- with the current state of the world (Wellman,gesting that diverse species of primates under- 1990; Wimmer & Perner, 1983) (see alsostand the actions of others in terms of goals (Onishi & Baillargeon, 2005; Southgate et al.,and intentions, and furthermore can reason 2007; and Surian et al., 2007 for possibleabout some, but probably not all, kinds of evidence at an even earlier age).psychological states. We then examine the To what extent do primates share thesehypothesis that primates show their most com- human developmental achievements? Do theyplex social skills in competitive contexts, and come to reason about others’ behavior in termssuggest that inquiry into other aspects of pri- of internal, unobservable psychological states?mate social life, such as during cooperative Many of the abilities that are of interest to devel-interactions, may prove to be the next impor- opmental psychologists have been the topic oftant step for experimental inquiries into pri- extensive research in nonhuman primates, andmate social-cognitive skills. Finally, we examine often the same paradigms used with childrenprimate social cognition in a broader evolu- have been directly translated into primate stu-tionary context that may allow us to better dies (Tables 7.1 through 7.4). Here we firstunderstand both primate and human cognitive review evidence addressing what various pri-skills. mate species understand about intentional

PRIMATE SOCIAL COGNITION: THIRTY YEARS AFTER PREMACK AND WOODRUFF 119action, and then examine what primates under- human demonstrator’s actions are relevant tostand about perceptions, knowledge, and beliefs. his or her goal. For example, when confronted with a human demonstrator performing variousGoal-directed Behavior and Intentional actions to obtain food from a causally confusingAction opaque puzzle box, chimpanzees faithfully imitate the actor’s complete sequence of actions. InEvidence that at least some primates treat the contrast, when the box is transparent and thus theactions of others in terms of their underlying causal nature of the box and the actor’s goal aregoals and intentions comes from several different clear, chimpanzees engage in goal emulation,sources (Table 7.1). Some of the earliest evidence excluding actions that were irrelevant to the goalthat primates understand the goals of others (Horner & Whiten, 2005).2 In addition to imi-emerged through studies of social learning.1 tating only goal-relevant actions, other evidenceSuch research has revealed that apes may repre- suggests that apes are more likely to exactly copy asent the actions of another individual specifically human demonstrator’s behavior when thatin terms of that person’s goal. That is, whenconfronted with an individual engaging in a demonstrator successfully completes his or hernovel action, apes rarely engage in exact copying goal than when he or she fails (Call et al., 2005;of that action, but rather are more likely to engage Myowa-Yamakoshi & Matsuzawa, 2000). Takenin behavior toward the same goal that the actor together, this work suggests that apes seem towas pursuing, a process referred to as goal emula- naturally parse the behavior of others in termstion (Tomasello, 1990; Tomasello et al., 1987). of goals, and will only copy the superficial beha-Chimpanzees (Pan troglodytes) also seem to vior when the link between the actions and goal atreact differentially depending on whether a hand is not readily apparent, and no otherTable 7.1 Studies of Goal and Intention Understanding Across Nonhuman Primate Species Distinguishing Inferring Goals Intentions OntogenyHominoids Chimpanzees Buttleman et al., 2007; Call et al., Call & Tomasello, 1998; Tomasello & 2005; Horner & Whiten, 2005; Call et al., 2004; Carpenter, Myowa-Yamakoshi & Matsuzawa, Povinelli et al., 1998; 2005; Uller, 2000; Tomasello & Carpenter, Tomasello & Carpenter, 2004 2005; Tomasello et al., 1987; Uller, 2005 2004; Uller & Nichols, 2000; Warneken & Tomasello, 2006; Warneken et al., 2007*; Other great Orangutans: Call & apes Tomasello, 1998 Lesser apesOld World Macaques Rhesusmonkeys Baboons OtherNew World Capuchins Brown: Kuroshima et al., 2008 Brown: Lyons & Santos,monkeys submitted; Phillips et al., 2009 Callitrichids Cotton-top tamarins OtherStrepsirrhines Lemurs* Indicates that the study involved both human and conspecific social partners.

120 PRIMATE NEUROETHOLOGYappropriate means is available. To our knowl- whether these abilities represent convergent cog-edge, there is only limited work testing more nitive evolution between apes and capuchins, adistantly related species on similar goal emulation distantly related New World monkey, or aretasks, with mixed results (e.g., Kuroshima et al., rather a set of abilities that are widely shared2008). across primates. A more widely used assessment A second line of evidence suggesting that pri- of intention understanding in primates is amates have some understanding of others’ inten- method referred to as the ‘‘‘object-choice’’ para-tional action comes from studies in which similar digm, in which animals are presented with inten-or identical actions are performed, but the intentional communicative cues (Table 7.2). The goaltion underlying these actions vary. Orangutans of such studies is to examine whether primates(Pongo pygmaeus) and chimpanzees can tell can successfully use communicative gestures towhether an action is intentional versus accidental locate hidden objects, typically desirable food(Call & Tomasello, 1998). Importantly, this capa- items. In a typical version of this type of task, acity is likely not limited to apes—capuchins human experimenter might point at one of several(Cebus apella) show similar abilities (Lyons & cups that contains a piece of food, and then allowSantos, submitted). Moreover, Chimpanzees and the subject to choose between the cups (the sub-capuchins seem to differentiate between different ject only knows something is hidden but does nottypes of underlying intentions. When chimpan- know where).zees are confronted with a human who fails to give Many studies utilizing this sort of object-them food, they are more likely to produce beg- choice paradigm suggest that while apes areging and other relevant behaviors (and are less able to spontaneously use such gestures to findlikely to leave the room) when the human is food, their performance is fragile and often onlyunable to give them the food (e.g., because he or successful at the group level (see reviews of thisshe dropped it) than when the human is unwilling object-choice work in Hare & Tomasello, 2005;to give the food (e.g., because he or she is teasing). Call & Tomasello, 2008). However, other evi-That is, the chimpanzees did not react only to the dence suggests that the fragility of the apes’ per-superficial result of the human’s behavior—not formance may be less because apes cannot usegetting any food—but also to the reason the gestures to find food and more due to the diffi-human failed to give the food (Call et al., 2004). culty in understanding the cooperative-commu-Capuchin monkeys also seem to discriminate nicative intentions underlying these gestures.between actors that are either unwilling or For example, chimpanzees are more successfulunable, remaining for a longer period in the when a human competitor reaches for a foodtesting area when a human is unable to give cup that they also want than when a humanthem food (because a second human keeps simply points to a cup in a cooperative fashionstealing it) than when a human teases them with (Hare & Tomasello, 2004). Similarly, apes arefood. Furthermore, capuchins make these distinc- more successful using prohibitive hand gesturestions specifically when the relevant actor is an (‘‘Don’t touch that one!’’) to find food then theyagent (i.e., a human hand), but not when an are using a standard cooperative pointing cueinanimate object (e.g., a stick) enacts the same (Herrmann & Tomasello, 2006). This is sur-behavior (Phillips et al., 2009). prising given that both the reaching and prohibitive gestures have nearly identical surface features to the pointing gesture. One interpreta-Understanding Intentional tion of this pattern of performance is that pri-Communicative Cues mates are more successful at using these reachingAlthough these findings support the idea that at and prohibitive types of cues because apes moreleast chimpanzees and capuchins perceive others’ often compete with others over food rather thanbehaviors in terms of goals and intentions, such cooperatively share information about its loca-studies have been conducted with very few spe- tion with others. These competitive cues maycies; this limitation makes it difficult to assess therefore be more ecologically valid, and

Table 7.2 Studies of Social-Cue Use Across Nonhuman Primate Species Touch, Body Position, and Gaze Points and Other Gestures Physical Markers Ontogeny Hominoids Chimpanzees Barth et al., 2005; Braeuer et al., Barth et al., 2005; Braeuer et al., Barth et al., 2005; Call et al., Herrmann et al., 2007; 2006; Call et al., 1998, 2000; 2006; Hare & Tomasello, 2004,* 2000; Herrmann et al., 2006, Okamoto et al., 2002; Herrmann et al., 2007; Itakura 2006; Herrmann et al., 2007; 2007; Itakura et al., 1999*; Okamoto-Barth et al., 2008; et al., 1999; Okamoto-Barth Itakura et al., 1999*; Okamoto-Barth et al., 2008 Tomasello & Carpenter, 2005; et al., 2008; Povinelli et al., Okamoto-Barth et al., 2008; Tomonage et al., 2004 1999* Povinelli et al., 1990, 1997, 1999; Wood et al., 2007 Other Bonobos: Bonobos: Braeuer et al., 2006; Bonobos: Herrmann et al., 2006 Orangutans: great apes Braeuer et al., 2006 Herrmann & Tomasello, 2006; Gorillas: Herrmann et al., 2006 Herrmann et al., 2007 Orangutans: Herrmann et al., Orangutans: Herrmann et al., Orangutans: Herrmann et al., 2007 2007 2006, 2007 Lesser apes White-handed gibbons: White-handed gibbons: Inoue et al., 2004 Inoue et al., 2004 Old World Macaques Rhesus: Anderson et al., 1996; Rhesus: Anderson et al., 1996; monkeys Hauser et al., 2007 Hauser et al., 2007; Wood et al., 2007 Baboons Olive: Vick & Anderson, 2003; Vick et al., 2001 Other New World Capuchins Brown: Anderson et al., 1995; Brown: Anderson et al., 1995; Brown: Vick & Anderson, 2000 monkeys Vick & Anderson, 2000 Vick & Anderson, 2000 Callitrichids Cotton-top tamarins: Neiworth Cotton-top tamarins: Neiworth Cotton-top tamarins: Neiworth et al., 2002 et al., 2002; Wood et al., 2007 et al., 2002 Common marmosets: Burkhart Common marmosets: Burkhart Common marmosets: Burkhart & Heschl, 2006 & Heschl, 2006 & Heschl, 2006 Other Strepsirrhines Lemurs * Indicates that the study involved both human and conspecific social partners.121

122 PRIMATE NEUROETHOLOGYpotentially more motivating for some apes (as primates lack even a very gross understanding ofreviewed in Hare, 2001; Lyons & Santos, 2006; the nature of visual perceptions. For example,Santos et al., 2007a). In the context of intention inspired by Premack (1988), Povinelli and Eddyunderstanding, interpreting others’ behaviors in (1996c) taught young chimpanzees to use a visualterms of competitive goals (‘‘I want the food begging gesture to obtain food from a humantoo!’’) in these social cuing paradigms may be experimenter. The researchers then presentedmore transparent than interpreting their beha- the chimpanzees with a situation in which theyvior in terms of cooperative goals (‘‘I want to tell could choose one of two experimenters fromyou where the food is for your benefit.’’). whom to beg. The trick was that the two experi- In contrast to the studies with apes, however, menters differed in their perceptual access to themany monkey species fail to use communicative chimpanzees: One experimenter could see thecues in similar kinds of studies, at least in the chimpanzees, whereas the other could not for aabsence of extensive training. For example, capu- variety of reasons. Although the chimpanzeeschins can learn to use a pointing cue to find food, spontaneously chose the human with visualbut only following several dozens or even hun- access to their gestures in the condition involvingdreds of trials (Anderson et al., 1995; Vick & the most contrast between the two humans (e.g.,Anderson, 2000). Rhesus macaques (Macaca preferring to beg from a human facing them thanmulatta; Anderson et al., 1996) and cotton-top with her back turned), they failed to discriminatetamarins (Saguinus oedipus; Neiworth et al., 2002) between the two humans in a variety of other,also perform poorly on these tasks. However, more subtle situations (such as a person with hermore recent evidence complicates this picture. face turned away versus one oriented toward theFor example, common marmosets (Callithix jac- subject or one with a blindfold over her eyeschus) are more successful using a pointing cue on versus another with a blindfold over hera modified version of the task (Burkhart & Heschl, mouth). Early experiments such as these seemed2006), while rhesus monkeys tested with a more to provide strong evidence that primates do notspecies-specific looking gesture are better able to understand what others can and cannot see.determine the location of hidden food (Hauser However, converging evidence from manyet al., 2007). Wood and colleagues (2007) further different paradigms and species now appears toargue that cotton-top tamarins, rhesus, and chim- refute this early view of primates’ understandingpanzees are sensitive to hand gestures when such of others’ perspective: Many primates are at leasthand gestures are indicative of an intentional com- behaviorally responsive to the direction ofponent of a goal-directed action plan. As it is others’ gaze and attention, and there is a subsetunclear why this result is discrepant with results of these species that appears to have a flexiblefrom past studies involving chimpanzees, further understanding of what others perceive. At theinvestigations would profit from parsing out why most basic level, diverse species of primatesprimates may demonstrate understanding of spontaneously follow the gaze of human experi-intentions in some contexts but not others. menters or conspecifics. Gaze-following beha-Taken together, then, studies of communicative viors allow individuals to apprehend importantgesture use suggest that primates’ performance objects and events that others have detected inmay be fragile and context dependent, but pri- the environment, including food sources, preda-mates do seem to readily use information tors, and conspecifics. Thus, gaze followingregarding another individual’s intentions and allows individuals to exploit the informationgoals in more competitive paradigms. that others have acquired about the world. Species including chimpanzees (Povinelli & Eddy, 1996a; Tomasello et al., 1998) and theGaze Following and the Roots of Mind other great apes (Braeuer et al., 2005; Okamoto-Reading Barth et al., 2007); Old World monkeys such asEarly studies exploring what primates know various macaques (rhesus: Macaca mulatta;about others’ visual attention suggested that stumptail: M. arctoides; pigtail: M. memstrina;

PRIMATE SOCIAL COGNITION: THIRTY YEARS AFTER PREMACK AND WOODRUFF 123Emery et al., 1997; Tomasello et al., 1998), man- and bonobos—appear to be especially sophisti-gabeys (Cercocebus atys torquats; Tomasello et al., cated in these contexts even compared to other1998) and olive baboons: mangalys (Cercocebus great apes (Okamoto-Barth et al., 2007).atys torquats; Tomasello et al., 1998), New World The evidence for such behavior in more dis-monkeys including capuchins (Vick & Anderson, tantly related primate species is less complete,2000), cotton-top tamarins (Neiworth et al., mostly because few studies have been conducted2002), and common marmosets (Callithrix jac- (Table 7.3). Macaques, like apes, habituate tochus; Burkhart & Heschl, 2006); and even some repeated gaze cues when they repeatedly cannotlemur species (ring tailed: Lemur catta; brown locate the target of another’s gaze (Goossens et al.,lemurs: Eulemur fulvus; black lemurs: Eulemur 2008; Tomasello et al., 2001). However, studies ofmacaco; Shepherd & Platt, 2008; Ruiz et al., New World monkeys and lemurs suggest that the2009) all follow gaze, at least in some contexts. co-orienting behaviors in some of these speciesAlthough there is variation in the degree to are more reflexive. For example, cotton-topwhich various species can successfully follow tamarins will co-orient with conspecifics at higheye position alone (e.g., apes: Tomasello et al., rates during natural interactions (although the2007; olive baboons (e.g., apes: Tomasello et al., cause of this co-orienting is unclear), but fail to2007) or rather can only follow shifts in the follow the explicit gaze cues provided in con-position of the face, head, or even entire body trolled experimental settings (Neiworth et al.,(e.g. capuchins: Vick & Anderson, 2000; cotton- 2002). Similarly, some lemur species co-orienttop tamarins: Neiworth et al., 2002; ring-tailed with conspecifics during their natural behaviorslemurs: Shepherd & Platt, 2008), this variation (Shepherd & Platt, 2008), but seem less able tomay be due to variation in the amount of follow gaze in experimental contexts (Andersoninformation that the eye carries due to differ- & Mitchell, 1999; but see Ruiz et al., 2009ences in morphology across different taxa for an experimental study using Conspecific(Kobayashi & Kohshima, 1997, 2001). Photographs). Thus, although behavioral co- Although gaze-following behaviors are orienting may be common to all primates, notwidely shared across the primate order, the psy- all primates necessarily follow gaze because theychological basis of these co-orienting behaviors understand that others see things.seems to vary widely. For example, the nature ofgaze following in chimpanzees and other great Using Information About Gaze andapes suggests that individuals of these species Attentionfollow gaze because they understand somethingabout the nature of ‘‘seeing.’’ Apes not only Further evidence supporting the potential dis-direct their own gaze in the direction of others tinction between apes and other species comesbut also follow gaze around barriers and past from social cuing (or object-choice) studies.distracting objects that are not the target of This paradigm is similar to those involvinganother’s gaze, sometimes by physically reor- pointing gestures, although here the experimen-ienting their own bodies (Povinelli & Eddy, ter’s cue involves looking at the correct option1996a; Tomasello et al., 1999). They may also (Table 7.2). Overall, evidence suggests that apes‘‘check back’’ with the actor in an attempt to are generally successful at spontaneously usingverify the direction of the other’s gaze or quickly gaze cues to find the food, although, like withstop following the gaze cues when they cannot gesture cues, the effects are often small (e.g. Calllocate the target of the other’s gaze (Braeuer et al., 1998; Itakura et al., 1999). Notably, apes’et al., 2005; Call et al., 1998; Tomasello et al., performance may change dramatically2001). These flexible shifts in behavior across depending on the specific paradigm utilized.contexts suggest that apes follow the gaze of For example, chimpanzees are much more suc-others because they expect there to be something cessful using gaze cues when the experimenterinteresting to see. Interestingly, those species looks into an object whose contents he or shemost closely related to humans—chimpanzees alone can see (such as a tube) than when the

124 Table 7.3 Studies of Gaze Following Across Nonhuman Primate Species Check Back with Follow Eye Actor; Habituate Follow Head Orientation Follow Gaze Around in Absence of Orientation Alone Barriers Target Ontogeny Hominoids Chimpanzees Braeuer et al., 2005; Call Herrmann et al., Barth et al., 2005; Braeuer Braeuer et al., 2005; Barth et al., 2005; Braeuer et al., et al., 1998; Herrmann et al., 2007; Povinelli & et al., 2005; Okamoto- Call et al., 1998; 2005; Herrmann et al., 2007; 2007; Itakura, 1991; Eddy, 1996a; Barth et al., 2007; Okamoto-Barth Tomasello & Carpenter, 2005; Okamoto-Barth et al., 2007; Tomasello et al., Povinelli & Eddy, 1996a; et al., 2007; Tomasello et al., 2001; Povinelli & Eddy, 1996a; 2007 Tomasello et al., 1999 Tomasello et al., Tomonage et al., 2004 Tomasello et al., 1998*, 2001 2001, 2007 Other great Bonobos: Bonobos: Bonobos: Bonobos: Bonobos: apes Braeuer et al., 2005; Tomasello et al., Braeuer et al., 2005; Braeuer et al., 2005; Braeuer et al., 2005 Okamoto-Barth et al., 2007; 2007 Okamoto-Barth et al., Okamoto-Barth Gorillas: Tomasello et al., 2007 Gorillas: 2007 et al., 2007 Braeuer et al., 2005 Gorillas: Tomasello et al., Gorillas: Gorillas: Orangutans: Braeuer et al., Braeuer et al., 2005; 2007 Braeuer et al., 2005; Braeuer et al., 2005; 2005; Herrmann et al., 2007 Okamoto-Barth et al., 2007; Orangutans: Okamoto-Barth et al., Okamoto-Barth Tomasello et al., 2007 Herrmann et al., 2007 et al., 2007 Orangutans: Braeuer et al., 2007; Tomasello Orangutans: Braeuer Orangutans: 2005; Herrmann et al., 2007; et al., 2007 et al., 2005; Okamoto- Braeuer et al., 2005; Okamoto-Barth et al., 2007; Barth et al., 2007 Okamoto-Barth Tomasello et al., 2007 et al., 2007 Lesser apes Pileated gibbons: Horton & Caldwell, 2006† (continued)

Old World Macaques Rhesus: Emery et al., 1997*; Rhesus: Tomasello Rhesus: Tomasello et al., 2001 monkeys Itakura, 1996; Tomasello et al., 2001 et al., 1998,* 2001 Long-tailed: Stumptail: Anderson & Goossens et al., Mitchell, 1999; Itakura, 2008 1996; Tomasello et al., 1998* Pigtail: Itakura, 1996; Tomasello et al., 1998* Long-tail: Goossens et al., 2008 Tonkean: Itakura, 1996 Baboons Olive: Vick et al., 2001 Olive: Vick et al., 2001 Other Sooty mangabey: Tomasello et al., 1998* New World Capuchins Brown: Itakura, 1996; Vick Brown: Vick & monkeys & Anderson, 2000 Anderson, 2000 White-faced: Itakura, 1996 Callitrichids Cotton-top tamarins: Cotton-top Neiworth et al., 2002† tamarins: Common marmosets: Neiworth et al., 2002 Burkhart & Heschl, 2006 Other Squirrel monkey: Itakura, 1996 Strepsirrhines Lemurs Ring-tailed: Shepherd & Platt, 2008* Black: Anderson & Mitchell, 1999; Itakura, 1996; Ruiz et al., 2009* Brown: Itakura, 1996; Ruiz et al., 2009* * Indicates that the study involved conspecific social partners; unless noted, the study involved human experimenters as actors. † Indicates that the study involved both human and conspecific social partners.125

126 PRIMATE NEUROETHOLOGYexperimenter just looks at the external surface of a our ability to move our eyes independent of ourcup, an act that is divorced from actually seeing general head direction (Kobayashi & Kohshima,something (Call et al., 1998). In contrast to the 1997, 2001). For example, chimpanzees, bonobos,results with apes, studies with monkeys suggest and orangutans spontaneously adjust their ges-that whereas monkeys will often follow gaze, they ture frequency to the attentional state of thetend not to use gaze as a social cue when searching observer (i.e., they produce more gestures whenfor food in experiments. In many experiments, an experimenter can see them), but they treatboth Old and New World monkeys require exten- body and face orientation, rather than eye posi-sive training with cues or fail to use gaze cues at all tion, as the most relevant factors (Kaminski et al.,(e.g., olive baboons: Vick et al., 2001; rhesus 2004). Chimpanzees do, however, attend tomacaques: Anderson et al., 1996; capuchins: whether an experimenter’s eyes are open whenAnderson et al., 1995; cotton-top tamarins: this is the only cue available (Hostetter et al.,Neiworth et al., 2002; but see Hauser et al., 2007). Furthermore, chimpanzees will adjust the2007, for successful use of gaze cues in rhesus location of their gesture depending on the focusmonkeys). This trend of failures suggests that, in of their partner’s attention (Povinelli et al., 2003),contrast to apes, some monkey species may follow and all four species of great ape will move to facegaze without actually understanding anything an experimenter so that they can execute theirabout the nature of attention and visual percep- gestures in that person’s line of sight, rather thantions. However, as mentioned previously, there is perform the gesture behind his or her back (Liebalsome evidence that modification of the standard et al., 2004b). Similar results have come fromtwo-option object-choice paradigm may improve naturalistic observations of the gestures thatthe performance of some species (e.g., common apes use when interacting with each other; apesmarmosets: Burkhart & Heschl, 2006), so future modulate their gesture use to the attentional stateresearch is warranted with a wider range of spe- of their conspecific partner (Liebal et al., 2004a;cies and paradigms before any strong conclusions Pika et al., 2003, 2005) and may use loud noises tocan be made about a clade-level distinctions attract attention before making visual gesturesbetween apes and monkeys groups. (Call & Tomasello, 2007; Poss et al., 2006). More converging evidence that apes have Although monkey species do not producesome understanding of the nature of visual per- gestures with the flexibility that apes do (Call &ception comes from studies examining their ges- Tomasello, 2007), evidence that other primateture use in response to others who vary in species understand something about the natureattentional state. One such study (Povinelli & of visual perception comes from studies lookingEddy, 1996c, described previously) suggested at how the attentional state of others influencesthat chimpanzees understand very little about the predictions that monkeys make about thethe nature of seeing in an experimental setting. behavior of others after they look at an object.However, other research by these researchers sug- For example, when cotton-top tamarins saw agests that chimpanzees may be sensitive to head human actor look at one of two objects, theymovements and eye contact in similar contexts expected the actor to reach for and grab that(Povinelli & Eddy, 1996b), although it is not clear object rather than another, previously unat-what factors drive this sensitivity. Nonetheless, tended object, demonstrating longer looking atmore recent research has suggested that apes the unexpected outcome (Santos & Hauser,may have performed poorly in these early ges- 1999). Diana monkeys seem to have similarture-use studies because they favor head and body expectations about the directed gaze of conspe-orientation over eye position as cues to what cifics (Scerif et al., 2004), but two other Newothers are seeing (Tomasello et al., 2007). One World monkey species (tufted capuchins andpossibility is that the low degree of contrast squirrel monkeys) fail to demonstrate an under-between the iris and sclera makes it difficult to standing of the link between attention and beha-discriminate eye direction in almost all primates vior at least when tested using an expectancybut humans; humans also appear to be unique in violation looking method (Anderson et al.,

Table 7.4 Studies of Understanding Perceptions and Knowledge Across Nonhuman Primate Species Attention and Visual Perspective Auditory Deception False Beliefs Ontogeny Predictions About Perspective Seeing Hominoids Chimpanzees Herrmann et al., 2007; Braeuer et al., 2007,* ; Braeuer et al., Hare et al., 2006; Call & Tomasello, Herrmann et al., Hostetter et al., 2007; Hare et al., 2000,* 2007*; Melis et al., Hirata & 1999; Hare et al., 2007; Povinelli et al., Kaminski et al., 2004; 2001,* 2006; Hirata & 2006a Matsuzawa, 2001*; 2001*; Kaminski 2002; Reaux et al., Liebal et al., 2004a,* Matsuzawa, 2001*; Melis et al., 2006a et al., 2008; 1999; Tomasello & 2004b; Povinelli & Melis et al., 2006a; Krachun et al., Carpenter, 2005; Eddy, 1996b, 1996c; Povinelli & Eddy, 1996c; 2009 Tomasello et al., Povinelli et al., 1997, Povinelli et al., 1990 1994* 2002, 2003; Reaux et al., 1999; Theall & Povinelli, 1999; Tomasello & Carpenter, 2005; Tomasello et al., 1994* Other great Bonobos: Kaminski Orangutans: Orangutans: apes et al., 2004; Liebal et al., Call & Tomasello, Herrmann et al., 2004b; Pika et al., 2005* 1999 2007 Gorillas: Kaminski et al., 2004; Liebal et al., 2004b; Pika et al., 2003*; Poss et al., 2006 Orangutans: Herrmann et al., 2007; Kaminski et al., 2004; Liebal et al., 2004b; Poss et al., 2006 Lesser apes Siamangs: Liebal et al., 2003* Pileated gibbons: Horton & Caldwell, 2006127 (continued)

128 Table 7.4 (Continued) Attention and Visual Perspective Auditory Deception False Beliefs Ontogeny Predictions About Perspective Seeing Old World Macaques Rhesus: Flombaum & Rhesus: Santos Rhesus: Santos monkeys Santos, 2005; Povinelli et al., 2006 et al., 2007b et al., 1991 Long-tailed: Kummer et al., 1996 Baboons Olive: Vick & Anderson, 2003 Other Diana monkeys: Scerif et al., 2004* New World Capuchins Brown: Anderson Brown: Fujita et al., Brown: Fujita monkeys et al., 2004; Kuroshima 2002*; Hare et al., et al., 2002 * et al., 2002; Kuroshima 2003*; et al; 2003 Callitrichids Cotton-top tamarins: Common marmosets: Santos & Hauser, 1999 Burkhart & Heschl, Common marmosets: 2007* Burkhart & Heschl, 2007* Other Squirrel monkeys: Anderson et al., 2004 Strepsirrhines Lemurs Black: Genty & Roeder, 2006 * Indicates that the study involved conspecific social partners; unless noted, the study involved human experimenters as actors.

PRIMATE SOCIAL COGNITION: THIRTY YEARS AFTER PREMACK AND WOODRUFF 1292004). Thus, whereas there is robust evidence and now a dominant could see both pieces ofthat apes understand something about the food and was released before a subordinate whonature of attention, the results are more variable could only see one, the dominant targeted theacross monkey species, suggesting again the visible (at-risk) piece of food before taking theimportance of the kind of behavioral task and second piece hidden from the subordinate’scontext employed. view. A number of controls ruled out the possibility that such strategies were due to behavioral monitoring of the dominant individual (e.g., theFrom Perspective Taking to Understanding subjects were forced to make a decision beforeof Knowledge and Beliefs they ever saw their competitor make a move;Together, the evidence from gesture use and Hare et al., 2000). A second set of studies indi-looking-time work suggests that apes and pos- cated that chimpanzees demonstrated these pre-sibly some monkeys may be sensitive to the ferences because they understood somethingvisual perception of others. Perhaps the most about the link between seeing and knowing:conclusive evidence that some primates have an When subordinates had to decide whether orunderstanding of visual attention, however, not to approach a piece of food hidden from acomes from studies of perspective taking. As dominant’s view, they made more attempts topreviously mentioned, several primate species obtain the food when the dominant had nottend to perform poorly in early studies testing been present when the food was hidden thantheir understanding of visual attention and per- when she had been present during the baitingspective taking (e.g., Povinelli & Eddy, 1996c; (Hare et al., 2001).Reaux et al., 1999). However, these studies typi- Following these initial studies, several experi-cally used a cooperative-communicative para- ments using competitive paradigms havedigm in which a human experimenter shared demonstrated perspective-taking skills in bothfood with the chimpanzees, a situation that apes and monkeys. For example, Flombaummay be highly unnatural or unmotivating for and Santos (2005) developed a paradigm inprimates, as previously noted. Faced with this which rhesus macaques could choose to stealproblem, researchers have more recently tried to food from one of two experimenters, and thendevelop more ecologically valid tests of perspec- varied the degree to which those experimenterstive taking, ones that are designed around a could see the food. In many ways this setupcontext that may be more natural (and moti- therefore parallels the preferential begging para-vating) for primates: food competition. The digm developed by Povinelli and colleagues,basic setup of the original studies by Hare and except that the decision was placed in a competi-colleagues (2000, 2001) using this logic involved tive context. Rhesus monkeys showed sensitivitytwo chimpanzees competing with each other for to a wide variety of variations in visual access,access to food. However, the two chimpanzees even when the manipulations involved veryhad differing knowledge about the food that was subtle differences in eye position. Studies withavailable. For example, in one series of studies, chimpanzees have similarly shown that theythe more subordinate of the two chimpanzees prefer to retrieve a piece of food that a competi-could see two pieces of food, and the dominant tive human cannot see over one he or she can,individual could only see one (the second piece even engaging in attempts to disguise theirwas blocked from her view). Researchers then interest in the food as they approach it (Haremeasured which piece of food the subordinate et al., 2006; Melis et al., 2006a). Moreover, sometargeted when she was released with a slight head evidence suggests that the perspective taking thatstart over the dominant. Using this technique, a chimpanzees and rhesus monkeys engage inseries of experiments demonstrated that subor- extends to the auditory modality. For example,dinate chimpanzees were more likely to choose when rhesus macaques are confronted with athe food that dominant individuals could not human competitor sitting in front of two boxessee. In addition, when the roles were reversed containing food where one box has functional

130 PRIMATE NEUROETHOLOGYbells attached to it while the other box has non- chimpanzees use information about true but notfunctional bells, they preferentially steal food false beliefs to find food hidden (Kaminski et al.,from the box that is silent, and do so only when 2008; Krachun et al., 2009; see also the informed-the competitor cannot already see their actions misinformed condition in Hare et al., 2001).(Santos et al., 2006). This suggests that rhesus Similarly, when tested in a looking-time viola-monkeys recognize how their behavior will alter tion-of-expectation false-belief test (see Onishithe psychological state of the human: If the & Baillargeon, 2005), rhesus monkeys make cor-human cannot see them, then the noise will alert rect predictions about where a human actor willhim or her to their presence. If the human can search when they have a true belief about a foodalready see them, then noise will have no impact item’s location, but make no predictions abouton the human’s knowledge about their behavior. the actor’s behavior when the actor has a falseChimpanzees also prefer a silent approach over a belief (Santos et al., 2007b). These findings usingnoisy one when competing with a human over various false-belief tasks suggest that while pri-food (Melis et al., 2006a; but see Braeuer et al., mates can represent whether others are knowl-2008). Despite these successes, other monkey spe- edgeable or ignorant, they may not representcies have demonstrated poor performance in beliefs of others in cases where those beliefs con-similar visual perspective-taking tasks, providing flict with the true state of the world.further converging evidence that the psycholo-gical mechanisms supporting social interactions Conclusions: Understanding Psychologicalvary across primates. For example, both capu- Stateschins (Hare et al., 2003) and common marmosets(Burkhart & Heschl, 2007) have been tested in Overall, research from the past decade has greatlyversions of the conspecific competition paradigm illuminated the cognitive skills underlying theused with chimpanzees, but appear to depend complex social behaviors of primates. First, bothheavily on the behavior of the competitor, apes and at least some species of monkey seem torather than reasoning about what the competitor parse the actions of others in terms of underlyingsees or knows, when making food choices. goals and intentions. Similarly, apes and some The current evidence suggests that at least monkeys seem to understand something aboutchimpanzees and rhesus macaques know some- the perspective of others. Most research addressesthing about what others can and cannot perceive, whether primates understand visual perspective,and use this information to guide their own but other studies suggest that this capacity maybehavioral decisions. However, an open question also encompass perception in other modalities.concerns the issue of what primates are actually These social-cognitive abilities may not be sharedrepresenting when faced with these kinds of social by all primates: Although some behaviors, such asproblems. Although recent studies provide strong gaze following, seem to be widely shared, someevidence that rhesus monkeys and chimpanzees species engage in superficially similar behaviorsunderstand something about others’ perception but do not seem to understand the nature ofand knowledge, thus far there is no evidence that seeing in the same way that chimpanzees andprimates go beyond a distinction between knowl- rhesus macaques do. However, the currentedge and ignorance to actually represent the false research supports the conclusion that at leastbeliefs of others. For example, chimpanzees per- some primates understand others’ behavior inform at chance when confronted with two terms of psychological states such as goals andhumans trying to direct them to food, one of knowledge, rather than merely in terms of obser-whom had seen the food being hidden and the vable behavioral features (but see Povinelli &other who had originally seen the food being Vonk, 2003, 2004, for alternative interpretationshidden, but had a false belief about its location of these results).due to a subsequent switching out of her view However, there are, at present, still many(Call and Tomasello, 1999). Competitive versions limitations to our understanding of primateof false-belief tasks further confirm that social cognition. As mentioned previously,

PRIMATE SOCIAL COGNITION: THIRTY YEARS AFTER PREMACK AND WOODRUFF 131many of the paradigms used to examine social- into play in different contexts and differentiallycognitive skills in primates have been adapted impact different kinds of social interactions. Tofrom the human developmental literature. As take one example, wild-living male chimpanzeessuch, the ‘‘interesting’’ topics in primate social engage in several complex cooperative behaviorscognition tend to grow out of developmental (Muller & Mitani, 2005), including meat sharingstudies of theory of mind. As there is some (Mitani & Watts, 2001), group hunting (Boesch &indication that primates look more skillful in Boesch, 1989), coalitionary mate guardingstudies involving ecologically valid paradigms, (Watts, 1998), and territorial boundary patrolssuch as competition for food, directly adapting (Watts & Mitani, 2001). Other primates alsodevelopmental paradigms for primates may not have complex patterns of cooperation and alliancebe the only productive way to study primate formation (e.g., de Waal, 1996; Kappeler & vansocial cognition. Indeed, what these kinds of Schaik, 2006). As such, primates may possessparadigms emphasize is that social-cognitive sophisticated social-cognitive skills to deal withskills are functional, guiding effective behavior both competitive and cooperative interactions,and allowing organisms to choose the most but the kinds of skills they use may be very dif-advantageous course of action. For example, a ferent in these disparate contexts. Indeed, thestudy varying the ‘‘intensity’’ of competition in cooperative-communicative paradigms (such asthe sort of conspecific-competition paradigm object-choice) used so often in primate researchdescribed earlier illustrates how perspective may fail to demonstrate robust social-cognitivetaking is an ability that chimpanzees use strate- abilities in various species not because thesegically. When chimpanzees only have time to tasks cooperative per se, but because they utilizeretrieve one piece of food, perspective taking specific forms of cooperation (sharing informa-increases their payoff—they will therefore tion or sharing food) that may not be a part oftarget the piece that their competitor cannot species-typical social interactions. In fact, studiessee. However, if the physical properties of the of human cooperation suggest many ways totask are altered such that chimpanzees can approach the problem in nonhuman primatespotentially retrieve all the food regardless of that might lead to a better understanding ofwhat their competitor can see, they will simply breadth of possible social-cognitive skills beyonduse a ‘‘fast’’ strategy and race to take both pieces the competitive contexts studied thus far. Awhile choosing indiscriminately (Braeuer et al., variety of social-cognitive skills play important2007; see also Karin-D’Arcy & Povinelli, 2002). roles in shaping human cooperation, includingThis finding emphasizes the importance of knowledge about the intentions of others, theexamining primate social-cognitive skills in a social relationship between cooperative partners,functional framework. Researchers will therefore and reputation management (see reviews inprofit from critically considering the kinds of Gintis et al., 2005).skills that might allow primates to be more effec- Do similar social-cognitive mechanismstive social decision makers in their natural envir- underlie the cooperative behaviors of nonhumanonments, and when it actually benefits them to primates? Increasing evidence suggests that theyuse the skills they possess. do, at least in some species and contexts. For example, apes appear to have some knowledge of the quality of the relationships they shareFROM COMPETITION TO with social partners as well as being able toCOOPERATION remember how those partners behaved in pastCompetition is just one example of an ecologically cooperative interactions. Chimpanzees will spon-relevant domain—primates certainly do not taneously cooperate to acquire food in an instru-spend all their time competing with others for mental task requiring joint action withfood! Rather, primate social life is a complex conspecifics that they share a tolerant relationshippatchwork of both competition and coopera- with, but will not cooperate with intoleranttion—but these two opposing forces may come partners (Melis et al., 2006b; see Hare et al.,

132 PRIMATE NEUROETHOLOGY2007, for a comparison of chimpanzees and Chimpanzee’s lack of communication in a nego-bonobos in a similar task), and will also prefer- tiation game). The lack of communication mayentially choose to cooperate with more skillful suggest that there is a lack of motivation by non-partners over less skillful partners (Melis et al., human primates to assess the cooperative-com-2006c). Correlations of natural behaviors further municative intent of others.suggest that chimpanzees prefer to cooperate with Altogether, these results suggest that manythose who have cooperated with them in the past primates do engage in ‘‘cognitive’’ cooperation,(Mitani, 2006). Although there are several very using their social-cognitive skills to engage indifferent types of mechanisms that could underlie more efficient and more successful forms ofsuch behaviors (see de Waal & Lutrell, 1988), cooperation. Critically, the payoffs of manyexperimental evidence supports the hypothesis cooperative interactions depend not only onthat chimpanzees show calculated reciprocity in whether two individuals act together but also ongrooming (Koyama et al., 2006) and collaborative their level of skill when performing the act. That(Melis et al., 2008) contexts. Together, these is, if one partner cannot successfully perform hisresults suggest that apes remember something or her role, both members of the pair will fail toabout the behavior of others and use this infor- get anything. The, use of social-cognitive skillsmation when making social decisions—that is, can increase the rewards associated with coopera-they are guided by something like direct reputation, so it can pay for individuals to sustaintion when deciding who to interact with (See also relationships with potential partners, selectivelySubiaul et al., 2008). But primate social-cognitive cooperate with good partners, and be adept at theskills are not limited to direct interactions with mutualistic activity itself (e.g., coordinate withothers: Some primates also seem to represent the the partner and be sensitive to the other’s inten-ongoing relations of other members of their tions). Thus, future studies of the social-cognitivegroups. For example, experiments with wild abilities that primates use in cooperative interac-baboons suggest that this species understands tions will likely reveal that these abilities arenot only their relations with others but also the different than those needed in cooperative inter-third-party relationships between other members actions, but not any less complex.of their groups (see Cheney & Seyfarth, 2007), anability that may function as a precursor to THE EVOLUTION OF PRIMATEindirect reputation formation. SOCIAL COGNITION There is also evidence that primates use social-cognitive skills such as intention reading in coop- Despite the major inroads that research examiningerative contexts. For example, chimpanzees’ use primate social cognition have made in the lasttheir understanding of both humans’ and conspe- decade, there are still some major limitations tocifics’ goals to help them when they fail to reach current research. First of all, although we began bythose goals (Warneken & Tomasello, 2006; asserting that we would review the cognitive skillsWarneken et al., 2007). Chimpanzees also use that primates use during social interactions, it isinformation about whether a conspecific was the notable that the vast majority of studies we havecause of their losing access to food when deciding reviewed involve primates interacting withwhether to punish that individual (Jensen et al., humans (Tables 7.1 through 7.4; studies that2007), which may involve some form of intention involve conspecifics are marked studies withreading. If this is so, chimpanzees then possess an human partners are unmarked). Consequently,ability thought to be an important mechanism for while we know a lot about the cognitive skillssustaining cooperation across repeated interac- primates can utilize, we are less sure about whentions. Notably, however, there is little evidence and how primates actually use these skills whenfrom any of this work that chimpanzees or other interacting with conspecifics in natural contexts.species understand the potential of using overt Similarly, few studies have examined the ontoge-forms of communication to enhance success in netic development of these skills (Tables 7.1cooperative endeavors (See Melis et al., 2009 for through 7.4; the last column in each category

PRIMATE SOCIAL COGNITION: THIRTY YEARS AFTER PREMACK AND WOODRUFF 133references studies with a developmental compo- techniques in evolutionary biology (Mayr, 1982).nent). Developmental studies have provided cri- The comparative method allows us to reconstructtical insights into human social cognition, so they a phenomenon (evolution via natural selection)could potentially do the same for nonhuman pri- that often cannot be directly observed, and there-mate social cognition. For example, divergent fore address not just what the differences aredevelopmental trajectories may be evidence of dif- between different groups of organisms, but alsoferent underlying psychological mechanisms why those differences arose. Consequently, it mayacross species, even when adult behaviors appear be the most powerful technique we have to answersimilar (e.g., see Tomasello et al., 2001 for a devel- functional questions about social-cognitive abil-opmental comparison of rhesus and chimpanzees). ities across primate taxa.But perhaps the most salient limitation of current One such approach is to test closely relatedresearch into primate social cognition is the one species on a battery of tests that can be used toeasiest to remedy: whereas almost every major identify whole suites of shared and derived traitscategory of social-cognitive research has several across different domains of cognition (e.g.,studies examining that ability in chimpanzees, the Herrmann et al., 2007). However, several moreexisting data across other taxa are more patchy— specific hypotheses about the role of social andwith only one or two relevant studies—and often ecological factors in the evolution of particularnonexistent (see Tables 7.1 through 7.4; the first cognitive abilities can also be addressed withrow lists studies with chimpanzees). This missing comparative data. For example, one predictionevidence becomes all the more striking for tasks of the competition hypothesis described earlierthat do not involve gaze-following paradigms. The is that there will be critical differences betweenconsequence of this imbalance is that most of what the social-cognitive skills of more despotic,we know about ‘‘primate social cognition’’ is really aggressive species compared to more egalitarian,‘‘chimpanzee social cognition.’’ This paucity of tolerant species. Specifically, as despotic speciesdata on the social-cognitive skills of the vast face more intense competition for food, as wellmajority of the Primate Order makes it difficult as a steeper dominance hierarchy limiting theirto draw any broad conclusions about either the access to that food (de Waal & Lutrell, 1989),social-cognitive skills of nonhuman primates or they may more readily show sophisticatedthe evolutionary pressures shaping these skills. social-cognitive skills when competing withConsequently, many empirical tests of these others. Conversely, more egalitarian speciesmodels involve very rough quantifications of intel- might show greater skills in cooperative contextsligence via morphological correlates such as brain (e.g., Hare et al., 2007; Petit et al., 1992).size (Dunbar, 1992), making it difficult to assess the Notably, the two species that have been success-very evolutionary hypotheses that originally fully studied using competitive paradigms—spurred interest in primate social cognition. rhesus macaques and chimpanzees—are bothHowever, several new approaches to the study of more despotic than closely related egalitarianprimate social cognition—including comparisons sister species. Thus, comparing the social-cogni-between closely related species and studies of con- tive skills of chimpanzees and rhesus to bonobosvergence with other taxa—have begun to tackle and Tonkean macaques, respectively, in a foodthis problem. competition paradigm would be helpful and could provide a direct test of this hypothesis. This kind of framework raises additional issuesThe Comparative Method: Identifying about evolutionary interpretations of the compara-the Forces Shaping Social Cognition tive data that we do have. For example, recentThe comparative method—examining the traits of studies have indicated that many of the social-different populations or species that have been cognitive abilities identified in chimpanzees, suchshaped by differing ecological or social forces in as perspective taking and intention reading, areorder to better understand how natural selection also present in more distantly related monkeysproceeded—is one of the most important such as rhesus macaques and capuchins. One

134 PRIMATE NEUROETHOLOGYinterpretation of these data are that such mechan- For example, primates seem to be relativelyisms are quite evolutionarily ancient, extending unskilled at interpreting communicative beha-back to approximately 40 Mya (Steiper & Young, vior—making it difficult to assess how such2006) when the primate lineage leading to New abilities arose in humans. Consequently, someWorld monkeys such as capuchins split from the researchers have begun to use dogs and wolves aslineage leading to Old World monkeys and apes. helpful models for understanding the evolutionHowever, another possibility is that these similar of communicative gestures such as pointing andbehaviors actually represent instances of social- gaze cues. Whereas wolves are not very suc-cognitive convergence, or parallel evolution, in cessful at using pointing or gaze cues in thedifferent lineages. Capuchins—who engage in absence of extensive experience with humans,both sophisticated tool use (Visalberghi, 1990; see dogs from a very young age appear to be highlyChapter 29), and hunting behaviors (Rose, tuned to human communication, following such1997)—are often considered behaviorally conver- cues spontaneously (Hare et al., 2002; Riedelgent with chimpanzees (Fragaszy et al., 2004). et al., 2007; Viranyi et al., 2008). These differ-Similarly, if food competition is a critical selective ences suggest that the changes that occurredforce driving the evolution of perspective taking, during domestication may be important forthen rhesus—with their highly despotic social some kinds of social cognition, and many psy-system (de Waal & Lutrell, 1989)—might also chological mechanisms have been proposed forrepresent a case of convergence. However, such the behavioral changes that resulted from thisinstances of possible convergence are certainly not selection, including increased attention to facesa problem for studies of social cognition—in fact, of humans (Miklosi et al., 2003) and reducedthey provide a critical method for testing how and fear responses (Hare & Tomasello, 2005).why these abilities evolve. Indeed, some of the Studies of other domesticated species, such asstrongest tests of the evolutionary forces driving an experimental population of domesticatedsocial-cognitive evolution comes from outside the foxes (Hare et al., 2005), domestic goatsprimates. (Kaminski et al., 2005, 2006b), and cats (Miklosi, 2003) further support the possibility that domestication can influence some forms ofUsing Convergence in Other Taxa as social-cognitive abilities. These findings suggesta Model for Primate Evolution that interpersonal tolerance may be a criticalStudies of social-cognitive evolution in primates prerequisite for some kinds of human-likeface two major problems: Often the critical taxa social-cognitive skills, particularly thoseare extinct (e.g., we cannot compare humans to involving cooperation (Melis et al., 2006b).other hominid species to identify uniquely Studies of convergence can also illuminate thehuman cognitive traits) or most primates share evolution of social traits that likely emerged inthe feature in question (e.g., most anthropoids are basal primate groups, such as in catarrhines, andhighly social to some degree, so it is difficult to thus are widely shared across large taxonomicuse monkeys to address coarse-grained evolu- spaces. For example, wild spotted hyenastionary questions about how the presence or (Crocutta crocutta) live in large social groupsabsence of sociality impacts social cognition). with Old World primate–like linear dominanceLuckily, evolution has provided an alternative hierarchies and engage in cooperative huntingroute—studies of convergence in other taxa can behaviors (Holekamp et al., 2007). This suggestsoften remedy these kinds of difficulties that arise that these social mammals may possess sophisti-when looking within primates. Such studies also cated social-cognitive skills to deal with theirprovide a critical check to primate-centric views social landscape much like those observed inof social-cognitive evolution, as some ‘‘general’’ some monkey species (e.g., Drea & Carter,principles of social-cognitive evolution do not 2009). As spotted hyenas have two closely relatedseem to hold up very well in other taxa (e.g., relatives with significant variation in their socialDunbar & Schultz, 2007). structure—striped hyenas (Hyaena hyaena)

PRIMATE SOCIAL COGNITION: THIRTY YEARS AFTER PREMACK AND WOODRUFF 135appear to be solitary, and brown hyenas difficulty’’ facing anyone interested in the evolu-(Parahyaena brunnea) live in smaller, less gregar- tion of human social cognition. Namely, whatious social groups (Watts & Holekamp, 2007)— aspects of primate social cognition are trulycomparative studies of these species with an eye to unique to our own species? In recent years, pri-variation in their natural ecologies could illumi- mate researchers have gained some new tractionnate why such complex abilities emerge. on this question. Recent findings using more Arguably, the most sophisticated social- ecologically relevant tasks have led to a growingcognitive skills are actually found outside mam- consensus that humans and at least some othermals—in corvids, a taxa that includes jays, ravens, primates share the capacity to represent theand crows. Studies of these birds have revealed intentions, perceptions, and knowledge ofstartling parallels with the abilities of primates others. Thus, several new or more specific(Emery & Clayton, 2004). Specifically, corvids hypotheses have arisen that attempt to pinpointappear to use many primate-like social-cognitive the major social-cognitive differences betweenskills (such as perspective taking) to protect their humans and other primates. For example, therefood stores when they engage in caching behaviors. is currently little evidence that primates shareFor example, ravens and jays employ protective the capacity to reason about others’ beliefstrategies when they cache (Emery & Clayton, states; indeed, there is some evidence that pri-2001), and seem to use information about the mates fail to reason about others’ belief statesvisual perspective of others when doing so even when tested using a variety of different(Bugnyar & Kotrschal, 2002; Dally et al., 2004; methodologies (Call & Tomasello, 1999;Heinrich & Pepper, 1998). Furthermore, they not Kaminski et al., 2006a; Krachun et al., 2007;only respond to the behavior of competitors but Santos et al., 2007b)—which suggests that repre-also seem to differentiate between some kinds of senting others’ beliefs might be a capacity lim-knowledge states, much like chimpanzees and ited to our own species (e.g., Povinelli &rhesus macaques (Bugnyar & Heinrich, 2005; Giambrone, 2001). Other proposals haveDally et al., 2006). Ravens even appear to predict focused on other aspects of intentionality, suchhow humans will behave in a caching context as the ability to represent (and the motivation tobased on their past interactions with the humans share) joint goals and shared intentionsin a noncaching context, suggesting they represent (Tomasello et al., 2005). This proposal high-the ‘‘reputation’’ of social partners (Bugnyar et al., lights that many human-unique behaviors,2007). Some corvids can even make social infer- such as participation in cultural endeavors, areences from watching third-party interactions fundamentally collaborative in nature. Although ˜(Paz-y-Mino et al., 2004), suggesting that some the available work to date suggests that apesof their social-cognitive skills are also employed perform very differently than human childrenoutside of caching contexts. Taken together, com- on collaborative tasks with shared goals (e.g.,parative work examining social cognition in other Tomasello et al., 2005), more work is needed totaxa makes it clear that a complete understanding directly test both this hypothesis and the beliefof the evolutionary pressures that led to the devel- representation hypothesis. Indeed, such workopment of primate social cognition will require a will allow us to not only gain insight into socio-more thorough understanding of the mechanisms cognitive capacities that might be unique toin similarly sophisticated social cognition in humans but also discover why these purportedlydistantly related taxa as well. unique capacities evolved in the first place.Human Evolution and Social Cognition ConclusionsA final limitation of present work on the nature The past decade has produced significantof primate social cognition involves what is pos- advances in our understanding of primatesibly the toughest question of all— the question social cognition. The development of novelthat Darwin (1871) defined as ‘‘the greatest experimental methodologies has led to

136 PRIMATE NEUROETHOLOGYincreasing evidence that some primates can sciureus) and capuchin monkeys (Cebus apella)assess the psychological states of others in predict that looking leads to touching? Animalsome contexts. Thus, while human social-cog- Cognition, 7, 185–192.nitive abilities may still be outstanding, they Anderson, J. R., & Mitchell, R. W. (1999).nonetheless appear to have deep evolutionary Macaques but not lemurs co-orient visually with humans. Folia Primatologica, 70,roots. However, researchers still have a multi- 17–22.tude of fascinating questions to attack in the Anderson, J. R., Montant, M., & Schmitt, D.future, as research has suggested that even very (1996). Rhesus monkeys fail to use gazesuperficially similar social behaviors (such as direction as an experimenter-given cue in angaze following) can be supported by very dif- object-choice task. Behavioral Processes, 37,ferent underlying psychologies. The question 47–55.has therefore shifted from not just if the sophis- Anderson, J. R., Sallabery, P., & Barbier, H. (1995).ticated social behaviors of primates are the con- Use of experimenter-given cues during object-sequence of sophisticated cognitive skills, but choice tasks by capuchin monkeys. Animalwhy they might be so. With increasing com- Behaviour, 49, 201.parative data, researchers can begin to address Barth, J., Reaux, J. E., D.J. Povinelli (2005).the ultimate causes that shape social cognition Chimpanzees’ (Pan troglodytes) use of gaze cues in object-choice tasks: different methods yieldin both human and nonhuman primates. different results. Animal Cognition, 8, 84–92.Armed with a new appreciation of the impor- Behne, T., Carpenter, M., Call, J., & Tomasello, M.tance of ecologically relevant tasks that can be (2005). Unwilling versus unable: Infants’used across species, the stage is now set for understanding of intentional action.primate cognition researchers to answer Developmental Psychology, 41, 492–499.Darwin’s question. Boesch, C., & Boesch, H. (1989). Hunting behavior of wild chimpanzees in the Tai National Park. American Journal of Physical Anthropology, 78,ACKNOWLEDGMENTS 547–573.We thank Felix Warneken for comments on an Braeuer, J., Call, J., & Tomasello, M. (2005). Allearlier draft of the manuscript. great ape species follow gaze to distant locations and around barriers. Journal of Comparative Psychology, 119, 145–154. Braeuer, J., Call, J., & Tomasello, M. (2007).NOTE Chimpanzees really know what others can see in a competitive situation. Animal Cognition,1. Note that a complete review of the vast literature 10, 439–448. on social learning in primates is outside the Braeuer, J., Call, J., & Tomasello, M. (2008). scope of this chapter (see Tomasello & Call, Chimpanzees do not take into account what 1997, for a review of this extensive work). others can hear in a competitive situation.2. Interestingly, chimpanzees goal emulation Animal Cognition, 11, 175–178. differs from the performance of children in Braeuer, J., Kaminski, J., Riedel, J., Call, J., & this task, who faithfully imitate all of the Tomasello, M. (2006). Making inferences actions of a human actor even when some of about the location of hidden food: Social dog, those actions are clearly irrelevant to obtaining causal ape. Journal of Comparative Psychology, the goal (e.g., Gergely, et al., 2002; Horner & 120, 38–47. Whiten, 2005; Meltzoff, 1995; Nagell et al., 1993; Brooks, R., & Meltzoff, A. N. (2002). The see Lyons & Keil, 2007, for a discussion of this importance of the eyes: How infants interpret species difference). adult looking behavior. Developmental Psychology, 38, 958–966. Bugnyar, T., & Heinrich, B. (2005). Ravens, CorvusREFERENCES corax, differentiate between knowledgeable andAnderson, J. R., Kuroshima, H., Kuwahata, H., & ignorant competitors. Proceedings of the Royal Fujita, K. (2004). Do squirrel monkeys (Saimiri Society B, 272, 1641–1646.

PRIMATE SOCIAL COGNITION: THIRTY YEARS AFTER PREMACK AND WOODRUFF 137Bugnyar, T., & Kotrschal, K. (2002). Observation Carpenter, M., Akhtar, N., & Tomasello, M. (1998). learning and the raitding of food caches in Fourteen- through 18-month old infants ravens, Corvus corax: Is it ’tactical’ deception. differentially imitate intention and accidental Animal Behaviour, 64, 185–195. actions. Infant Behavior and Development, 21,Bugnyar, T., Schwab, C., Schloegl, C., Kotrschal, K., & 315–330. Heinrich, B. (2007). Ravens judge competitors Cheney, D. L., & Seyfarth, R. M. (1990). How through experience with play caching. Current monkeys see the world. Chicago: Chicago Biology, 17, 1–5. University Press.Burkhart, J., & Heschl, A. (2006). Geometrical gaze Cheney, D. L., & Seyfarth, R. M. (2007). Baboon following in common marmosets (Callithrix metaphysics: The evolution of a social mind. jacchus). Journal of Comparative Psychology, Chicago: University of Chicago Press. 120, 120–130. Dally, J. M., Emery, N. J., & Clayton, N. S. (2004).Burkhart, J., & Heschl, A. (2007). Understanding Cache protection strategies by western scrub- visual access in common marmosets, Callithix jays (Aphelocoma californica): Hiding food in jacchus: Perspective taking or behavior reading? the shade. Proceedings of the Royal Society B, 271, Animal Behaviour, 73, 457–469. S387–S390.Buttleman, D., Carpenter, J., & Tomasello, M. Dally, J. M., Emery, N. J., & Clayton, N. S. (2006). (2007). Enculturerated chimpanzees imitate Food-caching Western scrub jays keep track of rationally. Developmental Science, 10, F31–F38. who was watching when. Science, 312, 1662–1665.Byrne, R. W. W. (1988). Machiavellian intelligence: Darwin, C. (1871). The Descent of Man. Appleton Social expertise and the evolution of intellect in and company: New York. monkeys, apes, and humans. Oxford: Oxford de Waal, F. B. M. (1982). Chimpanzee politics: University Press. Power and sex among apes. Baltimore: JohnCall, J., Agnetta, B., & Tomasello, M. (2000). Cues Hopkins University Press. that chimpanzees do and do not use to find de Waal, F. B. M. (1996). Good natured: The origins hidden objects. Animal Cognition, 3, 23–34. of right and wrong in humans and other animals.Call, J., Carpenter, M., & Tomasello, M. (2005). Cambridge, MA: Harvard University Press. Copy results and copying actions in the de Waal, F. B. M., & Lutrell, L. M. (1988). process of social learning: chimpanzee (Pan Mechanisms of social reciprocity in three troglodytes) and human children (Homo primate species: Symmetrical relationship sapiens). Animal Cognition, 8, 151–163. characteristics or cognition? Ethology andCall, J., Hare, B., Carpenter, M., & Tomasello, M. Sociobiololgy, 9, 101–118. (2004). ‘Unwilling’ versus ‘unable’: Chimpanzees’ de Waal, F. B. M., & Lutrell, L. M. (1989). Toward a understanding of human intentional action. comparative socioecology of the genus Macaca: Developmental Science, 7, 488–498. Different dominance styles in rhesus andCall, J., Hare, B., & Tomasello, M. (1998). stumptail monkeys. American Journal of Chimpanzees gaze following in an object- Primatology, 19, 83–109. choice task. Animal Cognition, 1, 89–99. Drea, C. M., & Carter, A. N. (2009). CooperativeCall, J., & Tomasello, M. (1998). Distinguishing problem solving in a social carnivore. Animal intentional actions in orangutans (Pongo Behaviour. doi:10.1016/j.anbehav.2009.06.030 pygmaeus), chimpanzees (Pan troglodytes), Dunbar, R. I. M. (1992). Neocortex size as a and human children (Homo sapiens). Journal constraint on group size in primates. Journal of Comparative Psychology, 112, 192–206. of Human Evolution, 20, 469–493.Call, J., & Tomasello, M. (1999). A nonverbal false Dunbar, R. I. M. (2003). The social brain: Mind, belief task: The performance of children and language, and society in evolutionary great apes. Child Development, 70, 381–395. perspective. Annual Review of Anthropology,Call, J., & Tomasello, M. (2007). The gestural 32, 163–181. communication of apes and monkeys. London: Dunbar, R. I. M., & Schultz, S. (2007). Evolution in Erlbaum. the social brain. Science, 317, 1344.Call, J., & Tomasello, M. (2008). Does the Emery, N. J., & Clayton, N. S. (2001). Effects of chimpanzee have a theory of mind? 30 experience and social context on prospective years later. Trends in Cognitive Sciences, 12, caching strategies by scrub jays. Nature, 414, 187–192. 443–446.

138 PRIMATE NEUROETHOLOGYEmery, N. J., & Clayton, N. S. (2004). The mentality Hare, B., Brown, M., Williamson, C., & Tomasello, of crows: Convergent evolution of intelligence M. (2002). The domestication of social in corvids and apes. Science, 306, 1903–1907. cognition in dogs. Science, 298, 1634–1636.Emery, N. J., Lorincz, E. N., Perrett, D. I., Hare, B., Call, J., Agnetta, B., & Tomasello, M. Oram, M. W., & Baker, C. I. (1997). Gaze (2000). Chimpanzees know what conspecifics following and joint attention in rhesus do and do not see. Animal Behaviour, 59, monkeys (Macaca mulatta). Journal of 771–785. Comparative Psychology, 111, 286–293. Hare, B., Call, J., & Tomasello, M. (2001). DoFlavell, J. H. (1992). Perspectives on perspective- chimpanzees know what conspecifics know? taking. In: H. Beilin & P. B. Pufall (Eds.), Animal Behaviour, 61, 139–151. Piaget’s theory: Prospects and possibilities Hare, B., Call, J., & Tomasello, M. (2006). (pp. 107–139). Hillsdale, NJ: Erlbaum. Chimpanzees deceive a human competitor byFlombaum, J. I., & Santos, S. (2005). Rhesus hiding. Cognition, 101, 495–514. monkeys attribute perceptions to others. Hare, B., Melis, A. P., Woods, V., Hastings, S., & Current Biology, 15, 447–452. Wrangham, R. (2007). Tolerance allowsFragaszy, D. M., Visalberghi, E., & Fedigan, L. M. bonobos to outperform chimpanzees on a (2004). The complete capuchin: The Biology of cooperative task. Current Biology, 17, 619–623. the genus Cebus. Cambridge: Cambridge Hare, B., Plyusnina, I., Ignacio, N., Schepina, O., University Press. Stepika, A., Wrangham, R., et al. (2005). SocialFujita, K., Kuroshima, H., & Masuda, T. (2002). Do cognitive evolution in captive foxes in a tufted capuchin monkeys (Cabus apella) correlated by-product of experimental spontaneously deceive opponents? A domestication Current Biology, 15, 226–230. preliminary analysis of an experimental food- Hare, B., & Tomasello, M. (2004). Chimpanzees are competition contest between monkeys. Animal more skillful in competitive than cooperative Cognition, 5, 19–25. cognitive tasks. Animal Behaviour, 68, 571–581.Genty, E., & Roeder, J. J. (2006). Can lemurs learn Hare, B., & Tomasello, M. (2005). Human-like to deceive? A study in the black lemur. Journal of social skills in dogs? Trends in Cognitive Experimental Psychology: Animal Behavior Sciences, 9, 439–444. Processes, 32, 196–200. Hauser, M. D., Glynn, D., & Wood, J. (2007).Gergely, G., Bekkering, H., & Kiraly, I. (2002). Rhesus monkeys correctly read the goal Rational imitation in preverbal infants. relevant gesture of a human agent. Proceedings Nature, 415, 755. of the Royal Society B, 274, 1913–1918.Gergely, G., Nadasdy, Z., Csibra, G., & Biro, S. Heinrich, B., & Pepper, J. W. (1998). Influence of (1995). Taking the intentional stance at 12 competitors on caching behaviour in the months of age. Cognition, 56, 165–193. common raven, Corvus corax. AnimalGintis, H., Bowles, S., Boyd, R., & Fehr, E. (2005). Behaviour, 56, 1083–1090. Moral sentiments and material interests: On the Herrmann, E., Call, J., Hernadez-Lloreda, M. V., foundations of cooperation in economic life. Hare, B., & Tomasello, M. (2007). Humans Cambridge: MIT Press. have evolved specialized skills of socialGoossens, B. M. A., Dekleva, M., Reader, S. M., cognition: The cultural intelligence hypothesis. Sterck, E. H. M., & Bolhuis, J. J. (2008). Science, 317, 1360–1366. Gaze following in monkeys is modulated by Herrmann, E., Melis, A. P., & Tomasello, M. observed facial expression. Animal Behaviour, (2006). Apes’ use of iconic cues in the object- 75, 1673–1681. choice task. Animal Cognition, 9, 118–130 Hare, B. (2001). Can competitive paradigms Herrmann, E., & Tomasello, M. (2006). Apes’ and increase the validity of experiments on children’s understanding of cooperative and primate social cognition? Animal Cognition, competitive motives in a communicative 4, 269–280. situation. Developmental Science, 9, 518–529.Hare, B., Addessi, E., Call, J., Tomasello, M., & Heyes, C. M. (1998). Theory of mind in nonhuman Visalberghi, E. (2003). Do capuchin primates. Behavioral and Brain Sciences, 21, monkeys, Cebus apella, know what 101–148. conspecifics do and do not see? Animal Kuroshima, H., Fujita, K., Adachi, I., Iwata, K., & Behaviour, 65, 131–142. Fuyuka, A. (2003). A capuchin monkey (Cebus

PRIMATE SOCIAL COGNITION: THIRTY YEARS AFTER PREMACK AND WOODRUFF 139 apella) recognizes when people do and do not Kaminski, J., Call, J., & Tomasello, M. (2008). know the location of food. Animal Cognition, 6., Chimpanzees know what others know, but not 283–291. what they believe. Cognition, 109, 224–234.Hirata, S., & Matsuzawa, T. (2001). Tactics to Kaminski, J., Call, J., & Tomasello, M. (2006b). obtain a hidden food item in chimpanzee Goats’ behaviour in a competitive food pairs (Pan troglodytes). Animal Cognition, 4, paradigm: Evidence for perspective taking? 285–295. Behaviour, 143, 1341–1356.Holekamp, K. E., Sakai, S. T., & Lundrigan, B. L. Kaminski, J., Riedel, J., Call, J., & Tomasello, M. (2007). Social intelligence in the spotted hyena (2005). Domestic goats, Capra hircus, follow (Crocuta crocuta). Philosophical Transactions of gaze direction and use social cues in an object the Royal Society B, 362, 523–538. choice task. Animal Behavior, 69, 11–18.Horner, V., & Whiten, A. (2005). Causal Kappeler, P., & van Schaik, C. P. (2006). knowledge and imitation/emulation Cooperation in primates and humans: switching in chimpanzees (Pan troglodytes) Mechanisms and evolution. Berlin: Springer- and children (Homo saiens). Animal Verlag. Cognition, 8, 164–181. Karin-D’Arcy, R., & Povinelli, R. J. (2002). DoHorton, K. E., & Caldwell, C. A. (2006). Visual co- chimpanzees know what each other see? A orientation and expectations about attentional closer look. International Journal of orientation in pileated gibbons (Hylobytes Comparative Psychology, 15, 21. pileatus). Behavioral Processes, 72, 65–73. Kobayashi, H., & Kohshima, S. (1997). UniqueHostetter, A. B., Rseel, J. L., Freeman, H., & morphology of the human eye. Nature, 387., Hopkins, W. D. (2007). Now you see me, now 767–768. you don’t: Evidence that chimpanzees Kobayashi, H., & Kohshima, S. (2001). Unique understand the role of the eyes in attention. morphology of the human eye and its adaptive Animal Cognition, 10, 55–62. meaning: Comparative studies on externalHumphrey, N. K. (1976). The social function of morphology of the primate eye. Journal of intellect. In: P. P. G. Bateson & R. A. Hinde Human Evolution, 40, 419–435. (Eds.), Growing points in ethology (pp. 303– Koyama, N. F., Caws, C., & Aureli, F. (2006). 317). Cambridge: Cambridge University Press. Interchange of grooming and agonisticInoue, Y., Inoue, E., & Itakura, I. (2004). Use of support in chimpanzees. International Journal experimenter-given directional cues by a young of Primatology, 27, 1293–1309. white-handed gibbon (Hylobytes lar). Japanese Krachun, C., Carpenter, M., Call, J., & Tomasello, M. Psychological Research, 46, 262–267. (2009). A competitive nonverbal false belief taskItakura, S. (1991). An exploratory study of gaze- for children and apes. Developmental Science, 12: monitoring in nonhuman primates. Japanese 521–535. Psychological Research, 38, 174–180. Kummer, H., Anzenberger, G., & Hemelrijk, C. K.Itakura, S. (1996). An exploratory study of gaze- (1996). Hiding and perspective taking in long- monitoring in nonhuman primates. Japanese tailed macaques (Maccaca fasicularis). Journal Psychological Research, 38, 174–180. of Comparative Psychology, 110, 97–102.Itakura, S., Agnetta, B., Hare, B., & Tomasello, M. Kuroshima, H., Fujita, H., Fuyuki, A., & Masuda, (1999). Chimpanzees use of human and T. (2002). Understanding the relationship conspecific social cues to locate hidden food. between seeing and knowing by tufted Developmental Science, 2, 448–456. capuchin monkeys (Cebus apella). AnimalJensen, K., Call, J., & Tomasello, M. (2007). Cognition, 5, 41–48. Chimpanzees are vengeful but not spiteful. Kuroshima, H., Kuwahata, H., & Fujita, K. (2008). Proceedings of the National Academy of Learning from others’ mistakes in capuchin Sciences, 104, 13046. monkeys (Cebus apella). Animal Cognition, 11,Jolly, A. (1966). Lemur social behavior and primate 599–609. intelligence. Science, 153, 501–506. Liebal, K., Call, J., & Tomasello, M. (2004a). Use ofKaminski, J., Call, J., & Tomasello, M. (2004). Body gesture sequences in chimpanzees. American orientation and face orientation: Two factors Journal of Primatology, 64, 377–396. controlling ape’ begging behavior from Liebal, K., Pika, S., Call, J., & Tomasello, M. humans. Animal Cognition, 7, 216–223. (2004b) To move or not to move: How apes

140 PRIMATE NEUROETHOLOGY adjust to the attentional state of others. Mitani, J. C. (2006). Reciprocal exchange in Interaction Studies, 5., 199–219. chimpanzees and other primates. In: P. M.Liebal, K., Pika, S., & Tomasello, M. (2003). Social Kappeler & C. P. Schaik (Eds.), Cooperation in communication in siamangs (Symphalangus primates and humans. Mechanisms and syndactylus): Use of gestures and facial evolution. Berlin: Springer-Verlag. expressions. Primates, 45, 41–57. Mitani, J. C., & Watts, D. P. (2001). Why doLyons, D. E., & Santos, L. R. (2006). Ecology, chimpanzees hunt and share meat? Animal domain specifics, and the origins of theory of Behaviour, 61, 915–924. mind: Is competition the catalyst? Philosophy Moll, H., & Tomasello, M. (2004). 12- and 18- Compass, 1, 481–492. month-olf infants follow gaze to spaces behindLyons, D., & Santos, L. R. (Submitted). Capuchin barriers. Developmental Science, 7, F1–F9. monkeys (Cebus apella) discriminate between Muller, M. N., & Mitani, J. C. (2005). Conflict and intentional and unintentional human actions. cooperation in wild chimpanzees. Advances in Developmental Psychology. the Study of Behavior, 35, 275–331.Lyons, D. E., Young, A. G., & Keil, F. C. (2007). The Myowa-Yamakoshi, M., & Matsuzawa, T. (2000). hidden structure of overimitation. Proceeding of Imitation of intentional manipulatory actions the National Academy of Sciences, 104, 19751– in chimpanzees (Pan troglodytes). Journal of 19756. Comparative Psychology, 114, 381–391.Mayr, E. (1982). The growth of biological thought. Nagell, K., Olguin, R., Tomasello, M. (1993). Cambridge, MA: Harvard University Press. Processes of social learning in the tool use ofMelis, A., Call, J., & Tomasello, M. (2006a). chimpanyees (Pan troglodytes) and human Chimpanzees conceal visual and auditory children (Homo Sapiens). Journal of information from others. Journal of Comparative Psychology, 107, 174–186. Comparative Psychology, 120, 154–162. Neiworth, J. J., Burman, M. A., Basile, B. M., &Melis, A. P., Hare, B., & Tomasello, M. (2006b). Lickteig, M. T. (2002). Use of experimenter- Engineering cooperation in chimpanzees: given cues in visual co-orienting and in an Tolerance constraints on cooperation. Animal object-choice task by a New World monkey Behaviour, 72, 275–286. species, cotton-top tamarins (SanguinusMelis, A. P., Hare, B., & Tomasello, M. (2006c). oedipus). Journal of Comparative Psychology, Chimpanzees recruit the best collaborator. 116, 3–11. Science, 311, 1297–1300. Okamoto-Barth, S., Call, J., & Tomasello, M.Melis, A. P., Hare, B., & Tomasello, M. (2008). Do (2007). Great apes’ understanding of other chimpanzees reciprocate received favors? individuals’ line of sight. Psychological Science, Animal Behaviour, 76, 951–962. 18, 462–468.Melis, A. P., Hare, B., & Tomasello, M. (2009). Okamoto, S., Tomonaga, M., Ishii, K., Kawai, N., Chimpanzees coordinate in a negotiation Tanaka, M., & Matsuzawa, T. (2002). An infant game. Evolution and Human Behavior. doi: chimpanzee (Pan troglodytes) follows human 10.1016/j.evolhumbehav.2009.05.003 gaze. Animal Cognition, 5, 107–114.Meltzoff, A. N. (1995). Understanding the Okamoto-Barth, S., Tomonaga, M., Tanaka, M., intentions of others: Re-enactment of intended & Matsuzawa, T. (2008). Development of acts by 18-month-old children. Developmental using experimenter-given cues in infant Psychology, 31, 838. chimpanzees: Longitudinal changes inMiklosi, A., Pongracz, P., Lakatos, G., Topal, J., behavior and cognitive development. Csanyi, V. (2005). A comparative study of Developmental Science, 11, 98–108. the use of visual communicative signals in Onishi, K. H., & Baillargeon, R. (2005). Do dog–human and cat–human interactions. 15-month-old infants understand false beliefs? Journal of Comparative Psychology. 2, Science, 308, 255. 179–186. ˜ Paz-y-Mino, G., Bond, A. B., Kamil, A. C., & Balda,Miklosi, A., Kubinyi, E., Topal, J., Gacsi, M., R. P. (2004). Pinyon jays use transitive inference Viranyi, Z., & Csanyi, V. (2003). A simple to predict social dominance. Nature, 430, reason for a big difference: Wolves do not look 778–781. back at humans, but dogs do. Current Biology, Petit, O., Desportes, C., & Thierry, B. (1992). 13, 763–766. Differential probability of coproduction in two

PRIMATE SOCIAL COGNITION: THIRTY YEARS AFTER PREMACK AND WOODRUFF 141 species of macaque (Macaca tonkeana, knowing by chimpanzees. Journal of M. mulatta). Ethology, 90, 107–120. Comparative Psychology, 104, 203–210.Phillips, A. T. (2002). Infant’s ability to connect Povinelli, D. J., Parks, K. A., & Novak, M. A. gaze and emotional expression to intentional (1991). Do rhesus monkeys (Macaca mulatta) action. Cognition, 85, 53–78. attribute knowledge and ignorance to others?Phillips, W., Barnes, J. L., Mahajan, N., Journal of Comparative Psychology, 105, Yamaguchi, M., & Santos, L. R. (2009). 318–325. ‘Unwilling versus unable’: Capuchins’ (Cebus Povinelli, D. J., Perilloux, H. K., Reaux, J. E., & apella) understanding of human intentional Bieschwale, D. T. (1998). Young and juvenile action. Developmental Science. doi:10.1111/ chimpanzees’ (Pan troglodytes) reactions to j.1467–7687. 2009. 00840. intentional versus accidental and inadvertentPika, S., Liebal, K., & Tomasello, M. (2003). Gestural actions. Behavioral Processes, 42, 205–218. communication in young gorilla (Gorilla Povinelli, D. J., Reaux, J. E., Bierschwale, D. T., gorilla): Gestural repetoire, learning, and use. Allain, A. D., & Simon, B. B. (1997). American Journal of Primatology, 60, 95–111. Exploitation of pointing as a referential gesturePika, S., Liebal, K., & Tomasello, M. (2005). in young children, but not adolescent Gestural communication in subadult bonobos chimpanzees. Cognitive Development, 12, (Pan paniscus): Repertoire and use. American 327–365. Journal of Primatology, 65, 39–61. Povinelli, D. J., Theall, L. A., Reaux, J. E., &Poss, S. R., Kuhar, C., Stoinski, T. S., & Hopkins, W. Dunphy-Leli, S. (2003). Chimpanzees D. (2006). Differential use of attentional and spontaneously alter the location of their visual communicative signaling by orangutans gestures to match the attentional (Pongo pygmaeus) and Gorillas (Gorilla gorilla) orientation of others. Animal Behaviour, 66, in response to the attentional status of a human. 71–79. American Journal of Primatology, 68, 978–992. Povinelli, D. J., & Vonk, J. (2003). ChimpanzeePovinelli, D. J., Bierschwale, D. T., & Cech, C. minds: suspiciously human? Trends in (1999). Comprehension of seeing as a Cognitive Sciences, 7, 157. referential act in young children, but not Povinelli, D. J., & Vonk, J. (2004). We don’t need a juvenile chimpanzees. British Journal of microscope to explore the chimpanzee’s mind. Developmental Psychology, 17, 37–60. Mind & Language, 19, 1–28.Povinelli, D. J., Dunphy-Llii, S., Reaux, J. E., & Premack, D. (1988). "Does the chimpanzee have a Mazza, M. P. (2002). Psychological diversity in theory of mind?" revisited. In: R. B. A. Whiten chimpanzees and humans: New longitudinal (Ed.), Machiavellian intelligence (pp. 160–179). assessments of chimpanzees’ understanding of New York: Oxford University Press. attention. Brain, Behavior, & Evolution, 59, Premack, D. W. (1978). Does the chimpanzee have 33–53. a theory of mind? Behavioral and Brain Sciences,Povinelli, D. J., & Eddy, T. J. (1996a). 4, 515–526. Chimpanzees: Joint visual attention. Reaux, J., Theall, L. A., & Povinelli, D. J. (1999). A Psychological Science, 7, 129–135. longitudinal investigation of chimpanzee’sPovinelli, D. J., & Eddy, T. J. (1996b). Factors understanding of visual perception. Child influencing young chimpanzees’ (Pan Development, 70, 275–290. troglodytes) recognition of attention. Journal Repacholi BM, Gopnik A (1997) Early reasoning of Comparative Psychlogy, 110, 336–345. about desires: Eidence from 14- and 18-Povinelli, D. J., & Eddy, T. J. (1996c). What young month-olds. Developmental Psychology chimpanzees know about seeing. Monographs 33:12–21. for the Society for Research in Child Riedel, J., Schumann, K., Kaminski, J., Call, J., & Development, 61, 1–152. Tomasello, M. (2007). The early ontogeny ofPovinelli, D. J., & Giambrone. S. (2001). Reasoning human-dog communication. Animal abut beliefs: A human specialization? Child Behaviour, 75, 1003–1014. Development, 72, 691–695. Rose, L. (1997). Vertebrate predation and food-Povinelli, D. J., Nelson, K. E., & Boysen, S. T. sharing in Cebus and Pan. International (1990). Inferences about guessing and Journal of Primatology, 18, 727.

142 PRIMATE NEUROETHOLOGYRuiz, A., Gomez, J. C., Roeder, J. J., * Byrne, R. W. (Eds.), ‘‘Language’’ and intelligence in monkeys (2009) Gaze following and gaze priming in and apes. Cambridge: Cambridge University Press. lemurs. Animal Cognition, 12, 427–434. Tomasello, M. (1999). The cultural origins ofSantos, L. R., Flombaum, J. I., & Phillips, W. human cognition. Cambridge, MA: Harvard (2007a). The evolution of human mindreading: University Press. How non-human primates can inform social Tomasello, M., & Call, J. (1997). Primate cognition. cognitive neuroscience. In: S. Platek (Ed.), New York: Oxford University Press. Evolutionary cognitive nueroscience. Cambridge, Tomasello, M., Call, C., & Hare, B. (1998). Five MA: MIT Press. primate species follow the visual gaze ofSantos, L. R., & Hauser, M. D. (1999). How conspecifics. Animal Behaviour, 55, 1063–1069. monkeys see the eyes: cotton-top tamarins’ Tomasello, M., Call, J., & Hare, B. (2003). reactions to changes in visual attention and Chimpanzees understand psychological action. Animal Cognition, 2, 131–139. states–the question is which ones and toSantos, L. R., Marticorena, D., & Goddu, A. what extent. Trends in Cognitive Sciences, 7, (2007b). Do monkeys reason about the false 153–156. beliefs of others? In: Symposium presented Tomasello, M., Call, J., Nagell, K., Olguin, R., & at the 14th Biennial Meeting of the Carpenter, M. ( 994). The learning and use of Society for Research in Child Development, gestural signals by young chimpanzees: Boston, MA. A trans-generational study. Primates, 35,Santos, L. R., Nissen, A. G., & Ferrugia, J. A. (2006). 137–154. Rhesus monkeys, Macaca mulatta, know what Tomasello, M., & Carpenter, M. (2005). The others can and cannot hear. Animal Behavior, emergence of social cognition in three 71, 1175–1181. young chimpanzees. Monographs of theScerif, G., Gomez, J. C., & Byrne, R. W. (2004). Society for Research in Child Development. What do Diana monkeys know about the focus 70, 1–154. of attention of a conspecific? Animal Behavior, Tomasello, M., Carpenter, M., Call, J., Behne, T., & 68, 1239–1247. Moll, H. (2005). Understanding and sharingShepherd, S. V., & Platt, M. L. (2008). Spontaneous intentions: The origins of cultural cognition. social orienting and gaze following in ringtailed Behavioral and Brain Sciences, 28, 675–735. lemurs (Lemur catta). Animal Cognition, 11, Tomasello, M., Davis-Dasilva, M., Camak, L., & 13–20. Bard, K. (1987). Obervational learning of tool-Southgate, V., Senju, A., & Csibra, G. (2007). ise by young chimpanzees. Human Evolution, 2, Action anticipation through attribution of 175–183. false belief by 2-year-olds. Psychological Tomasello, M., Hare, B., & Agnetta, B. (1999). Science, 18, 587. Chimpanzees, Pan troglodytes, follow gazeSteiper, M. E., & Young, N. M. (2006). Primate direction geometrically. Animal Behaviour, 58, molecular divergence dates. Molecular 769–777. Phylogenetics and Evolution, 41, 384–394. Tomasello, M., Hare, B., & Fogleman, T. (2001).Subiaul, F., Vonk, J., Okamoto-Barth, S. & Barth, J. The ontogeny of gaze following in chimpanzee, (2008). Do chimpanzees learn reputation by Pan troglodytes, and rhesus macaques, Macaca observation? Evidence from direct and indirect mulatta. Animal Behaviour, 61, 335–343. experience with generous and selfish strangers. Tomasello, M., Hare, B., Lehmann, H., & Call, J. Animal Cognition, 11, 611–623. (2007). Reliance on head versus eyes in the gazeSurian, L., Caldi, S., & Sperber, D. (2007). following of great apes and human infants: The Attribution of beliefs by 13-month-old infants. cooperative eye hypothesis. Journal of Human Psychological Science, 18, 580–586. Evolution, 52, 314–320.Theall, L. A., & Povinelli, D. J. (1999). Do Tomonage, M., Tanaka, M., Matsuzawa, T., chimpanzees tailor their gestural signals to fit Myowa-Yamakoshi, M., & Kosugi, D. (2004). the attentional state of others? Animal Development of social cognition in infant Cognition, 2, 207–214. chimpanzees (Pan troglodytes): FaceTomasello, M. (1990). Cultural transmission in the recognition, smiling, gaze, and the lack of tool use and communicatory signaling of triadic interactions. Japanese Psychological chimpanzees? In: S. T. Parker & K. R. Gibson Research, 46, 227–235.

PRIMATE SOCIAL COGNITION: THIRTY YEARS AFTER PREMACK AND WOODRUFF 143Uller, C. (2004). Disposition to recognize goals in Warneken, F., & Tomasello, M. (2006). Altruistic infant chimpanzees. Animal Cognition, 7, helping in human infants and young 154–161. chimpanzees. Science, 311, 1301–1303.Uller, C., & Nichols, S. (2000). Goal attribution in Watts, D. P. (1998). Coalitionary mate guarding by chimpanzees. Cognition, 76, B27–B34. male chimpanzees in Ngogo, Kibale NationalVick, S. J., & Anderson, J. R. (2000). Learning Park, Uganda. Behavioral Ecology and and limits of use of eye gaze by capuchin Sociobiology, 44, 43–55. monkeys (Cebus apella) in an object-choice Watts, D. P., & Mitani, J. C. (2001). Boundary task. Journal of Comparative Psychology, 114, patrols an intergroup encounters in wild 200–207. chimpanzees. Behaviour, 138, 299–327.Vick, S. J., & Anderson, J. R. (2003). Use of human Watts, H. E., & Holekamp, K. E. (2007). Hyena visual attention cues by olive baboons (Papio societies. Current Biology, 17, R557–R660. anubis) in a competitive task. Journal of Wellman, H. M. (1990). The child’s theory of mind. Comparative Psychology, 117, 209–216. Cambridge, MA: MIT Press.Vick, S. J., Bovet, D., & Anderson, J. R. (2001). Gaze Wimmer, H., & Perner, J. (1983). Beliefs about discrimination learning in olive baboons (Papio beliefs: Representation and constraining anubis). Animal Cognition, 4, 1–10. function of wrong beliefs in young children’sViranyi, Z., Gacsi, M., Kubinyi, E., Topal, J., understanding of deception. Cognition, 13, Belenyi, B., Ujfalussy, D., et al. (2008). 103– 128. Comprehension of human pointing gestures in Wood, J. N., Glynn, D. D., Phillips, B. C., & Hauser, oung human-reared wolves (Canis lupus) and M. D. (2007). The perception of rational, goal- dogs (Canis familiaris). Animal Cognition, 11, directed action in non-human primates. 373–387. Science, 317, 1402.Visalberghi, E. (1990). The manufacture and use Woodward, A. (1998). Infants selectively encode of tools by capuchin monkeys (Cebus the goal object of an actors reach. Cognition, apella). Journal of Comparative Psychology, 69, 1–34. 101, 159. Woodward, A., Sommerville, J. A., & Guajardo, J. J.Warneken, F., Hare, B., Melis, A. P., Haunus, D., & (2001). How infants make sense of intentional Tomasello, M. (2007). Spontaneous altruism by action. In: B. F. Malle, L. J. Moses, & D. A. Balin chimpanzees and young children. PLoS Biology, (Eds.), Intentions and intentionality. 5, 1414–1420. Cambridge: MIT Press.

CHAPTER 8Behavioral Signatures of Numerical CognitionElizabeth M. Brannon, Kerry E. Jordan and Sarah M. JonesINTRODUCTION IS IT NUMBER OR A SIMPLE TRICK?As adult humans, we rely on numbers foralmost everything we do. We use numbers to The study of numerical abilities in animals gotmeasure time, weight, height, money, speed, off to a fractious start with false claims that aand scores. We use numbers to organize the horse named ‘‘Clever Hans’’ could add, subtract,world around us by labeling things such as the multiply, and even divide. Nevertheless, over theclasses we teach or the IP addresses of our last century there have been a multitude of care-computers. We also use numbers as ordinals fully controlled studies that demonstrate thatto rank our place in a line or a race. Our math- many species of nonhuman animals can ignoreematical capacities are honed in school and perceptual variables such as surface area, density,become highly symbolic and abstract over perimeter, and intensity and attend to the purelydevelopment. No animal will ever achieve the numerical attribute of visual, auditory, and, inhuman capacity to precisely count the number rare demonstrations, even tactile stimulusof cars in a parking lot or use algebra to calcu- arrays.late the speed of a jet that travels 5,400 km in For example, with laboratory training rhesus9 hours against a jetstream of 190 km/hour. But monkeys are able to match arrays based onwhat are the evolutionary foundations that sup- numerosity and ignore continuous dimensionsport the uniquely human ability for mathema- of an array such as element size, cumulative sur-tical thinking? There has been over a century of face area, and density (Jordan & Brannon, 2006;research on the numerical abilities of non- Nieder & Miller, 2004; Nieder et al., 2002).human animals. The ability to represent Performance has been shown to be equivalentnumber approximately has been documented when continuous variables are congruent orin animals as diverse as lizards, rats, pigeons, incongruent with numerosity (Jordan &ferrets, raccoons, dolphins, lemurs, monkeys, Brannon, 2006). Studies of numerical orderingand apes. Here we focus on findings that estab- in monkeys have used similar stringent stimuluslish the nature of the approximate number controls and have also demonstrated that mon-system (ANS) in nonhuman primates, both keys can make purely numerical judgments (e.g.,because of the focus of this book and because Brannon & Terrace, 1998, 2000; Brannon et al.,of the extensive amount of work on this topic 2006; Cantlon & Brannon, 2005, 2006a, 2006b;with monkeys and apes and more recently pro- Judge et al., 2005; Smith et al., 2003; see Fig. 8.1simians. Our review reveals that nonhuman for example stimuli and a screenshot of a monkeyprimates and humans possess a shared ANS solving the task). For example, Beran (2008)that functions to represent number as an showed that rhesus monkeys could enumerateabstract variable permitting computations moving arrays of dots and attend to the numer-about the quantitative aspects of the world. osity of a subset of the dots within an array.144

BEHAVIORAL SIGNATURES OF NUMERICAL COGNITION 145Figure 8.1 Example stimuli used in a numerical ordering task. On some trials number covaried withcumulative surface area, density, or perimeter, while on other trials these variables were equated for thelarger and smaller numerosity. After Cantlon, J. F., & Brannon, E. M. (2006a). Shared system for orderingsmall and large numbers in monkeys and humans. Psychological Science, 17, 402–407. Used with permission.Arrays contained red and black dots that moved Given these carefully controlled studies, thereindependently and monkeys were rewarded for can be little doubt that nonhuman animals havechoosing the array with the larger number of the capacity to discriminate stimuli based solelyblack dots in each display (Fig. 8.2). Capuchin on number and ignore nonnumerical dimensionsmonkeys and rhesus monkeys were tested in this that typically covary with number in nature.paradigm; however, only the rhesus monkeyswere above chance on trials in which the correct The Behavioral Signature of Animalanswer involved choosing an array with a larger Numerical Competencenumber of black dots but a smaller number of Symbolic representations of number (e.g., wordstotal dots. and Arabic numerals) allow us to make precise Furthermore, a great deal of work also suggests discriminations, such as distinguishing betweenthat rats and pigeons, among many other non- 33 and 34 children on a school bus. In contrast, aprimate species, have numerical competence ubiquitous feature of the ANS is that discrimi-(e.g., Emmerton et al., 1997; Meck & Church, nation depends on the ratio between numerical1983; Olthof & Roberts, 2000; Roberts, 2005). values. With count words, it is apparent thatFigure 8.2 Stimuli used to test the ability of capuchin and rhesus monkeys to enumerate a subset of anarray. Monkeys were rewarded for moving the cursor (the red dot below the display) to choose the arraywith the larger number of black dots. Beran, M. J. (2008). Monkeys (Macaca mulatta and Cebus apella)track, enumerate, and compare multiple sets of moving items. Journal of Experimental Psychology: AnimalBehavior Processes, 34, 63–74. Used with permission.

146 PRIMATE NEUROETHOLOGY1,053 and 1,054 differ by the same single unit as 3 demonstrated that when rhesus monkeys wereand 4, whereas with the ANS such comparisons presented with pairs of arrays of 2 to 30 ele-are markedly different given the discrepancy in ments and required to choose the numericallythe ratios for each pair. One explanation for smaller array, their accuracy and reaction timeratio dependence is that the underlying repre- to respond was modulated by the ratio betweensentation for numerosity is an analog format the compared values (Fig. 8.3). Furthermore,(e.g., Dehaene, 1992; Feigenson et al., 2004; when rats, pigeons, or monkeys are tested onGallistel & Gelman, 1992). In this format, numerical bisection tasks in which they are firstnumerosity is represented as a mental magni- trained to categorize small and large anchortude that is proportional to the quantity it repre- values and then required to classify inter-sents; consequently, discrimination obeys mediate values as closer to the small or largeWeber’s Law. Weber’s Law states that DI/I = C, anchor, their performance is best explained bywhere DI is the increase in stimulus intensity to a models that assume they use a ratio comparisonstimulus of intensity I that is required to pro- rule (Church & Deluty, 1977; Emmerton &duce a detectable change in intensity and C is a Renner, 2006; Fetterman, 1993; Jordan &constant. Therefore, if a graduate student detects Brannon, 2006; Meck & Church, 1983; Mecka change when her advisor adds a 2-pound book et al., 1985; Platt & Davis, 1983; Roberts, 2005;to her 10-pound backpack full of books, she Stubbs, 1976).would need a 4-pound book to detect a change Ratio dependence has been found in all thein a 20-pound backpack. systematic tests of numerical competence Ratio dependence can be seen when animals throughout the animal kingdom. This is espe-compare arrays of elements based on numer- cially noteworthy given that a shortcoming ofosity (e.g., Beran, 2008; Brannon & Terrace, research in this domain is that direct species1998, 2000; Brannon et al., 2006; Cantlon & comparisons have rarely been made. TestingBrannon, 2005, 2006a, 2006b; Judge et al., multiple species with the same stimuli and2005; Smith et al., 2003; Tomonaga, 2008), tasks would allow a test of whether differenceswhen they are required to match sets based on in socioecology have selected for differences innumerosity (e.g., Jordan & Brannon, 2006), and numerical sensitivity between genera or species.when they make comparisons of food quanti- Thus, a next step for comparative research onties (e.g., Addessi et al., 2008; Beran, 2001; numerical competence will be to standardizeBeran & Beran, 2004; Beran et al., 2008; Call, tasks and stimuli and test multiple species in an2000; Evans et al., 2008; Stevens et al., 2007). effort to understand the selective pressures thatFor example, Cantlon and Brannon (2006a) account for variability in numerical abilities. A 100 B 900 Monkeys Humans 800 Percentage Correct 80 RT (ms) 700 60 chance 600 40 500 20 Monkeys Humans 400 0 .0 .2 .4 .6 .8 1.0 .0 .2 .4 .6 .8 1.0Figure 8.3 Accuracy (a) and reaction time (b) for monkeys and humans as a function of the ratio betweenvalues in a numerical ordering task. For both monkeys and humans, accuracy decreased and reaction timeincreased as the ratio approached 1. After Cantlon, J. F., & Brannon, E. M. (2006a). Shared system forordering small and large numbers in monkeys and humans. Psychological Science, 17, 402–407. Used withpermission.

BEHAVIORAL SIGNATURES OF NUMERICAL COGNITION 147THE ANS SHARED BY ANIMALS either required to press at rates so fast they wereAND HUMANS ALIKE prevented from verbally counting or were engaged in a verbal distracter task. Like rats, theAlthough adult humans from most parts of the mean number of key presses made by humanworld are highly adept at verbal counting and subjects was proportional to the target number,using symbolic number systems to perform com- and representations showed scalar variability inplex mathematical operations, the ANS coexists that the standard deviation of the response dis-with language-mediated number representations tributions increased with target magnitudein adult humans. When adults are tested in non- (Cordes et al., 2001; Whalen et al., 1999).verbal tasks that are designed to emulate animal In another example, Cantlon and Brannontasks, their performance often looks virtually (2006a) directly compared adult humans’ andindistinguishable from their furry counterparts rhesus monkeys’ abilities to order numerical(e.g., Barth et al., 2006; Beran, 2008; Boisvert arrays. As shown in Figure 8.3, the two specieset al., 2003; Cantlon & Brannon, 2006a, 2007b; showed strikingly similar patterns in accuracyCordes et al., 2001; Moyer & Landauer, 1967; and reaction time. In a similar numericalWhalen et al., 1999). For instance, Whalen and ordering task, Beran (2006) uncovered anothercolleagues (1999) and Cordes and colleagues intriguing parallel between monkey and human(2001) adapted a classic paradigm from the numerical cognition. Monkeys, like humans,animal literature for use with humans. In the were sensitive to the regular-random numer-original experiment, rats were required to press osity illusion (RRNI). That is, both monkeysa lever a target number of times to gain reward and humans systematically overestimated the(Platt & Johnson, 1971). The mean number of number of items in sets of stimuli that wereresponses the rats made increased with the regularly (rather than randomly) arranged (seerequired number; importantly, the variability Fig. 8.4 for example stimuli).(standard deviation) in their response distribu- Another striking similarity between thetions was proportional to the required number of numerical representations of adult humans andresponses. In the human analog, subjects were rhesus monkeys is the effect of semanticFigure 8.4 Dot displays used with monkeys and humans. Both species overestimated the number of itemsin sets of stimuli that were regularly (rather than randomly) arranged. After Beran, M. J. (2006). Quantityperception by adult humans (Homo sapiens), chimpanzees (Pan troglodytes), and rhesus macaques (Macacamulatta) as a function of stimulus organization. International Journal of Comparative Psychology, 19,386–397. Used with permission.

148 PRIMATE NEUROETHOLOGYcongruity on both species’ numerical judgments rabbit) is smaller than they are at judging which(Cantlon & Brannon, 2005). The semantic con- of these two small animals is larger. In contrast,gruity effect has been found whenever adults we are faster at judging which of two large ani-compare the relative magnitude of values along mals (i.e., a zebra and an elephant) is larger thanan ordinal continuum including numerical com- we are at judging which of two large animals isparisons (Banks et al., 1976; Moyer & Bayer, smaller (Banks et al., 1983; Shaki & Algom,1976). For example, people are faster at judging 2002). Intuitively, this effect seems driven bywhich of two small animals (i.e., a rat and a language (Holyoak, 1978). However, Cantlon 1000 1000 Choose smaller 900 900 Choose larger 800 800 RT (ms) RT (ms) 700 700 600 600 Choose smaller 500 500 Choose larger 400 400 1–2 2–3 3–4 4–5 5–6 6–7 7–8 8–9 1–2 2–3 3–4 4–5 5–6 6–7 7–8 8–9 Numerical pair Numerical pairFigure 8.5 (A) Screen shots of monkeys tested in a conditional discrimination, whereby a blue screenindicated that they should choose the larger numerosity first and a red screen indicated that they shouldchoose the smaller numerosity first. (B) Reaction time as a function of numerical pair for monkeys makingan ordinal number judgment illustrates the semantic congruity effect. Monkeys were faster at choosing thesmaller of two small numbers than they were at choosing the larger of two small numbers (red line).Conversely, when required to choose the larger of two large numbers, they were faster than when they wererequired to choose the smaller of two large numbers (blue line). After Cantlon, J. F., & Brannon, E. M.(2005). Semantic congruity affects numerical judgments similarly in monkeys and humans. Proceedings ofthe National Academy of Sciences, 102, 16507–16511. Used with permission.

BEHAVIORAL SIGNATURES OF NUMERICAL COGNITION 149and Brannon (2005) found a semantic congruity (Boysen & Berntson, 1989). The chimpanzeeeffect for rhesus monkey numerical compari- was led around a room to various hidingsons. Monkeys were faster at choosing the places where she saw Arabic numerals or setssmaller of two small numerosities (i.e., 2 vs. 3) of oranges. She was then required to choose thecompared to the larger of two small numerosities Arabic numeral that corresponded to the sum(Fig. 8.5A, B). Conversely, when required to of the values she has seen. Amazingly, the chim-choose the larger of two large numerosities, panzee performed well on the 14 test trials.they were faster than when they were required Taking a different approach, Hauser and col-to choose the smaller of two large numerosities. leagues adapted a paradigm, first developed byThus, monkeys share with adult humans a sus- Wynn (1992) to study human infant numericalceptibility to the semantic congruity effect in cognition, to test whether untrained monkeysnumerical comparisons, suggesting that the spontaneously look longer at physical outcomeseffect may better reflect internal representations that violate their arithmetic expectationsof ordinal continua rather than linguistic (Hauser et al., 1996). Monkeys observed as egg-encoding. plants (or lemons) were placed on a stage. In summary, when humans are tested in non- A screen was then raised to occlude the egg-verbal tasks that circumvent verbal counting, plants, and additional eggplants were thentheir performance looks similar to nonhuman placed behind the screen. The screen was raisedanimals. This suggests that even educated adults to reveal either the expected outcome or anwith knowledge of higher-order symbolic arithmetically impossible outcome. In multiplemathematics simultaneously possess an ANS studies using this paradigm, untrained monkeysthat is shared with other animals. Additional have been found to look longer at mathemati-evidence for this conclusion comes from recent cally impossible compared to possible outcomesstudies with two Amazonian groups, both of (e.g., Flombaum et al., 2005; Hauser et al., 1996).which lack verbal counting systems. Despite an Beran and colleagues have also found that non-inability to label a large collection with a precise human primates reliably choose the larger of twonumerical value, both societies possess nonfood quantities, even when this requires trackingverbal numerical representations with sensitivity one-by-one additions to multiple caches overthat is roughly equivalent to Europeans with time (Beran, 2004; Beran & Beran, 2004).verbal counting systems (Frank et al., 2008; In the most parametric study of addition inGordon, 2004; Pica et al., 2004). Such data indi- nonhuman animals to date, Cantlon andcate that the ANS is likely universal among Brannon (2007b) directly compared monkeyshumans and not culturally specific. and humans in an explicit addition task. Subjects viewed two addend arrays presented in succession on a touch-sensitive screen and wereARITHMETIC OPERATIONS required to choose the array (from two arrays)As suggested by Gelman and Gallistel (Gallistel, that was equal to the sum of the two addend1990; Gallistel et al., 2005), nonverbal numer- arrays (Fig. 8.6A). The monkeys were trained onical representations are likely crucial a few addition problems (1 + 1 = 2, 4, or 8;throughout the animal kingdom insofar as 2 + 2 = 2, 4, or 8; 4 + 4 = 2, 4, or 8) and thenthey support calculations for behaviorally tested on novel problems. Monkeys performedimportant decisions. Bolstering this idea, a at a level significantly greater than chance onhandful of studies have demonstrated that non- each of these three problems within 500 trials.human animals can perform mathematical In the critical test, monkeys were presented withoperations on their numerical representations. all possible addends of the novel sums 3, 7, 11,In one compelling study, a chimpanzee named and 17. Monkeys were rewarded regardless ofSheba was first trained to map symbols to which of the two choice stimuli they selected asnumerosities and subsequently tested on her the sum to prevent learning during the experi-ability to add symbols or sets of objects ment. Performance on the novel problems was

150 PRIMATE NEUROETHOLOGY B Accuracy Response time 2.5 100% 2.0 Response time (s) Percent correct 80% 1.5 1.0 60% 0.5 Monkeys Humans 40% 0.0 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 Ratio RatioFigure 8.6 (A) Stimuli and task design used to test nonverbal addition in monkeys and college students.Monkeys and humans were presented with one set of dots (set 1), followed by a brief delay, after which asecond set of dots was presented (set 2). Then, two choices (the sum and the distractor) were presented, andmonkeys were rewarded for touching the choice that represented the numerical sum of the two sets.(B) Monkeys and humans exhibited ratio-dependent accuracy and response time when solving additionproblems. Accuracy (left panel) and response times (right panel). Error bars reflect the standard erroramong subjects. After Cantlon, J. F., & Brannon, E. M. (2007b). Basic math in monkeys and college students.PLoS Biology, 5, e328. Used with permission.significantly greater than that predicted by IS REPRESENTING NUMBER Achance, and performance was modulated by LAST-RESORT STRATEGY?numerical ratio. Importantly, as shown inFigure 8.6B, humans and monkeys tested on In this section we query the functional utility ofthe same nonverbal addition task showed very the ANS. One view has been that animals onlysimilar performance (see also Barth et al., 2006). represent number as a last resort after extensiveThus, it appears that the ANS supports arith- laboratory training (Breukelaar & Dalrymple-metic, and an important function of the ANS Alford, 1998; Davis, 1993; Davis & Memmot,system may be to make approximate calcula- 1982; Davis & Perusse, 1988; Seron & Pesenti,tions about the world around us. 2001). Under this view, animals may possess a

BEHAVIORAL SIGNATURES OF NUMERICAL COGNITION 151latent capacity to represent number, but the number (Stevens et al., 2007). Of course thiscapacity will only emerge with extensive training makes adaptive sense; all organisms should beand only when they are prevented from using an motivated to maximize food intake, not numberalternative strategy. While there are some experi- of food items.mental results consistent with the idea that ani- While most laboratory studies of numericalmals may under some circumstances avoid using competence in animals employ extensivenumber, by and large there is more support for training and focus on the question of what ani-an alternative view that many nonhuman animals are capable of rather than what they domals spontaneously use numerical representa- spontaneously, a few laboratory studies havetions in the wild. directly addressed the question of spontaneous For example, American coots have been numerical cognition. Hauser and colleaguesfound to keep track of the number of their own (2002) tested whether untrained cotton-topeggs in clutches that contain both their own eggs tamarins could recognize the numerical equiva-and those of conspecific parasites (Lyon, 2003). lence between auditory stimuli presented in dif-Females appear to terminate the production of ferent formats. Using a habituation-egg follicles based on the number of their own discrimination procedure, they presentedeggs, which are only a subset of the total eggs in cotton-top tamarins with a mixture of eitherthe nest. Territorial defense is another domain in two- or three-syllable speech sequences varyingwhich number might be particularly useful. An in overall duration, intersyllable duration, andanimal might benefit from being able to com- pitch. After the tamarins habituated to thispare the numerosity of her own group to a repeated presentation of speech syllables ofcompeting group. Wilson and colleagues either numerosity, they presented the tamarins(2001) designed a study to test the possibility with novel two- or three-tone sequences in a testthat wild chimpanzees might base decisions phase. They found that tamarins oriented to theabout whether to defend a territory against an speaker longer when the number of tones in theinvading animal on the number of individuals test phase differed from the number of speechavailable to help in the defense. They played the syllables in the habituation phase. Becausepant-hoot of a single male chimpanzee (who was Hauser and colleagues controlled for variousnot a member of the group being tested) to nonnumerical parameters such as duration andvariable numbers of adult males in the test tempo that are often confounded with numbergroup. They found that groups of three or in the real world, their study showed thatmore males consistently chorused loudly in without any reinforcement or training, tamarinresponse to the playback and approached the monkeys could discriminate numerical values inspeaker. In contrast, groups containing fewer the auditory modality—even when this requiredthan three males usually failed to make such comparing sounds that were quite different (alsoresponses. Chimpanzees may thus calculate see Hauser et al., 2003).whether they have ‘‘strength in numbers’’ Prosimian primates have also been tested inbefore deciding whether to defend their spontaneous numerical cognition tasks. Santosterritory. et al. (2005) modified a task used with human Another important reason a nonhuman infants and rhesus macaques (Wynn, 1992, andanimal might need to represent number is to Hauser et al., 1996, respectively) and testedmaximize food intake. Not surprisingly, mul- lemurs with a looking time experiment con-tiple studies have found that animals sponta- trasting a numerically possible event with aneously choose the larger of two food arrays numerically impossible event. The lemurs saw(e.g., Beran, 2001; Beran et al., 2005; Boysen & two lemons hidden one at a time behind aBerntson, 1995; Call, 2000). However, when screen. The screen was lifted to reveal one, two,number and quantity are in conflict in such or three lemons. Lemurs looked longer at thefood choice situations, a recent study suggests unexpected compared to the expected arith-that maximizing amount of food wins out over metic outcomes. The results showed that

152 PRIMATE NEUROETHOLOGYlemurs, like rhesus macaques and human CROSS-MODALinfants, are able to spontaneously quantify a REPRESENTATIONS OF NUMBERsmall set of objects and form precise expecta- IN ANIMALStions about a simple 1 + 1 addition problem. In another task adapted from research with Four cherries and four submarines have little inhuman infants (Feigenson & Carey, 2003), Lewis common except for their numerosity. Yetet al. (2005) tested the numerical capacities of clearly, adult humans have no trouble recog-mongoose lemurs. Lemurs observed as a number nizing such numerical equivalence. Numberof grapes (up to eight) were dropped into a can be extracted from sets of simultaneouslycontainer; they were then allowed to retrieve all visible individuals or from sequentially pre-or a subset of the grapes. Results showed that sented sets such as a sequence of tones.lemurs continued to search when the bucket Furthermore, numerical representations are notshould have still contained grapes and that they specific to the sensory modalities in which theyrequired a 1:2 ratio to discriminate the number are established. Three telephone rings, three tapsof grapes they obtained from the number that on your shoulder, and three political candidatesshould have been remaining in the bucket. are equally good examples of threeness. The Cantlon and Brannon (2007a) designed a ability to represent number without regard tostudy to directly assess the relative salience of stimulus modality is integral to any notion of anumber, color, shape, and cumulative surface truly abstract concept of number. Here, wearea for monkeys. They trained monkeys in a explore whether the ANS is sufficiently abstractmatch-to-sample task in which the sample and to represent number independent of stimuluscorrect match were the same in both numerosity modality. If so, is this a fundamental part ofand in an additional dimension (color, shape, or animals’ numerical perception and cognition,cumulative surface area of the elements). The or does it depend on extensive training?incorrect choice differed from the sample and Barth and colleagues (2003) have shown thatmatch both in numerosity and on the alternative in human adults the ANS operates indepen-dimension. After the monkeys reached a criterial dently of stimulus modality. Specifically, theylevel of performance, they were tested on non- found that adults showed virtually no perfor-differentially reinforced probe trials in which mance cost of comparing numerosities acrossone choice matched the sample in numerosity versus within visual and auditory stimulus sets.and the other choice matched the sample on an This leads to the prediction that nonhuman ani-alternative dimension (color, shape, or cumula- mals should also be able to detect the numericaltive surface area of the elements). They found correspondence between sets of entities pre-that, for all conditions (color, shape, or cumu- sented in different sensory modalities.lative surface area of the elements), the prob- A few studies have addressed this question.ability that monkeys made a numerosity match Church and Meck (1984) found that rats trainedincreased with the numerical distance between to discriminate two from four sounds or lightthe numerosity match and the numerosity of the flashes later responded to compound cues of twoalternative stimulus. Furthermore, in the case of lights and two sounds as if four events hadthe cumulative surface area condition, monkeys occurred, suggesting that rats can transferwere more likely to match stimuli based on numerical representations across modalities.numerosity at all distances. The bias for Yet, when Davis and Albert (1987) trained ratsnumber over area was found both for the mon- to discriminate three sequentially presentedkeys that had prior numerosity training and a sounds from two or four sounds and thensingle monkey who had never been trained on a exposed rats to sequences of two, three, andnumerical discrimination task. Thus, even for a four lights, they found no evidence that the ratsmonkey, number can be more salient than other transferred their auditory numerical discrimina-set summary statistics, such as cumulative sur- tion to the visual modality. The Davis and Albertface area. (1987) results raise the possibility that the rats in

BEHAVIORAL SIGNATURES OF NUMERICAL COGNITION 153the Church and Meck study (1984) made a peculiar to social judgments. It was possibledichotomous discrimination that was purely that these experiments succeeded because theyintensity based (i.e., they equated the less intense tapped into a socioecologically relevant scenario;sound with the less intense light). in their everyday lives, for example, territorial Field playback studies have yielded comple- animals might be helped by being able to assessmentary evidence suggesting that animals pre- the number of individuals they will likelydict the number of intruders they expect to see encounter by how many individuals they hear.based on the number of vocalizing intruders In addition, the Jordan et al. (2005) study wasthey hear. In these studies, the probability that restricted to the small values 2 and 3.an animal from a focal group approached a To ask whether monkeys could numericallyspeaker emitting vocalizations from foreign con- match arbitrary stimuli across sensory modal-specifics depended on the relationship between ities and to see whether the ability extendedthe number of vocalizing foreign animals and beyond the small values 2 and 3, Jordan andthe number of animals present in the focal colleagues (2008) used a task that required angroup (Kitchen, 2004; McComb et al., 1994). active choice to numerically match auditoriallyFor example, McComb and colleagues (1994) presented tones and visually presented squares.found that lions were more likely to approach a They trained two monkeys to choose a visualspeaker emitting the roar of a single unfamiliar array of two, three, six, or eight squares thatlion than a chorus of three unfamiliar lions, numerically matched a sample sequence ofsuggesting that lions decide whether to defend shapes or sounds. Monkeys numericallytheir territory based on the perceived number of matched across (audio-visual) and withinintruders. However, such studies did not control (visual-visual) modalities with equal accuracyfor all possible nonnumerical auditory cues that and transferred to novel numerical values (seecovary with number (e.g., some aspects of dura- Fig. 8.8A for an example trial). Monkeys weretion), leaving open the question of whether the also able to sum over shapes and tones andcalculations made by the animals were in fact choose the visual array that numericallybased on number. matched the total number of visual or auditory A recent experiment capitalized on the social events in a sequence. As shown in Figure 8.8B,expertise of nonhuman primates by framing a accuracy and reaction time depended on thenumerical problem within a social context: ratio between the correct numerical match andSpecifically, researchers tested whether rhesus incorrect choice. Thus, monkeys and humansmonkeys spontaneously matched the number appear to share an abstract numerical code thatof dynamic conspecific faces they saw with the can be divorced from the modality in whichnumber of vocalizations they simultaneously stimuli are first experienced.heard by employing the sort of preferentiallooking paradigm that had been used extensively HOW ARE ANALOG MAGNITUDEwith human infants (Jordan et al., 2005; see Fig. REPRESENTATIONS OF8.7A). The study employed strict stimulus con- NUMEROSITY FORMED?trols for temporal attributes that often covarywith number and used a between-subject A host of models have been proposed to explaindesign. Monkeys looked significantly longer at how the ANS forms analog magnitude represen-the matching display than the nonmatching dis- tations. Gelman and Gallistel (1978; Gallistel &play, and the effect held for individuals who Gelman, 1992) suggested that animals and youngheard two sounds or three sounds (Fig. 8.7A, children use a nonverbal process that obeys threeB). Thus, monkeys recognize the numerical cor- critical counting principles. The one-to-one prin-respondence between two or three dynamic faces ciple states that there is a one-to-one correspon-and two or three concurrent voices. However, dence between the number of labels applied andthese experiments left open the possibility that the number of to-be-counted elements in a setthis ability was some context-specific capacity (labels can be nonverbal). The stable order

154 PRIMATE NEUROETHOLOGYA B 20 15 Time (s) 10 5 0 Match NonmatchFigure 8.7 (A) Static images from video displays used to test whether rhesus monkeys spontaneouslymatched the number of dynamic conspecific faces they saw with the number of vocalizations theysimultaneously heard in a preferential looking paradigm. (B) Monkeys looked longer at the display thatnumerically matched the number of voices they heard. After Judge, P. G., Evans, T. A., & Vyas, D. K. (2005).Ordinal representation of numeric quantities by brown capuchin monkeys (Cebus apella). Journal ofExperimental Psychology: Animal Behavior Processes, 31, 79–94. Used with permission.principle states that the labels are applied in a used as a counter (the event mode) or a timerstable order across counting episodes (one (the run and stop modes). In all modes, the valuecannot count 1-2-3 today and 3-1-2 tomorrow). stored in the accumulator increases with dura-The cardinal principle states that the last label tion or number. When the stimulus stops, theapplied serves to represent the cardinal value of mode switch opens, stopping the gating of pulsesthe set. to the accumulator. The final magnitude is The mode-control model, developed to placed into memory, while the accumulator isdescribe how rats enumerate serial arrays reset to zero and the organism is considered(sequences of sounds or light flashes), follows ready to time or count another stimulus. Thethese counting principles (Meck & Church, representation of the magnitude in memory is1983). Under this model, a pacemaker emits a noisy and obeys Weber’s Law. Importantly, suchregular stream of pulses at a constant frequency. a mode-control mechanism can represent andWhen a stimulus begins, pulses are gated into an integrate sequentially processed numerical sti-accumulator, which integrates the number of muli presented in any sensory modality, poten-pulses over time. Depending on the nature of tially accounting for highly abstract numericalthe stimulus, a mode switch that closes after representations.stimulus onset gates the pulses to the accumu- The mode-control model works well forlator in one of three different modes: in the explaining how serially presented items are‘‘run’’ mode the stimulus starts an accumulation enumerated. But what happens when animalsprocess that continues until the end of the signal; encounter visual arrays? It is possible to convertin the ‘‘stop’’ mode the process occurs whenever a simultaneous array into a sequential array bythe stimulus occurs; and in the ‘‘event’’ mode serially allocating attention to each element ineach onset of the stimulus produces a relatively the array. However, the behavioral and neuro-fixed duration of the process regardless of sti- biological evidence to date suggests that it ismulus duration. This mechanism can thus be unlikely rhesus macaques take this approach.

BEHAVIORAL SIGNATURES OF NUMERICAL COGNITION 155 A Sample Element 1 Sample Element 2 350 ms 510 ms Sample Element 3 200 ms 333 ms Sample 1000 ms Element 4 921 ms Choices 422 ms B 100 90 80 70 Percent correct 60 Visual Sample 50 Auditory Sample 40 30 20 10 0 33 56 67 75 86 11 14 25 44 89 2 4 6 8 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. Ratio between correct and incorrect number matchFigure 8.8 (A) Example sample stimuli from an auditory-visual trial. Monkeys heard four successive tonesand were then rewarded for choosing an array of four visual elements. (B) Average accuracy as a function ofthe ratio between the numerical value of the two choice stimuli. Black squares indicate visual samples andgray triangles indicate auditory samples. Accuracy in both modalities was modulated by ratio. Dotted lineindicates chance performance (50%). After Jordan, K. E., MacLean, E., & Brannon, E. M. (2008). Monkeysmatch and tally quantities across senses. Cognition, 108, 617–625. Used with permission.Nieder and colleagues found that neurons in the of approximately 120 ms (Nieder et al., 2002).prefrontal cortex (PFC) of the macaque brain Furthermore, if animals were serially allocatingthat were selective for one, two, three, four, or attention to elements in an array, we mightfive elements showed similar response latencies expect to find that the number of visual saccades

156 PRIMATE NEUROETHOLOGYincreases with the number of elements in an volume, Nieder describes the mounting evidencearray. However, Nieder and Miller (2004) that homologous brain structures supportfound that the number of visual saccades made numerical representations in humans andby monkeys did not increase with the number of macaque monkeys (see Chapter 20). A strongdisplay items in a numerical same-different task possibility is that the ANS observed in nonhumanas set size increased from one to seven. Similarly, primates and humans represents an evolutiona-analysis of reaction time for monkeys engaged in rily ancient system embedded in the same neuralthe numerical delayed match-to-sample task hardware. Future work should systematicallydescribed earlier in this chapter provided no study variation in numerical cognition betweenevidence that monkeys serially enumerate species throughout the animal kingdom tovisual arrays (Jordan & Brannon, 2006). Since explore whether there are quantitative or qualita-the monkeys controlled the duration of the tive differences that can be tied to socioecology orsample array and hence the processing time for to phylogeny.the sample numerosity, a serial process shouldbe reflected by increased reaction times to larger REFERENCESnumerical samples. Reaction time, however, didnot increase with sample numerosity. Barth and Addessi, E., Crescimbene, L., & Elisabetta V. (2008).colleagues (2003) also found that it took adults Food and token quantity discrimination inno longer to compare large sets than to compare capuchin monkeys (Cebus apella). Animalsmall sets with comparable ratios, suggesting Cognition, 11, 275–282. Banks, W. P., Fujii, M., & Kayra-Stuart, F. (1976).that adults may also use a parallel enumeration Semantic congruity effects in comparativemechanism for nonverbally estimating and com- judgment effects in comparative judgments ofparing large sets. magnitudes of digits. Journal of Experimental Dehaene and Changeux’s (1993) neural net- Psychology: Human Perception and Performance,work model achieves numerical representations 2, 435–447.through such a parallel process. This model is Banks, W. P., White, H., Sturgill, W., &composed of four primary levels: the topographi- Mermelstein, R. (1983). Semantic congruitycally organized input retina, an organized map of and expectancy in symbolic judgments. Journalelement positions from this retina, summation of Experimental Psychology: Human Perceptionunits that sum the input from the organized posi- and Performance, 9, 580–582.tion map, and an array of numerosity detectors. Barth, H., Kanwisher, N., & Spelke, E. (2003). The construction of large number representations inThese numerosity detectors respond selectively to adults. Cognition, 86, 201–221.certain numerosities (i.e., certain ranges of activity Barth, H., La Mont, K., Lipton, J., Dehaene, S.,from the summation clusters). The numerosity Kanwisher, N., & Spelke, E. (2006). Non-detection system ultimately represents the numer- symbolic arithmetic in adults and youngosity of a set of simultaneously presented elements children. Cognition, 98, 199–222.regardless of the size and position of the elements Beran, M. J. (2001). Summation and numerousness(see also Verguts & Fias, 2004). judgments of sequentially presented sets of items by chimpanzees (Pan troglodytes). Journal of Comparative Psychology, 115, 181–191.CONCLUSION Beran, M. J. (2004). Chimpanzees (Pan troglodytes)Our review revealed strong behavioral parallels respond to nonvisible sets after one-by-one addition and removal of items. Journal ofbetween human and animal numerical cognition Comparative Psychology, 118, 25–36.such as ratio dependence, semantic congruity, Beran, M. J. (2006). Quantity perception by adultcross-modal matching, and nonverbal arithmetic. humans (Homo sapiens), chimpanzees (PanDespite early skepticism about animal numerical troglodytes), and rhesus macaques (Macacaabilities, it is now irrefutable that a wide variety of mulatta) as a function of stimulus organization.animal species are sensitive to the numerical attri- International Journal of Comparative Psychology,butes of the world around them. Elsewhere in this 19, 386–397.

BEHAVIORAL SIGNATURES OF NUMERICAL COGNITION 157Beran, M. J. (2008). Monkeys (Macaca mulatta and pygmaeus). Journal of Comparative Psychology, Cebus apella) track, enumerate, and compare 114, 136–147. multiple sets of moving items. Journal of Cantlon, J. F., & Brannon, E. M. (2005). Semantic Experimental Psychology: Animal Behavior congruity affects numerical judgments similarly Processes, 34, 63–74. in monkeys and humans. Proceedings of theBeran, M. J., & Beran, M. M. (2004). Chimpanzees National Academy of Sciences, 102, 16507– remember the results of one-by-one addition of 16511. food items to sets over extended time periods. Cantlon, J. F., & Brannon, E. M. (2006a). Shared Psychological Science, 15, 94–99. system for ordering small and large numbers inBeran, M. J., Beran, M. M., Harris, E. H., & monkeys and humans. Psychological Science, 17, Washburn, D. A. (2005). Ordinal judgments 402–407. and summation of nonvisible sets of food Cantlon, J. F., & Brannon, E. M. (2006b). The effect items by two chimpanzees and a rhesus of heterogeneity on numerical ordering in macaque. Journal of Experimental Psychology: rhesus monkeys. Infancy, 9, 173–189. Animal Behavior Processes, 31, 351–362. Cantlon, J. F., & Brannon, E. M. (2007a). How muchBeran, M. J., Evans, T. A., Leighty, K. A., Harris, does number matter to a monkey (Macaca E. H., & Rice, D. (2008). Summation and mulatta)? Journal of Experimental Psychology: quantity judgments of sequentially presented Animal Behavior Processes, 33, 32–41. sets by capuchin monkeys (Cebus apella). Cantlon, J. F., & Brannon, E. M. (2007b). Basic American Journal of Primatology, 70, 191–194. math in monkeys and college students. PLoSBoisvert, M. J., Abroms, B. D., & Roberts, W. A. Biology, 5, e328. (2003). Human nonverbal counting estimated Church, R. M., & Deluty, M. Z. (1977). Bisection of by response production and verbal report. temporal intervals. Journal of Experimental Psychonomic Bulletin and Review, 10, 683–690. Psychology: Animal Behavior Processes, 3, 216–228.Boysen, S. T., & Berntson, G. G. (1989). Church, R., & Meck, W. (1984). The numerical Numerical competence in a chimpanzee (Pan attribute of stimuli. In H. L. Roitblat, T. G. troglodytes). Journal of Comparative Psychology, Bever, & H.S. Terrace (Eds.), Animal cognition 103, 23–31. (pp. 445–464). Hillsdale, NJ: Erlbaum.Boysen, S. T., & Berntson, G. G. (1995). Responses Cordes, S., Gelman, R., & Gallistel, C. R. (2001). to quantity: Perceptual versus cognitive Variability signatures distinguish verbal from mechanisms in chimpanzees (Pan troglodytes). nonverbal counting for both large and small Journal of Experimental Psychology: Animal numbers. Psychological Bulletin and Review, 8, Behavior Processes, 21, 82–86. 698–707.Brannon, E. M., Cantlon, J. F., & Terrace, H. S. (2006). Davis, H. (1993). Numerical competence in animals: The role of reference points in ordinal numerical Life beyond Clever Hans. In S. T. Boysen & comparisons by Rhesus macaques (Macaca E. J. Capaldi (Eds.), The development of mulatta). Journal of Experimental Psychology: numerical competence: Animal and human Animal Behavior Processes, 32, 120–134. models (pp. 109–125). Hillsdale, NJ: Erlbaum.Brannon, E. M., & Terrace, H. S. (1998). Ordering Davis, H., & Albert, M. (1987). Failure to transfer of the numerosities 1 to 9 by monkeys. Science, or train a numerical discrimination using 282, 746–749. sequential visual stimuli in rats. Bulletin of theBrannon, E. M., & Terrace, H. S. (2000). Psychonomic Society, 25, 472–474. Representation of the numerosities 1–9 by Davis, H., & Memmott, J. (1982). Counting rhesus macaques (Macaca mulatta). Journal of behavior in animals: A critical evaluation. Experimental Psychology: Animal Behavior Psychological Bulletin, 92, 547–571. Processes, 26, 31–49. Davis, H., & Perusse, R. (1988). NumericalBreukelaar, J. W. C., & Dalrymple-Alford, J. C. competence in animals: Definitional issues, (1998). Timing ability and numerical current evidence, and a new research agenda. competence in rats. Journal of Experimental Behavioral and Brain Sciences, 11, 561–651. Psychology: Animal Behavior Processes, 24, Dehaene, S. (1992). Varieties of numerical abilities. 84–97. Cognition, 44, 1–42.Call, J. (2000). Estimating and operating on Dehaene, S., & Changeux, J. P. (1993). discrete quantities in orangutans (Pongo Development of elementary numerical

158 PRIMATE NEUROETHOLOGY abilities: A neuronal model. Journal of Cognitive Proceedings of the National Academy of Neuroscience, 5, 390–407. Sciences, 93, 1514–1517.Emmerton, J., Lohmann, A., & Niemann, J. (1997). Hauser, M. D., Tsao, F. T., Garcia, P., & Spelke, Pigeons’ serial ordering of numerosity with E. S. (2003). Evolutionary foundations of visual arrays. Animal Learning and Behavior, number: Spontaneous representation of 25, 234–244. numerical magnitudes by cotton-topEmmerton, J., & Renner, J. C. (2006). Scalar effects tamarins. Proceedings of the Royal Society, in the visual discrimination of numerosity by London, B, 270, 1441–1446. pigeons. Learning and Behavior, 34, 176–192. Holyoak, K. (1978). Comparative judgments withEvans, T. A., Beran, M. J., Harris, E. H., & Rice, D. F. numerical reference points. Cognitive (2008). Quantity judgments of sequentially Psychology, 10, 203–243. presented food items by capuchin monkeys Jordan, K. E., & Brannon, E. M. (2006). Weber’s (Cebus apella). Animal Cognition, 12, 97–105. Law influences the numerical representations inFeigenson, L., & Carey, S. (2003). Tracking individual rhesus macaques (Macaca mulatta). Animal via object-files: evidence from infants’ manual Cognition, 9, 159–172. search. Developmental Science, 6, 568–584. Jordan, K. E., Brannon, E. M., Logothetis, N. K., &Feigenson, L., Dehaene, S., & Spelke, E. (2004). Ghazanfar, A. A. (2005). Monkeys match the Core systems of number. Trends in Cognitive number of voices they hear to the number of Sciences, 8, 307–314. faces they see. Current Biology 15, 1–5.Fetterman, J. G. (1993). Numerosity discrimination: Jordan, K. E., MacLean, E., & Brannon, E. M. Both time and number matter. Journal of (2008). Monkeys match and tally quantities Experimental Psychology: Animal Behavior across senses. Cognition, 108, 617–625. Processes, 19, 149–164. Judge, P. G., Evans, T. A., & Vyas, D. K. (2005).Flombaum, J. I., Junge, J. A., & Hauser, M. D. Ordinal representation of numeric quantities by (2005). Rhesus monkeys (Macaca mulatta) brown capuchin monkeys (Cebus apella). spontaneously compute addition operations Journal of Experimental Psychology: Animal over large numbers. Cognition, 97, 315–325. Behavior Processes, 31, 79–94.Frank, M., Everett, D., Fedorenko, E., & Gibson, E. Kitchen, D. M. (2004). Alpha male black howler (2008). Number as a cognitive technology: monkey responses to loud calls: Effect of Evidence from Piraha language and cognition. numeric odds, male companion behaviour and Cognition, 108, 819–824. reproductive investment. Animal Behaviour, 67,Gallistel, C. R. (1990). The organization of learning. 125–139. Cambridge, MA: Bradford/MIT Press. Lewis, K. P., Jaffe, S., & Brannon, E. M.Gallistel, C. R., & Gelman, R. (1992). Preverbal and (2005). Analog number representations in verbal counting and computation. Cognition, mongoose lemurs (Eulemur mongoz): 44, 43–74. Evidence from a search task. AnimalGallistel, C. R., Gelman, R., & Cordes, S. (2005). Cognition, 8, 247–252. The cultural and evolutionary history of the real Lyon, B. E. (2003). Egg recognition and counting numbers. In S. Levinson & P. Jaisson (Eds.), reduce costs of avian conspecific brood Culture and evolution (pp. 247–276). Oxford: parasitism. Nature, 422, 495–499. Oxford University Press. McComb, K., Packer, C., & Pusey, A. (1994).Gelman, R., & Gallistel, C. R. (1978). The child’s Roaring and numerical assessment in contests understanding of number. Cambridge, MA: between groups of female lions, Panthera leo. Harvard University Press. Animal Behaviour, 47, 379–387.Gordon, P. (2004). Numerical cognition without Meck, W., & Church, R. (1983). A mode control words: Evidence from Amazonia. Science, 306, model of counting and timing processes. 496–499. Journal of Experimental Psychology: AnimalHauser, M., Dehaene, S., Dehaene-Lambertz, G., & Behavior Processes, 9, 320–334. Patalano, A. (2002). Spontaneous number Meck, W. H., Church, R. M., & Gibbon, J. discrimination of multi-format auditory stimuli (1985). Temporal integration in duration in cotton-top tamarins. Cognition, 86, B23–B32. and number discrimination. Journal ofHauser, M. D., MacNeilage, P., & Ware, M. (1996). Experimental Psychology: Animal Behavior Numerical representations in primates. Processes, 11, 591–597.

BEHAVIORAL SIGNATURES OF NUMERICAL COGNITION 159Moyer, R. S., & Bayer, R. H. (1976). Mental Seron, X., & Pesenti, M. (2001). The number sense comparison and the symbolic distance effect. theory needs more empirical evidence. Mind & Cognitive Psychology, 8, 228–246. Language, 16, 76–89.Moyer, R., & Landauer, T. (1967). Time required Shaki, S., & Algom, D. (2002). The locus and nature for judgments of numerical inequality. Nature, of semantic congruity in symbolic comparison: 215, 1519–1520. Evidence from the Stroop effect. Memory andNieder, A., Freedman, D. J., & Miller, E. K. (2002). Cognition, 30, 3–17. Representation of the quantity of visual items in Smith, B. R., Piel, A. K., & Candland, D. K. (2003). the primate prefrontal cortex. Science, 297, Numerity of a socially housed hamadryas 1708–1711. baboon (Papio hamadryas) and a sociallyNieder, A., & Miller, E. K. (2004). Analog housed squirrel monkey (Saimiri sciureus). numerical representations in rhesus monkeys: Journal of Comparative Psychology, 117, 217–225. Evidence for parallel processing. Journal of Stevens, J., Wood, J., & Hauser, M. (2007). Cognitive Neuroscience, 16, 889–901. Quantity trumps number: DiscriminationOtlhof, A., & Roberts, W. A. (2000). Summation of experiments in cotton-top tamarins (Saguinus symbols by pigeons (Columba livia): The oedipus) and common marmosets (Callithrix importance of number and mass of reward items. jacchus). Animal Cognition, 10, 429–437. Journal of Comparative Psychology, 114, 158–166. Stubbs, D. A. (1976). Scaling of stimulus durationPica, P., Lemer, C., Izard, V., & Dehaene, S. (2004). by pigeons. Journal of the Experimental Analysis Exact and approximate arithmetic in an of Behavior, 26, 15–25. Amazonian indigene group. Science, 306, Tomonaga, M. (2008). Relative numerosity discri- 499–503. mination by chimpanzees (Pan troglodytes):Platt, J. R., & Davis, E. R. (1983). Bisection of evidence for approximate numerical repre- temporal intervals by pigeons. Journal of sentations. Animal Cognition, 11, 43–57. Experimental Psychology: Animal Behavioral Verguts, T., & Fias, W. (2004). Representation of Processes, 9, 160–170. number in animals and humans: A neural model.Platt, J. R., & Johnson, D. M. (1971). Localization Journal of Cognitive Neuroscience, 16, 1493–1504. of position within a homogeneous behavior Whalen, J., Gallistel, C. R., & Gelman, R. (1999). chain: Effects of error contingencies. Learning Nonverbal counting in humans: The and Motivation, 2, 386–414. psychophysics of number representation.Roberts, W. A. (2005). How do pigeons represent Psychological Science, 10, 130–137. numbers? Studies of number scale bisection. Wilson, M., Hauser, M., & Wrangham, R. (2001). Behavioral Processes, 69, 33–43. Does participation in intergroup conflict dependSantos, L. R., Sulkowski, G. M., Spaepen, G. M., on numerical assessment, range location, or rank & Hauser, M. D. (2002). Object individuation for wild chimpanzees? Animal Behaviour, 61, using property/kind information in rhesus 1203–1216. macaques (Macaca mulatta). Cognition, 83, Wynn, K. (1992). Addition and subtraction by 241–264. human infants. Nature, 358, 749–750.

CHAPTER 9The Foundations of TransdisciplinaryBehavioral ScienceHerbert Gintis1INTRODUCTION species whose characteristics are the product of its particular evolutionary history. For humans,The behavioral sciences include economics, evolutionary dynamics are captured by gene-cul-biology, anthropology, sociology, psychology, ture coevolution. The centrality of culture andand political science, as well as their subdisci- complex social organization to the evolutionaryplines, including neuroscience, archaeology and success of Homo sapiens implies that individualpaleontology, and, to a lesser extent, such related fitness in humans will depend on the structure ofdisciplines as history, legal studies, and philo- cultural life. Since obviously culture is influ-sophy. These disciplines have many distinct con- enced by human genetic propensities, it followscerns, but each includes a model of individual that human cognitive, affective, and moral capa-human behavior. These models are not only dif- cities are the product of a unique dynamic offerent, which is to be expected given their distinct gene-culture interaction. This coevolutionaryexplanatory goals, but also incompatible. This process has endowed us with preferences thatsituation is well known, but does not appear dis- go beyond the self-regarding concerns empha-comforting to behavioral scientists, as there has sized in traditional economic and biologicalbeen virtually no effort to repair this condition. theory and embrace such non–self-regarding The behavioral sciences all include models of values as a taste for cooperation, fairness, andindividual human behavior. Therefore, these retribution; the capacity to empathize; and themodels should be compatible, and indeed, there ability to value such constitutive behaviors asshould be a common underlying model, enriched honesty, hard work, toleration of diversity, andin different ways to meet the particular needs of loyalty to one’s reference group.2each discipline. Realizing this goal at presentcannot be easily attained, since the various beha-vioral disciplines currently have incompatible Evolutionary Game Theorymodels. Yet, recent theoretical and empirical devel- The analysis of living systems includes one con-opments have created the conditions for rendering cept that does not occur in the nonliving world,coherent the areas of overlap of the various beha- and is not analytically represented in the naturalvioral disciplines, as outlined in this paper. The sciences. This is the notion of a strategic interac-analytical tools deployed in this task incorporate tion, in which the behavior of agents is derivedcore principles from several behavioral disciplines. by assuming that each is choosing a fitness-relevant response to the actions of other agents. TheEvolutionary Perspective study of systems in which agents choose fitness-Evolutionary biology underlies all behavioral relevant responses and in which such responsesdisciplines because Homo sapiens is an evolved evolve dynamically is called evolutionary game 160

THE FOUNDATIONS OF TRANSDISCIPLINARY BEHAVIORAL SCIENCE 161theory. Game theory provides a transdisciplinary The reason for this contrast between the nat-conceptual basis for analyzing choice in the pre- ural and the behavioral sciences is that livingsence of strategic interaction. However, the clas- systems are generally complex, dynamic adaptivesical game theoretic assumption that agents are systems with emergent properties that cannot beself-regarding must be abandoned except in spe- fully captured in analytical models that attendcific situations (e.g., anonymous market interac- only to the local interactions of the system. Thetions), and many characteristics that classical hypothetico-deductive methods of game theory,game theorists have considered deductions the BPC model, and even gene-culture coevolu-from the principles of rational behavior, tionary theory must therefore be complementedincluding the use of backward induction, are in by the work of behavioral scientists who adherefact not implied by rationality. Evolutionary to more empiricist and particularist traditions.game theory, whose equilibrium concept is that For instance, cognitive anthropology interfacesof a stable stationary point of a dynamic system, with gene-culture coevolution and the BPCmust thus replace classical game theory, which model by enhancing their capacity to modelerroneously favors subgame perfection and culture at a level of sophistication that fills insequentiality as equilibrium concepts. the black box of the physical instantiation of culture in coevolutionary theory.The Beliefs, Preferences, and Constraints A complex system consists of a large popula-(BPC) Model tion of similar entities (in our case, human individuals) who interact through regularizedGeneral evolutionary principles suggest that channels (e.g., networks, markets, social institu-individual decision making can be modeled as tions) with significant stochastic elements,optimizing a preference function subject to without a system of centralized organizationinformational and material constraints. Natural and control (i.e., if there a state, it controlsselection leads the content of preferences to only a fraction of all social interactions, andreflect biological fitness. The principle of itself is a complex system). A complex system isexpected utility extends this optimization to sto- adaptive if it evolves through some evolutionarychastic outcomes. The resulting model is called (genetic, cultural, agent-based silicon, or other)the rational actor model in economics, but I will process of hereditary reproduction, mutation,generally refer to this as the beliefs, preferences, and selection (Holland, 1975). To characterizeand constraints (BPC) model to avoid the often a system as complex adaptive does not explain itsmisleading connotations attached to the term operation, and does not solve any problems.‘‘rational.’’ However, it suggests that certain modeling tools are likely to be effective that have little useSociety as Complex Adaptive System in a noncomplex system. In particular, the tradi-The behavioral sciences advance not only by the tional mathematical methods of physics anddeveloping analytical and quantitative models chemistry must be supplemented by other mod-but also by the accumulating historical, descrip- eling tools, such as agent-based simulation andtive, and ethnographic evidence that pays close network theory.attention to the detailed complexities of life in Such novel research tools are needed becausethe sweeping array of wondrous forms that a complex adaptive system generally has emer-nature reveals to us. This situation is in sharp gent properties that cannot be analytically derivedcontrast with the natural sciences, which have from its component parts. The stunning successfound little use for narrative alongside analytical of modern physics and chemistry lies in theirmodeling. By contrast, historical contingency is ability to avoid or strictly limit emergence.a primary focus of analysis and causal explana- Indeed, the experimental method in naturaltion for many researchers working on sociolo- science is to create highly simplified laboratorygical, anthropological, ecological, and even conditions, under which modeling becomesbiological topics. analytically tractable. Physics is no more

162 PRIMATE NEUROETHOLOGYeffective than economics or biology in analyzing robotics, plate tectonics) from the basic laws ofcomplex real-world phenomena in situ. The var- physics, similarly, one cannot deduce the struc-ious branches of engineering (electrical, che- ture and dynamics of complex life forms frommical, mechanical) are effective because they basic biological principles. But, just as physicalrecreate in everyday life artificially controlled, principles inform model creation in the naturalnoncomplex, nonadaptive, environments in sciences, so must biological principles inform allwhich the discoveries of physics and chemistry the behavioral sciences.can be directly applied. This option is generallynot open to most behavioral scientists, whorarely have the opportunity of ‘‘engineering’’ THE FOUNDATIONS OF THE BPCsocial institutions and cultures. MODEL In addition to these conceptual tools, thebehavioral sciences of course share common For every constellation of sensory inputs, eachaccess to the natural sciences, statistical and decision taken by an organism generates a prob-mathematical techniques, computer modeling, ability distribution over fitness outcomes, theand a common scientific method. expected value of which is the fitness associated with that decision. Since fitness is a scalar variable, for each constellation of sensory inputs,EVOLUTIONARY PERSPECTIVE each possible action the organism might takeA replicator is a physical system capable of has a specific fitness value, and organismsdrawing energy and chemical building blocks whose decision mechanisms are optimized forfrom its environment to make copies of itself. this environment will choose the available actionChemical crystals, such as salt, have this prop- that maximizes this value.3 It follows that, givenerty, but biological replicators have the addi- the state of its sensory inputs, if an organismtional ability to assume myriad physical forms with an optimized brain chooses action A overbased on the highly variable sequencing of its action B when both are available, and chooses ¨chemical building blocks (Schrodinger called life action B over action C when both are available,an ‘‘aperiodic crystal’’ in 1943, before the struc- then it will also choose action A over action Cture of DNA was discovered). Biology studies when both are available. This is called choicethe dynamics of such complex replicators using consistency.the evolutionary concepts of replication, varia- The so-called rational actor model was devel-tion, mutation, and selection (Lewontin, 1974). oped in the twentieth century by John von Biology plays a role in the behavioral sciences Neumann, Leonard Savage, and many others.much like that of physics in the natural sciences. The model appears prima facie to apply onlyJust as physics studies the elementary processes when actors possess extremely strong informa-that underlie all natural systems, biology studies tion-processing capacities. However, the modelthe general characteristics of survivors of the in fact depends only on choice consistency andprocess of natural selection. In particular, the assumption that agents can trade off amonggenetic replicators, the epigenetic environments outcomes in the sense that for any finite set ofto which they give rise, and the effect of these outcomes A1,. . .,An, if A1 is the least preferredenvironments on gene frequencies account for and An the most preferred outcome, then for anythe characteristics of species, including the Ai, 1 £ i £ n there is a probability pi, 0 £ pi £ 1development of individual traits and the nature such that the agent is indifferent between Ai andof intraspecific interaction. This does not mean, a lottery that pays A1 with probability pi and paysof course, that behavioral science in any sense An with probability 1 – pi (Kreps, 1990). Clearly,reduces to biological laws. Just as one cannot these assumptions are often extremely plausible.deduce the character of natural systems (e.g., When applicable, the rational actor model’sthe principles of inorganic and organic chem- choice consistency assumption strongly enhancesistry, the structure and history of the universe, explanatory power, even in areas that have

THE FOUNDATIONS OF TRANSDISCIPLINARY BEHAVIORAL SCIENCE 163traditionally abjured the model (Coleman, 1990; decision making process in the brain synthesizes allHechter & Kanazawa, 1997; Kollock, 1996). available information into a single, unitary value In short, when preferences are consistent, (Glimcher, 2003; Parker & Newsome, 1998; Schallthey can be represented by a numerical function, & Thompson, 1999). Indeed, when animals areoften called a utility function, which the indivi- tested in a repeated trial setting with variabledual maximizes subject to his or her beliefs reward, dopamine neurons appear to encode the(including Bayesian probabilities) and con- difference between the reward that an animalstraints. Four caveats are in order. First, this expected to receive and the reward that ananalysis does not suggest that people consciously animal actually received on a particular trialmaximize something called ‘‘utility,’’ or any- (Schultz et al., 1997; Sutton & Barto, 2000), anthing else. Second, the model does not assume evaluation mechanism that enhances the environ-that individual choices, even if they are self- mental sensitivity of the animal’s decision-makingreferring (e.g., personal consumption), are system. This error-prediction mechanism has thealways welfare enhancing. Third, preferences drawback of only seeking local optima (Sugrue etmust be stable across time to be theoretically al., 2005). Montague and Berns (2002) address thisuseful, but preferences are ineluctably a function problem, showing that the orbitofrontal cortexof such parameters as hunger, fear, and recent and striatum contains a mechanism for moresocial experience, while beliefs can change dra- global predictions that include risk assessmentmatically in response to immediate sensory and discounting of future rewards. Their data sug-experience. Finally, the BPC model does not gest a decision-making model that is analogous topresume that beliefs are correct or that they are the famous Black-Scholes options pricing equationupdated correctly in the face of new evidence, (Black & Scholes, 1973).although Bayesian assumptions concerning The BPC model is the most powerful analy-updating can be made part of consistency in tical tool of the behavioral sciences. For most ofelegant and compelling ways (Jaynes, 2003). its existence this model has been justified in The rational actor model is the cornerstone terms of ‘‘revealed preferences’’ rather than byof contemporary economic theory, and in the the identification of neural processes that gen-past few decades has become the cornerstone of erate constrained optimal outcomes. The neu-the biological modeling of animal behavior roscience evidence, for the first time, suggests a(Alcock, 1993; Real, 1991; Real & Caraco, firmer foundation for the rational actor model.1986). Economic and biological theory thushave a natural affinity: The choice consistency GENE-CULTURE COEVOLUTIONon which the rational actor model of economictheory depends is rendered plausible by biolo- The genome encodes information that is used togical evolutionary theory, and the optimization construct a new organism, to instruct the newtechniques pioneered by economic theorists are organism how to transform sensory inputs intoroutinely applied and extended by biologists in decision outputs (i.e., to endow the newmodeling the behavior of a vast array of organism with a specific preference structure),organisms. and to transmit this coded information virtually In addition to the explanatory success of the- intact to the new organism. Since learning aboutories based on the rational actor model, supporting one’s environment is costly and error prone,evidence from contemporary neuroscience sug- efficient information transmission will ensuregests that expected utility maximization is not that the genome encodes all aspects of the organ-simply an ‘‘as if ’’ story. In fact, the brain’s neural ism’s environment that are constant, or thatcircuitry actually makes choices by internally change only very slowly through time andrepresenting the payoffs of various alternatives as space. By contrast, environmental conditionsneural firing rates and choosing a maximal such that vary across generations and/or in therate (Glimcher, 2003; Glimcher et al., 2005). course of the organism’s life history can beNeuroscientists increasingly find that an aggregate dealt with by providing the organism with the

164 PRIMATE NEUROETHOLOGYcapacity to learn, and hence phenotypically strong interactions between genetic and epige-adapt to specific environmental conditions. netic elements in human evolution, ranging There is an intermediate case that is not effi- from basic physiology (e.g., the transformationciently handled by either genetic encoding or of the organs of speech with the evolution oflearning. When environmental conditions are language) to sophisticated social emotions,positively but imperfectly correlated across gen- including empathy, shame, guilt, and revengeerations, each generation acquires valuable seeking (Zajonc, 1980, 1984).information through learning that it cannot Because of their common informational andtransmit genetically to the succeeding genera- evolutionary character, there are strong parallelstion, because such information is not encoded between genetic and cultural modelingin the germ line. In the context of such environ- (Mesoudi et al., 2006). Like biological transmis-ments, there is a fitness benefit to the transmission, culture is transmitted from parents to off-sion of epigenetic information concerning the spring, and like cultural transmission, which iscurrent state of the environment. Such epige- transmitted horizontally to unrelated indivi-netic information is quite common (Jablonka duals, so in microbes and many plant species,& Lamb, 1995) but achieves its highest and genes are regularly transferred across lineagemost flexible form in cultural transmission in boundaries (Abbott et al., 2003; Jablonka &humans and to a considerably lesser extent in Lamb, 1995; Rivera & Lake, 2004). Moreover,other primates (Bonner, 1983; Richerson & anthropologists reconstruct the history of socialBoyd, 1998). Cultural transmission takes the groups by analyzing homologous and analogousform of vertical (parents to children) horizontal cultural traits, much as biologists reconstruct the(peer to peer), and oblique (elder to younger), as evolution of species by the analysis of sharedin Cavalli and Feldman (1981); prestige (higher characters and homologous DNA (Mace &influencing lower status), as in Henrich and Gil- Pagel, 1994). Indeed, the same computer pro-White (2001); popularity related, as in Newman grams developed by biological systematists areand colleagues (2006); and even random popu- used by cultural anthropologists (Holden, 2002;lation-dynamic transmission, as in Shennan Holden & Mace, 2003). In addition, archeolo-(1997) and Skibo and Bentley (2003). gists who study cultural evolution have a similar The parallel between cultural and biological modus operandi as paleobiologists who studyevolution goes back to Huxley (1955), Popper genetic evolution (Mesoudi et al., 2006). Both(1979), and James (1880).4 The idea of treating attempt to reconstruct lineages of artifacts andculture as a form of epigenetic transmission was their carriers. Like paleobiology, archaeologypioneered by Richard Dawkins, who coined the assumes that when analogy can be ruled out,term ‘‘meme’’ in The Selfish Gene (Dawkins, similarity implies causal connection by inheri-1976) to represent an integral unit of informa- tance (O’Brien & Lyman, 2000). Like biogeo-tion that could be transmitted phenotypically. graphy’s study of the spatial distribution ofThere quickly followed several major contribu- organisms (Brown & Lomolino, 1998), beha-tions to a biological approach to culture, all vioral ecology studies the interaction of ecolo-based on the notion that culture, like genes, gical, historical, and geographical factors thatcould evolve through replication (intergenera- determine distribution of cultural forms acrosstional transmission), mutation, and selection space and time (Smith & Winterhalder, 1992).(Boyd & Richerson, 1985; Cavalli-Sforza & Perhaps the most common critique of theFeldman, 1982; Lumsden & Wilson, 1981). analogy between genetic and cultural evolution Cultural elements reproduce themselves is that the gene is a well-defined, discrete, inde-from brain to brain and across time, mutate, pendently reproducing and mutating entity,and are subject to selection according to their whereas the boundaries of the unit of cultureeffects on the fitness of their carriers (Boyd & are ill-defined and overlapping. In fact, however,Richerson, 1985; Cavalli-Sforza & Feldman, this view of the gene is simply outdated.1982; Parsons, 1964). Moreover, there are Overlapping, nested, and movable genes

THE FOUNDATIONS OF TRANSDISCIPLINARY BEHAVIORAL SCIENCE 165discovered in the past 35 years have some of the the absence of embarrassment, pride and regretfluidity of cultural units, whereas quite often the (Beer et al., 2003; Camille, 2004), and sociopathicboundaries of a cultural unit (a belief, icon, behavior (Miller et al., 1997). There is a likelyword, technique, stylistic convention) are quite genetic predisposition underlying sociopathy,delimited and specific. Similarly, alternative spli- and sociopaths comprise 3% to 4% of the malecing, nuclear and messenger RNA editing, cel- population, but they account for between 33%lular protein modification, and genomic and 80% of the population of chronic criminalimprinting, which are quite common, under- offenders in the United States (Mednick et al.,mine the standard view of the insular gene pro- 1977).ducing a single protein and support the notion It is clear from this body of empirical infor-of genes having variable boundaries and having mation that culture is directly encoded into thestrongly context-dependent effects. human brain, which of course is the central Dawkins added a second fundamental claim of gene-culture coevolutionary theory.mechanism of epigenetic information transmis-sion in The Extended Phenotype (Dawkins, GAME THEORY: THE UNIVERSAL1982), noting that organisms can directly LEXICON OF LIFEtransmit environmental artifacts to the next gen-eration, in the form of such constructs as beaver In the BPC model, choices give rise to prob-dams, bee hives, and even social structures (e.g., ability distributions over outcomes, the expectedmating and hunting practices). The phenom- values of which are the payoffs to the choiceenon of a species creating an important aspect from which they arose. Game theory extendsof its environment and stably transmitting this analysis to cases where there are multiplethis environment across generations, known as decision makers. In the language of game theory,niche construction, is a widespread form of epi- players (or agents) are endowed with a set ofgenetic transmission (Odling-Smee et al., 2003). available strategies and have certain informationMoreover, niche construction gives rise to what concerning the rules of the game, the nature ofmight be called a gene-environment coevolu- the other players and their available strategies,tionary process, since a genetically induced envir- and the structure of payoffs. Finally, for eachonmental regularity becomes the basis for combination of strategy choices by the players,genetic selection, and genetic mutations that the game specifies a distribution of individualgive rise to mutant niches will survive if they payoffs to the players. Game theory attempts toare fitness enhancing for their constructors. predict the behavior of the players by assumingThe analysis of the reciprocal action of genes that each maximizes its preference function sub-and culture is known as gene-culture coevolution ject to its information, beliefs, and constraints(Bowles & Gintis, 2005; Durham, 1991; Feldman (Kreps, 1990).& Zhivotovsky, 1992; Lumsden & Wilson, 1981). Game theory is a logical extension of evolu- Neuroscientific studies exhibit clearly the tionary theory. To see this, suppose there is onlygenetic basis for moral behavior. Brain regions one replicator, deriving its nutrients and energyinvolved in moral judgments and behavior from nonliving sources (the sun, the earth’sinclude the prefrontal cortex, the orbitofrontal core, amino acids produced by electrical dis-cortex, and the superior temporal sulcus (Moll charge, and the like). The replicator populationet al., 2005). These brain structures are virtually will then grow at a geometric rate, until it pressesunique to or most highly developed in humans upon its environmental inputs. At that point,and are doubtless evolutionary adaptations mutants that exploit the environment more effi-(Schulkin, 2000). The evolution of the human ciently will out-compete their less efficient con-prefrontal cortex is closely tied to the emergence specifics, and with input scarcity, mutants willof human morality (Allman et al., 2002). Patients emerge that ‘‘steal’’ from conspecifics who havewith focal damage to one or more of these areas amassed valuable resources. With the rapidexhibit a variety of antisocial behaviors, including growth of such predators, mutant prey will

166 PRIMATE NEUROETHOLOGYdevise means of avoiding predation, and preda- (1964) showed that Fisher’s Fundamentaltors will counter with their own novel predatory Theorem, which states that as long as there iscapacities. In this manner, strategic interaction is positive genetic variance in a population, fitnessborne from elemental evolutionary forces. It is increases over time, is false when more than oneonly a conceptually short step from this point to genetic locus is involved. Eshel and Feldmancooperation and competition among cells in a (1984) identified the problem with the popula-multicellular body, among conspecifics who tion genetic model in its abstraction from muta-cooperate in social production, between males tion. But how do we attach a fitness value to aand females in a sexual species, between parents mutant? Eshel and Feldman (1984) suggestedand offspring, and among groups competing for that payoffs be modeled game theoretically onterritorial control. the phenotypic level and a mutant gene be asso- Historically, game theory did not emerge ciated with a strategy in the resulting game. Withfrom biological considerations, but rather from this assumption, they showed that under somethe strategic concerns of combatants in World restrictive conditions, Fisher’s FundamentalWar II (Poundstone, 1992; Vonneumann & Theorem could be restored. Their results wereMorgenstern, 1944). This led to the widespread generalized by Liberman (1988), Hammersteincaricature of game theory as applicable only to and Selten (1994), Hammerstein (1996), Eshelstatic confrontations of rational self-regarding and colleagues (1998), and others.agents possessed of formidable reasoning and Third, the most natural setting for biologicalinformation-processing capacity. Developments and social dynamics is game theoretic.within game theory in recent years, however, Replicators (genetic and/or cultural) endowrender this caricature inaccurate. copies of themselves with a repertoire of stra- First, game theory has become the basic fra- tegic responses to environmental conditions,mework for modeling animal behavior (Alcock, including information concerning the condi-1993; Krebs & Davies, 1997; Maynard Smith, tions under which each is to be deployed in1982), and thus has shed its static and hyperra- response to character and density of competingtionalistic character, in the form of evolutionary replicators. Genetic replicators have been wellgame theory (Gintis, 2000). Evolutionary and understood since the rediscovery of Mendel’sbehavioral game theory do not require the for- laws in the early twentieth century. Culturalmidable information-processing capacities of transmission also apparently occurs at the neu-classical game theory, so disciplines that recog- ronal level in the brain, in part through thenize that cognition is scarce and costly can make action of mirror neurons (Meltzoff & Decety,use of game-theoretic models (Gintis, 2000; 2003; Rizzolatti et al., 2002; Williams et al.,Gigerenzer & Selten, 2001; Young, 1998). Thus, 2001). Mutations include replacement of strate-agents may consider only a restricted subset of gies by modified strategies, and the ‘‘survival ofstrategies (Simon, 1972; Winter, 1971), and they the fittest’’ dynamic (formally called a replicatormay use rule-of-thumb heuristics rather than dynamic) ensures that replicators with more suc-maximization techniques (Gigerenzer & Selten, cessful strategies replace those with less suc-2001). Game theory is thus a generalized schema cessful ones (Taylor & Jonker, 1978).that permits the precise framing of meaningful Fourth, behavioral game theorists nowempirical assertions but imposes no particular widely recognize that in many social interac-structure on the predicted behavior. tions, agents are not self-regarding, but rather Second, evolutionary game theory has often care about the payoffs to and intentions ofbecome key to understanding the most funda- other players, and will sacrifice to uphold per-mental principles of evolutionary biology. sonal standards of honesty and decency (Fehr &Throughout much of the twentieth century, ¨ Gachter, 2002; Gintis et al., 2005; Gneezy, 2005;classical population biology did not employ a Wood, 2003). Moreover, human actors caregame-theoretic framework (Haldane, 1932; about power, self-esteem, and behaving morallyFisher, 1930; Wright, 1931). However, Moran (Bowles & Gintis, 2005; Gintis, 2003; Wood,

THE FOUNDATIONS OF TRANSDISCIPLINARY BEHAVIORAL SCIENCE 1672003). Because the rational actor model treats on behalf of the fitness of an unrelated other—action as instrumental toward achieving rewards, was virtually unknown, even in the case of Homoit is often inferred that action itself cannot have sapiens.reward value. This is an unwarranted inference. The selfish nature of human nature wasFor instance, the rational actor model can be used touted as a central implication of rigorous bio-to explain collective action (Olson, 1965), since logical modeling. In The Selfish Gene, foragents may place positive value on the process of instance, Richard Dawkins (1976) asserts, ‘‘Weacquisition (for instance, ‘‘fighting for one’s are survival machines—robot vehicles blindlyrights’’), and can value punishing those who programmed to preserve the selfish moleculesrefuse to join in the collective action (Moore, known as genes. . . . Let us try to teach generosity1978; Wood, 2003). Indeed, contemporary and altruism, because we are born selfish.’’ (p. 1)experimental work indicates that one can apply Similarly, in The Biology of Moral Systems, R. D.standard choice theory, including the derivation Alexander (1987, p. 3) asserts, ‘‘Ethics, morality,of demand curves, plotting concave indifference human conduct, and the human psyche are to becurves, and finding price elasticities, for such understood only if societies are seen as collec-preferences as charitable giving and punitive tions of individuals seeking their own self-retribution (Andreoni & Miller, 2002). interest.’’ More poetically, Michael Ghiselin As a result of its maturation of game theory (1974) writes, ‘‘No hint of genuine charity ame-over the past quarter century, game theory is liorates our vision of society, once sentiment-well positioned to serve as a bridge across the alism has been laid aside. What passes forbehavioral sciences, providing both a lexicon for cooperation turns out to be a mixture of oppor-communicating across fields with divergent and tunism and exploitation. . .. Scratch an altruist,incompatible conceptual systems and a theore- and watch a hypocrite bleed.’’tical tool for formulating a model of human In economics, the notion that enlightenedchoice that can serve all the behavioral self-interest allows agents to cooperate in largedisciplines. groups goes back to Bernard Mandeville’s ‘‘private vices, public virtues’’ (Mandeville, 1705) and Adam Smith’s ‘‘invisible hand’’ (Smith,EXPERIMENTAL GAME THEORY 1759). Full analytical development of this ideaAND NON–SELF-REGARDING awaited the twentieth-century development ofPREFERENCES general equilibrium theory (Arrow & Debreu,Contemporary biological theory maintains that 1954; Arrow & Hahn, 1971) and the theory ofcooperation can be sustained based on inclusive repeated games (Axelrod & Hamilton, 1981;fitness, or cooperation among close genealogical Fudenberg & Maskin, 1986). So powerful inkin (Hamilton, 1963) by individual self-interest economic theory is the notion that cooperationin the form of reciprocal altruism (Trivers, 1971). among self-regarding agents is possible that it isReciprocal altruism occurs when an agent helps hard to find even a single critic of the notion inanother agent, at a fitness cost to itself, with the the literature, even among those that are other-expectation that the beneficiary will return the wise quite harsh in their evaluation of neoclas-favor in a future period. The explanatory power sical economics.of inclusive fitness theory and reciprocal By contrast, sociological, anthropological,altruism convinced a generation of biologists and social psychological theory generally explainthat what appears to be altruism—personal that human cooperation is predicated uponsacrifice on behalf of others—is really just long- affiliative behaviors among group members,run genetic self-interest. Combined with a vig- each of whom is prepared to sacrifice a modicumorous critique of group selection (Dawkins, of personal well-being to advance the collective1976; Maynard Smith, 1976; Williams, 1966), a goals of the group. The vicious attack on ‘‘socio-generation of biologists became convinced that biology’’ (Segerstrale, 2001) and the widespreadtrue altruism—one organism sacrificing fitness rejection of the bare-bones Homo economicus in

168 PRIMATE NEUROETHOLOGYthe ‘‘soft’’ social sciences (DiMaggio, 1994; whom we will call the Advisor, is shown theEtzioni, 1985; Hirsch et al., 1990) is due in part contents of two envelopes, labeled A and B.to this clash of basic explanatory principles. Each envelope has two compartments, the first Behavioral game theory assumes the BPC containing money to be given to the Advisor,model of choice and subjects individuals to stra- and the other to be given to player 2. We will calltegic settings, such that their behavior reveals player 2 the Chooser, because this player gets totheir underlying preferences. This controlled set- choose which of the two envelopes will be dis-ting allows us to adjudicate between these contributed to the two players. The catch, however,trasting models. One behavioral regularity that is that the Chooser is not permitted to see thehas been found thereby is strong reciprocity, contents of the envelopes. Rather, the Advisor,which is a predisposition to cooperate with who did see the contents, was required to adviseothers and to punish those who violate the the Chooser which envelope to pick.norms of cooperation, at personal cost, even The games all begin with the experimenterwhen it is implausible to expect that these costs showing both players the two envelopes, andwill be repaid. Strong reciprocity is other- asserting that one of the envelopes is better forregarding, as a strong reciprocator’s behavior the Advisor and the other is better for thereflects a preference to cooperate with other Chooser. The Advisor is then permitted tocooperators and to punish noncooperators, inspect the contents of the two envelopes, andeven when these actions are personally costly. say to the Chooser either ‘‘A will earn you more The result of the laboratory and field research money than B,’’ or ‘‘B will earn you more moneyon strong reciprocity is that humans indeed often than A.’’ The Chooser then picks either A or B,behave in ways that have traditionally been and the game is over.affirmed in sociological theory and denied in Suppose both players are self-regarding, eachbiology and economics (Andreoni, 1995; Fehr & caring only about how much money he earns ¨Gachter, 2000, 2002; Fehr et al., 1997, 1998; from the transaction. Suppose also that both ¨Gachter & Fehr, 1999; Henrich et al., 2005; players believe their partner is self-regarding.Ostrom et al., 1992). Moreover, it is probable The Chooser will then reason that the Advisorthat this other-regarding behavior is a prerequi- will say whatever induces him, the Chooser, tosite for cooperation in large groups of nonkin, choose the envelope that gives him, the Chooser,since the theoretical models of cooperation in the lesser amount of money. Therefore, nothinglarge groups of self-regarding nonkin in biology the Advisor says should be believed, and theand economics do not apply to some important Chooser should just make a random pickand frequently observed forms of human coop- between the two envelopes. The Advisor caneration (Boyd & Richerson, 1992; Gintis, 2005). anticipate the Chooser’s reasoning, and will pick randomly which envelope to advise the Chooser to choose. Economists call theCharacter Virtues in the Laboratory Advisor’s message ‘‘cheap talk,’’ because itAnother form of prosocial behavior conflicting costs nothing to give, but is worth nothing towith the maximization of personal material gain either party.is that of maintaining such character virtues as By contrast, suppose the Chooser believeshonesty and promise keeping, even when there is that the Advisor places a positive value on trans-no chance of being penalized for unvirtuous mitting honest messages, and so will be predis-behavior. Our first example of non–self- posed to follow whatever advice he is given, andregarding behavior will be of this form. suppose the Advisor does value honesty, and Gneezy (2005) studied 450 undergraduate believes that the Chooser believes that he valuesparticipants paired off to play several games of honesty, and hence will follow the Advisor’sthe following form. There are two players who suggestion. Then, the Advisor will weigh thenever see each other (anonymity) and they financial gain from lying against the cost ofinteract exactly once (one shot). Player 1, lying, and unless the gain is sufficiently large,

THE FOUNDATIONS OF TRANSDISCIPLINARY BEHAVIORAL SCIENCE 169he will tell the truth, the Chooser will believe willingness to lie increasing with an increasedhim, and the Chooser will get his preferred cost of truth telling to themselves, andpayoff. decreasing with an increase in their partner’s Gneezy (2005) implemented this experiment cost of being deceived.as a series of three games with the aforemen- Similar results were found by Boles and col-tioned structure (his detailed protocols were leagues (2000) and Charness and Dufwenbergslightly different). The first game, which we will (2004). Gunnthorsdottir and colleagues (2002)write A = (6,5), B = (5,6), pays the Advisor 6 and and Burks and colleagues (2003) have shownthe Chooser 5 if the Chooser picks A, and the that a social-psychological measure of ‘‘Machi-reverse if the Chooser picks B. The second game, avellianism’’ predicts which subjects are likely toA = (6,5), B = (5,15), pays the Advisor 6 and the be trustworthy and trusting, although theirChooser 5 if the Chooser picks A, but pays the results are not completely compatible.Advisor 5 and the Chooser 15 if the Chooserpicks B. The third game, A = (15,5), B = (5,15), The Public Goods Gamepays the Advisor 15 and the Chooser 5 if theChooser picks A, but pays the Advisor 15 and the The public goods game has been analyzed in aChooser 5 if the Chooser picks B. series of papers by the social psychologist Toshio Before having the subjects play any of the Yamagishi (1986, 1988a,b), by the politicalgames, he attempted to determine whether scientist Elinor Ostrom and her coworkersAdvisors believed that their advice would be (Ostrom et al., 1992), and by economists Ernstfollowed, because, if they did not believe this, ¨ Fehr and his coworkers (Fehr & Gachter, 2000,then it would be a mistake to interpret their ¨ 2002; Gachter & Fehr, 1999). These researchersgiving advice favorable to Choosers to the uniformly found that groups exhibit a muchAdvisor’s honesty. Gneezy elicited truthful higher rate of cooperation than can be expectedbeliefs from Advisors by promising to pay an assuming the standard economic model of the self-additional sum of money at the end of the ses- regarding actor, and this is especially the casesion to each Advisor who correctly predicted when subjects are given the option of incurringwhether his advice would be followed. He a cost to themselves in order to punish freefound that 82% of Advisors expected their riders.advice to be followed. In fact, the Advisors A typical public goods game consists of awere remarkably accurate, since the actual per- number of rounds, say 10. The subjects are toldcent was 78%. the total number of rounds, as well as all other The most honesty was elicited in game 2, aspects of the game. The subjects are paid theirwhere A = (5,15) and B = (6,5), so lying was winnings in real money at the end of the session.very costly to the Chooser and the gain to lying In each round, each subject is grouped withfor the Advisor was small. In this game, a full several other subjects—say three others—under83% of Advisors were honest. In game 1, where conditions of strict anonymity. Each subject isA = (5,6) and B = (6,5), so the cost of lying to the then given a certain number of ‘‘points,’’ say 20,Chooser was small and equal to the gain to the redeemable at the end of the experimental ses-Advisor, 64% of the Advisors were honest. In sion for real money. Each subject then placesother words, subjects were loathe to lie, but some fraction of his or her points in aconsiderably more so when it was costly to ‘‘common account’’ and the remainder in thetheir partner. In game three, where A = (5,15) subject’s ‘‘private account.’’ The experimenterand B =(15,5), so the gain from lying was large then tells the subjects how many points werefor the Advisor and equal to the loss to the contributed to the common account, and addsChooser, only 48% of the Advisors were to the private account of each subject some frac-honest. This shows that many subjects are tion, say 40%, of the total amount in thewilling to sacrifice material gain to avoid lying common account. So if a subject contributesin a one-shot, anonymous interaction, their his or her whole 20 points to the common

170 PRIMATE NEUROETHOLOGYaccount, each of the four group members will Sato, 1987; Yamagishi, 1988a,b, 1992). Forreceive eight points at the end of the round. In instance, in the Ostrom et al. (1992) study sub-effect, by putting the whole endowment into the jects interacted for 25 periods in a public goodscommon account, a player loses 12 points but game, and by paying a ‘‘fee,’’ subjects couldthe other three group members gain in total 24 impose costs on other subjects by ‘‘fining’’(= 8 Â 3) points. The players keep whatever is in them. Since fining costs the individual whotheir private account at the end of the round. uses it, but the benefits of increased compliance A self-regarding player will contribute nothing accrue to the group as a whole, the only Nashto the common account. However, only a fraction equilibrium in this game that does not dependof subjects in fact conform to the self-interest on incredible threats is for no player to pay themodel. Subjects begin by contributing on average fee, so no player is ever punished for defecting,about half of their endowment to the public and all players defect by contributing nothing toaccount. The level of contributions decays over the common pool. However, the authors foundthe course of the 10 rounds, until in the final a significant level of punishing behavior.rounds most players are behaving in a self- These studies allowed individuals to engageregarding manner (Dawes & Thaler, 1988; in strategic behavior, since costly punishment ofLedyard, 1995). In a meta-study of 12 public defectors could increase cooperation in futuregoods experiments, Fehr and Schmidt (1999) periods, yielding a positive net return for thefound that in the early rounds, average and ¨ punisher. Fehr and Gachter (2000) set up anmedian contribution levels ranged from 40% to experimental situation in which the possibility60% of the endowment, but in the final period of strategic punishment was removed. They used73% of all individuals (N = 1,042) contributed 6- and 10-round public goods games with groupnothing, and many of the remaining players con- sizes of four, and with costly punishmenttributed close to zero. These results are not com- allowed at the end of each round, employingpatible with the selfish actor model, which three different methods of assigning memberspredicts zero contribution on all rounds, though to groups. There were sufficient subjects to runthey might be predicted by a reciprocal altruism between 10 and 18 groups simultaneously.model, since the chance to reciprocate declines as Under the Partner treatment, the four subjectsthe end of the experiment approaches. However, remained in the same group for all 10 periods.this is not in fact the explanation of moderate but Under the Stranger treatment, the subjects weredeteriorating levels of cooperation in the public randomly reassigned after each round. Finally,goods game. under the Perfect Stranger treatment, the subjects The explanation of the decay of cooperation were randomly reassigned and assured that theyoffered by subjects when debriefed after the would never meet the same subject more thanexperiment is that cooperative subjects became once. Subjects earned an average of about $35angry at others who contributed less than them- for an experimental session.selves, and retaliated against free-riding low con- ¨ Fehr and Gachter (2000) performed theirtributors in the only way available to them—by experiment for 10 rounds with punishmentlowering their own contributions (Andreoni, and 10 rounds without.5 Their results are illu-1995). This view is confirmed by the fact that strated in Figure 9.1. We see that when costlywhen subjects play the repeated public goods punishment is permitted, cooperation does notgame sequentially several times, each time they deteriorate, and in the Partner game, despitebegin by cooperating at a high level, and their strict anonymity, cooperation increases almostcooperation declines as the end of the game to full cooperation, even on the final round.approaches. When punishment is not permitted, however, Experimental evidence supports this inter- the same subjects experience the deteriorationpretation. When subjects are allowed to punish of cooperation found in previous public goodsnoncontributors, they do so at a cost to them- games. The contrast in cooperation ratesselves (Orbell, Dawes, & Van de Kragt, 1986; between the Partner and the two Stranger

THE FOUNDATIONS OF TRANSDISCIPLINARY BEHAVIORAL SCIENCE 171 20 u Pnishment u Pnishment option permitted removed 18 16 artner P Average contribution 14 12 Stranger artner P 10 erfect stranger P 8 Stranger 6 4 2 erfect stranger P 0 0 2 4 6 8 10 12 14 16 18 20 eriod PFigure 9.1 Average contributions over time in the Partner, Stranger, and Perfect Stranger treatments ¨when the punishment condition is played first. Adapted from Fehr, E., & Gachter, S. (2000). Cooperationand punishment. American Economic Review, 90, 980–994. Used with permission.treatments is worth noting, because the strength have outlined several of the key ideas needed toof punishment is roughly the same across all specify a unified analytical framework for thetreatments. This suggests that the credibility of behavioral sciences.the punishment threat is greater in the Partnertreatment because in this treatment the pun-ished subjects are certain that, once they havebeen punished in previous rounds, the pun- NOTESishing subjects are in their group. This resultfollows from the fact that a majority of subjects 1. The arguments in this paper are developed inshowed themselves to be strong reciprocators, greater depth in Gintis (2007).both contributing a large amount and enthusias- 2. I use the term ‘‘self-regarding’’ rather than ‘‘self-tically punishing noncontributors. The prosoci- interested’’ (and similarly ‘‘non–self-regarding’’ or ‘‘other-regarding’’ rather than ‘‘non–self-ality impact of strong reciprocity on cooperation interested’’ or ‘‘unselfish’’) for a situation inis thus more strongly manifested the more which the payoffs to other agents are valued bycoherent and permanent the group in question. an agent. 3. This argument was presented verbally by Darwin (1872) and is implicit in the standardCONCLUSION notion of ‘‘survival of the fittest,’’ but formal proof is recent (Grafen, 1999, 2000, 2002). TheI have shown that the core theoretical constructs case with frequency-dependent (nonadditiveof the various behavioral disciplines currently genetic) fitness has yet to be formallyinclude mutually contradictory principles, but demonstrated, but the informal arguments inthat progress over the past couple of decades this case are no less strong.has generated the instruments necessary to 4. For a more extensive analysis of the parallelsresolve the interdisciplinary contradictions. I between cultural and genetic evolution, see

172 PRIMATE NEUROETHOLOGY Mesoudi et al. (2006). I have borrowed heavily Bowles, S., & Gintis, H. (2005). Prosocial emotions. from this paper in this section. In: L. E. Blume & S. N. Durlauf (Eds.), The5. For additional experimental results and analysis, economy as an evolving complex system III. see Bowles and Gintis (2002) and Fehr and Santa Fe, NM: Santa Fe Institute. ¨ Gachter (2002). Boyd, R., & Richerson, P. J. (1985). Culture and the evolutionary process. Chicago: University of Chicago Press. Boyd, R., & Richerson, P. J. (1992). PunishmentREFERENCES allows the evolution of cooperation (or anything else) in sizeable groups. Ethology andAbbott, R. J., James, J. K., Milne, R. I., & Gillies, A. Sociobiology, 113, 171–195. C. M. (2003). Plant introductions, hybridization Brown, J. H., & Lomolino, M. V. (1998). and gene flow. Philosophical Transactions of the Biogeography. Sunderland, MA: Sinauer. Royal Society of London B, 358, 1123–1132. Burks, S. V., Carpenter, J. P., & Verhoogen, E.Alcock, J. (1993). Animal behavior: An evolutionary (2003). Playing both roles in the trust game. approach. Sunderland, MA: Sinauer. Journal of Economic Behavior and Organization,Alexander, R. D. (1987). The biology of moral 51, 195–216. systems. New York: Aldine. Camille, N. (2004). The involvement of theAllman, J., Hakeem, A., & Watson, K. (2002). Two orbitofrontal cortex in the experience of regret. phylogenetic specializations in the human Science, 304, 1167–1170. brain. Neuroscientist, 8, 335–346. Cavalli-Sforza, L. L., & Feldman, M. W. (1981).Andreoni, J. (1995). Cooperation in public goods Cultural transmission and evolution. Princeton, experiments: Kindness or confusion. American NJ: Princeton University Press. Economic Review, 85, 891–904. Cavalli-Sforza, L. L., & Feldman, M. W. (1982).Andreoni, J., & Miller, J. H. (2002). Giving Theory and observation in cultural according to garp: An experimental test of the transmission. Science, 218, 19–27. consistency of preferences for altruism. Coleman, J. S. (1990). Foundations of social theory. Econometrica, 70, 737–753. Cambridge, MA: Belknap.Arrow, K. J., & Debreu, G. (1954). Existence of an Darwin, C. (1872). The origin of species by means of equilibrium for a competitive economy. natural selection (6th ed.). London: John Econometrica, 22, 265–290. Murray.Arrow, K. J., & Hahn, F. (1971). General competitive Dawes, R. M., & Thaler, R. (1988). Cooperation. analysis. San Francisco: Holden-Day. Journal of Economic Perspectives, 2, 187–197.Axelrod, R., & Hamilton, W. D. (1981). The Dawkins, R. (1976). The selfish gene. Oxford: evolution of cooperation. Science, 211, Oxford University Press. 1390–1396. Dawkins, R. (1982). The extended phenotype:Beer, J. S., Heerey, E. A., Keltner, D., Skabini, D., & The gene as the unit of selection. Oxford: Knight, R. T. (2003). The regulatory function of Freeman. self-conscious emotion: Insights from patients DiMaggio, P. (1994). Culture and economy. In: with orbitofrontal damage. Journal of N. Smelser & R. Swedberg (Eds.), The Personality and Social Psychology, 65, 594–604. handbook of economic sociology (pp. 27–57).Black, F., & Scholes, M. (1973). The pricing of Princeton, NJ: Princeton University Press. options and corporate liabilities. Journal of Durham, W. H. (1991). Coevolution: Genes, culture, Political Economy, 81, 637–654. and human diversity. Stanford, CA: StanfordBoles, T. L., Croson, R. T. A., & Murnighan, J. K. University Press. (2000). Deception and retribution in repeated Eshel, I., & Feldman, M. W. (1984). Initial increase ultimatum bargaining. Organizational Behavior of new mutants and some continuity properties and Human Decision Processes, 83, 235–259. of ESS in two locus systems. AmericanBonner, J. T. (1984). The evolution of culture in Naturalist, 124, 631–640. animals. Princeton, NJ: Princeton University Eshel, I., Feldman, M. W., & Bergman, A. (1998). Press. Long-term evolution, short-term evolution,Bowles, S., & Gintis, H. (2002). Homo reciprocans. and population genetic theory. Journal of Nature, 415, 125–128. Theoretical Biology, 191, 391–396.

THE FOUNDATIONS OF TRANSDISCIPLINARY BEHAVIORAL SCIENCE 173Etzioni, A. (1985). Opening the preferences: A Glimcher, P. W. (2003). Decisions, uncertainty, socio-economic research agenda. Journal of and the brain: The science of neuroeconomics. Behavioral Economics, 14, 183–205. Cambridge, MA: MIT Press. ¨Fehr, E., & Gachter, S. (2000). Cooperation and Glimcher, P. W., Dorris, M. C., & Bayer, H. M. punishment. American Economic Review, 90, (2005). Physiological utility theory and the 980–994. neuroeconomics of choice. New York: Center for ¨Fehr, E., & Gachter, S. (2002). Altruistic Neural Science, New York University. punishment in humans. Nature, 415, 137–140. Gneezy, U. (2005). Deception: The role of ¨Fehr, E., Gachter, S., & Kirchsteiger, G. (1997). consequences. American Economic Review, 95, Reciprocity as a contract enforcement device: 384–394. Experimental evidence. Econometrica, 65, 833–860. Grafen, A. (1999). Formal darwinism, theFehr, E., Kirchsteiger, G., & Riedl, A. (1998). Gift individual-as-maximizing-agent analogy, and exchange and reciprocity in competitive bet-hedging. Proceedings of the Royal Society B, experimental markets. European Economic 266, 799–803. Review, 42, 1–34. Grafen, A. (2000). Developments of price’sFehr, E., & Schmidt, K. M. (1999). A theory of equation and natural selection under fairness, competition, and cooperation. uncertainty. Proceedings of the Royal Society B, Quarterly Journal of Economics, 114, 817– 267, 1223–1227. 868. Grafen, A. (2002). A first formal link between theFeldman, M. W., & Zhivotovsky, L. A. (1992). price equation and an optimization program. Gene-culture coevolution: Toward a general Journal of Theoretical Biology, 217, 75–91. theory of vertical transmission. Proceedings Gunnthorsdottir, A., McCabe, K., & Smith, V. of the National Academy of Sciences, 89, (2002). Using the machiavellianism instrument 11935–11938. to predict trustworthiness in a bargaining game.Fisher, R. A. (1930). The genetical theory of natural Journal of Economic Psychology, 23, 49–66. selection. Oxford: Clarendon Press. Haldane, J. B. S. (1932). The Causes of Evolution.Fudenberg, D., & Maskin, E. (1986). The folk London: Longmans, Green Co. theorem in repeated games with discounting Hamilton, W. D. (1963). The evolution of altruistic or with incomplete information. Econometrica, behavior. American Naturalist, 96, 354–356. 54, 533–554. Hammerstein, P. (1996). Darwinian adaptation, ¨Gachter, S., & Fehr, E. (1999). Collective action as a population genetics and the streetcar theory of social exchange. Journal of Economic Behavior evolution. Journal of Mathematical Biology, 34, and Organization, 39, 341–369. 511–532.Ghiselin, M. T. (1974). The economy of nature and Hammerstein, P., & Selten, R. (1994). Game theory the evolution of sex. Berkeley, CA: University of and evolutionary biology. In: R. J. Aumann & S. California Press. Hart (Eds.), Handbook of game theory withGigerenzer, G., & Selten, R. (2001). Bounded economic applications (pp. 929–993). rationality. Cambridge, MA: MIT Press. Amsterdam: Elsevier.Gintis, H. (2000). Game theory evolving. Princeton, Hechter, M., & Kanazawam S. (1997). Sociological NJ: Princeton University Press. rational choice. Annual Review of Sociology, 23,Gintis, H. (2003). Solving the puzzle of human 199–214. prosociality. Rationality and Society, 15, 155– Henrich, J., Boyd, R., Bowles, S., Camerer, C., Fehr, 187. E., & Gintis, H. (2005). Economic man’ inGintis, H. (2005). Behavioral game theory and cross-cultural perspective: Behavioral contemporary economic theory. Analyze experiments in 15 small-scale societies. Kritik, 27, 48–72. Behavioral and Brain Sciences, 28, 795–815.Gintis, H. (2007). A framework for the unification Henrich, J., & Gil-White, F. (2001). The evolution of the behavioral sciences. Behavioral and Brain of prestige: Freely conferred status as a Sciences, 30, 1–61. mechanism for enhancing the benefits ofGintis, H., Bowles, S., Boyd, R., & Fehr, E. (2005). cultural transmission. Evolution and Human Moral sentiments and material interests: On the Behavior, 22, 165–196. foundations of cooperation in economic life. Hirsch, P., Michaels, S., & Friedman, R. (1990). Cambridge, MA: MIT Press. Clean models vs. dirty hands: Why economics

174 PRIMATE NEUROETHOLOGY is different from sociology. In: S. Zukin & Mandeville, B. (1705). The fable of the bees: Private DiMaggio (Eds.), Structures of capital: The vices, publick benefits. Oxford: Clarendon. social organization of the economy (pp. 39–56). Maynard Smith, J. (1976). Group selection. New York: Cambridge University Press. Quarterly Review of Biology, 51, 277–283.Holden, C. J. (2002). Bantu language trees reflect Maynard Smith, J. (1982). Evolution and the theory the spread of farming across sub-saharan of games. Cambridge: Cambridge University africa: A maximum-parsimony analysis. Press. Proceedings of the Royal Society of London Mednick, S. A., Kirkegaard-Sorenson, L., Series B, 269, 793–799. Hutchings, B., Knop, J., Rosenberg, R., &Holden, C. J., & Mace, R. (2003). Spread of cattle Schulsinger, F. (1977). An example of bio- led to the loss of matrilineal descent in Africa: A social interaction research: The interplay of coevolutionary analysis. Proceedings of the Royal socio-environmental and individual factors in Society of London Series B, 270, 2425–2433. the etiology of criminal behavior. In: S. A.Holland, J. H. (1975). Adaptation in natural and Mednick & K. O. Christiansen (Eds.), Biosocial artificial systems. Ann Arbor, MI: University of bases of criminal behavior (pp. 9–24). New York: Michigan Press. Gardner Press.Huxley, J. S. (1955). Evolution, cultural and Meltzhoff, A. N., & Decety, J. (2003). What biological. Yearbook of Anthropology, 2–25. imitation tells us about social cognition: AJablonka, E., & Lamb, M. J. (1995). Epigenetic rapprochement between developmental inheritance and evolution: The Lamarckian case. psychology and cognitive neuroscience. Oxford: Oxford University Press. Philosophical Transactions of the Royal SocietyJames, W. (1880). Great men, great thoughts, and the of London B, 358, 491–500. environment. Atlantic Monthly, 46, 441–459). Mesoudi, A., Whiten, A., & Laland, K. N. (2006).Jaynes, E. T. (2003). Probability theory: The logic of Towards a unified science of cultural evolution. science. Cambridge: Cambridge University Behavioral and Brain Sciences, 29, 329–383. Press. Miller, B. L., Darby, A., Benson, D. F., Cummings,Kollock, P. (1997). Transforming social dilemmas: J. L., & Miller, M. H. (1997). Aggressive, socially Group identity and cooperation. In: P. disruptive and antisocial behaviour associated Danielson (Ed.), Modeling rational and moral with fronto-temporal dementia. British Journal agents. Oxford: Oxford University Press. of Psychiatry, 170, 150–154.Krebs, J. R., & Davies, N. B. (1997). Behavioral Moll, J., Zahn, R., di Oliveira-Souza, R., Krueger, ecology: An evolutionary approach (4th ed.). F., & Grafman, J. (2005). The neural basis of Oxford: Blackwell Science. human moral cognition. Nature Neuroscience,Kreps, D. M. (1990). A course in microeconomic 6, 799–809. theory. Princeton, NJ: Princeton University Montague, P. R., & Berns, G. S. (2002). Neural Press. economics and the biological substrates ofLedyard, J. O. (1995). Public goods: A survey of valuation. Neuron, 36, 265–284. experimental research. In: J. H. Kagel & A. E. Moore, B., Jr. (1978). Injustice: The social bases of Roth (Eds.), The handbook of experimental obedience and revolt. White Plains, NY: M.E. economics (pp. 111–194). Princeton, NJ: Sharpe. Princeton University Press. Moran, P. A. P. (1964). On the nonexistence ofLewontin, R. C. (1974). The genetic basis of adaptive topographies. Annals of Human evolutionary change. New York: Columbia Genetics, 27, 338–343. University Press. Newman, M., Barabasi, A. L., & Watts, D. J. (2006).Liberman, U. (1988). External stability and ess criteria The structure and dynamics of networks. for initial increase of a new mutant allele. Journal Princeton, NJ: Princeton University Press. of Mathematical Biology, 26, 477–485. O’Brien, M. J., & Lyman, R. L. (2000). ApplyingLumsden, C. J., & Wilson, E. O. (1981). Genes, evolutionary archaeology. New York: Kluwer mind, and culture: The coevolutionary process. Academic. Cambridge, MA: Harvard University Press. Odling-Smee, F. J., Laland, K. N., & Feldman, M.Mace, R., & Pagel, M. (1994). The comparative W. (2003). Niche construction: The neglected method in anthropology. Current process in evolution. Princeton, NJ: Princeton Anthropology, 35, 549–564. University Press.

THE FOUNDATIONS OF TRANSDISCIPLINARY BEHAVIORAL SCIENCE 175Olson, M. (1965). The logic of collective action: Schulkin, J. (2000). Roots of social sensitivity and Public goods and the theory of groups. neural function. Cambridge, MA: MIT Press. Cambridge, MA: Harvard University Press. Schultz, W., Dayan, P., & Montague, P. R. (1997).Orbell, J. M., Dawes, R. M., & Van de Kragt, J. C. A neural substrate of prediction and reward. (1986). Organizing groups for collective Science, 275, 1593–1599. action. American Political Science Review, 80, Segerstrale, U. (2001). Defenders of the truth: The 1171–1185. sociobiology debate. Oxford: Oxford UniversityOstrom, E., Walker, J., & Gardner, R. (1992). Press. Covenants with and without a sword: Self- Shennan, S. (1997). Quantifying archaeology. governance is possible. American Political Edinburgh: Edinburgh University Press. Science Review, 86, 404–417. Simon, H. (1972). Theories of bounded rationalityParker, A. J., & Newsome, W. T. (1998). Sense and In: C. B. McGuire & R. Radner (Eds.), Decision the single neuron: Probing the physiology of and organization (pp. 161–176). New York: perception. Annual Review of Neuroscience, 21, American Elsevier. 227–277. Skibo, J. M., & Bentley, R. A. (2003). ComplexParsons, T (1964). Evolutionary universals in society. systems and archaeology. Salt Lake City: American Sociological Review, 29, 339–357. University of Utah Press.Popper, K. (1979). Objective knowledge: An Smith, A. (1759). The theory of moral sentiments. evolutionary approach. Oxford: Clarendon New York: Prometheus. Press. Smith, E. A., & Winterhalder, B. (1992).Poundstone, W. (1992). Prisoner’s dilemma. New Evolutionary ecology and human behavior. New York: Doubleday. York: Aldine de Gruyter.Real, L. A. (1991). Animal choice behavior and the Sugrue, L. P., Corrado, G. S., & Newsome, W. T. evolution of cognitive architecture. Science, 253, (2005). Choosing the greater of two goods: 980–986. Neural currencies for valuation and decisionReal, L., & Caraco, T. (1986). Risk and foraging in making. Nature Reviews Neuroscience, 6, 363–375. stochastic environments. Annual Review of Sutton, R., & Barto, A. G. (2000). Reinforcement Ecology and Systematics, 17, 371–390. learning. Cambridge, MA: MIT Press.Richerson, P. J., & Boyd, R. (1998). The evolution Taylor, P., & Jonker, L. (1978). Evolutionarily of ultrasociality. In: I. Eibl-Eibesfeldt & F. Salter stable strategies and game dynamics. (Eds.), Indoctrinability, idology and warfare (pp. Mathematical Biosciences, 40, 145–156. 71–96). New York: Berghahn Books. Trivers, R. L. (1971). The evolution of reciprocalRivera, M. C., & Lake, J. A. (2004). The ring of life altruism. Quarterly Review of Biology, 46, 35– provides evidence for a genome fusion origin of 57). eukaryotes. Nature, 431, 152–155. Von Neumann, J., & Morgenstern, O. (1944).Rizzolatti, G., Fadiga, L., Fogassi, L., & Gallese, V. Theory of games and economic behavior. (2002). From mirror neurons to imitation: Princeton, NJ: Princeton University Press. Facts and speculations. In: A. N. Meltzhoff & Williams, G. C. (1966). Adaptation and natural W. Prinz (Eds.), The imitative mind: selection: A critique of some current evolutionary Development, evolution and brain bases (pp. thought. Princeton, NJ: Princeton University 247–266). Cambridge: Cambridge University Press. Press. Williams, J. H. G., Whiten, A., Suddendorf, T., &Sato, K. (1987). Distribution and the cost of Perrett, D. I. (2001). Imitation, mirror neurons maintaining common property resources. and autism. Neuroscience and Biobeheavioral Journal of Experimental Social Psychology, 23, Reviews, 25, 287–295. 19–31. Winter, S. G. (1971). Satisficing, selection and theSchall, J. D., & Thompson, K. G. (1999). Neural innovating remnant. Quarterly Journal of selection and control of visually guided eye Economics, 85, 237–261. movements. Annual Review of Neuroscience, Wood, E. J. (2003). Insurgent collective action and 22, 241–259. civil war in El Salvador. Cambridge: Cambridge ¨Schrodinger, E. (1944). What is Life?: The Physical University Press. Aspect of the Living Cell. Cambridge: Cambridge Wright, S. (1931). Evolution in mendelian University Press. populations. Genetics, 6, 111–178.

176 PRIMATE NEUROETHOLOGYYamagishi, T. (1986). The provision of a (Eds.), Social dilemmas: Theoretical issues and sanctioning system as a public good. Journal of research findings (pp. 267–287). Oxford: Personality and Social Psychology, 51, 110–116. Pergamon Press.Yamagishi, T. (1988a). The provision of a Young, H. P. (1998). Individual strategy and sanctioning system in the United States and social structure: An evolutionary theory Japan. Social Psychology Quarterly, 51, 265–271. of institutions. Princeton, NJ: PrincetonYamagishi, T. (1988b). Seriousness of social University Press. dilemmas and the provision of a sanctioning Zajonc, R. B. (1980). Feeling and thinking: system. Social Psychology Quarterly, 51, 32–42. Preferences need no inferences. AmericanYamagishi, T. (1992). Group size and the provision Psychologist, 35, 151–175. of a sanctioning system in a social dilemma. Zajonc, R. B. (1984). On the primacy of affect. In: W. Liebrand, D. M. Messick, & H. Wilke American Psychologist, 39, 117–123.

CHAPTER 10Sensory and Motor Systems in PrimatesJon H. KaasINTRODUCTION brains with proportionately little neocortex. We can infer a lot about how sensory andThe task of describing the sensory and motor motor systems were organized in these mam-systems of primates, and how they interrelate to mals by seeing how features or traits of thesemediate complex behaviors, is clearly daunting. systems are presently distributed across pre-There are a number of systems, all of which are sent-day mammals (Kaas, 2007b). It has beenvariable in organization and function across especially informative to study the organizationsprimate taxa, and they are incompletely under- of these systems in those present-day mammalsstood, some more than others. In addition, this that have small brains and proportionately littlereview needs to be reasonably short, although cortex (Fig. 10.1). A survey of such brains indi-much can be said, as a recent six-volume series cates that early mammals likely had, as oneon sensory systems demonstrates (Basbaum et al., might expect, rather simple sensory and motor2008). Thus, this review is necessarily selective, systems, with few cortical areas devoted to eachwith a focus on the evolution of primate sensory system. Remarkably, there was probably noand motor systems, especially in regard to the motor cortex that was distinct from somatosen-evolution of a parietal-frontal cortical system sory cortex in the first mammals. Separate motorthat uses information produced by early stages areas appear to have emerged with the evolutionof sensory processing to guide ongoing behavior. of placental mammals. The major features ofWe start with an overview of the components of sensory and motor systems of early mammalssensory and motor systems that have been are outlined below.retained from non-primate ancestors, and pro-ceed to an attempt to reconstruct the organiza- The Visual Systemtion of these systems from early primates tobranches of the anthropoid radiation, including Early mammals had most of the subcorticalhumans. Aspects of the present discussion can be components of the visual system that are nowfound in previous reviews (Kaas, 2007ab; Kaas & found in primates. The retina had several classesPreuss, 2008; see also Preuss, this volume). of ganglion cells, and they projected to the superchiasmatic nucleus of the hypothalamus, the nuclei of the accessory optic system, theOUR LEGACY: SENSORY AND ventral lateral geniculate nucleus, the dorsal lat-MOTOR SYSTEMS OF EARLY eral geniculate nucleus, the pretectum, and theMAMMALS superior colliculus (Berson, 2008). The dorsalThe sensory and motor systems of early mam- lateral geniculate nucleus provided the majormals are of interest because early primates inher- activating input to visual cortex (Kaas et al.,ited these systems around 60 to 80 Mya and 1972). Retinal ganglion cell classes resemblingmodified them. Early mammals had small the P and M classes of primates (see later) 177

178 PRIMATE NEUROETHOLOGYFigure 10.1 A dorsolateral view of the brain of a Madagascar tenrec (Echinops telfairi), a smallAfrotherian mammal. Unlike most extant mammals, the neocortex is only a small cap on the brain. Theprimary visual (V1), somatosensory (S1), and auditory (A1) and motor (M1) areas have been identified, aswell as a secondary somatosensory area (S2). Cortex caudal to A1 may be visual (V?) or multisensory. Noticehow much of the forebrain is devoted to olfaction (olf trac = olfactory tract, olf tub = olfactory tubercle).Because of the limited extent of neocortex, the inferior colliculus (IC) and superior colliculus (SC) of the ¨midbrain are exposed. Based on Krubitzer, L.A., Kunzle, H., and Kaas, J. (1997). The organization of sensorycortex in a Madagascan insectivore, the tenrec (Echinaps telfairi). Journal of Comparative Neurology, 379,399–414.innervated about half of the neurons in the superior colliculus and visual cortex, and thisdorsal lateral geniculate, and retinal inputs information was relayed to visual cortex.were segregated to form cryptic contralaterally Another part of this poorly differentiated pul-and ipsilaterally innervated layers. A more vinar received inputs from visual cortex andcaudal portion of the geniculate received inputs relayed back to visual cortex. Thus, the visualfrom thin, more slowly conducting axons resem- pulvinar possibly had three divisions, and visualbling those of the primate K-cell class (see later). cortex had three or four. A sector of the reticularMost of the geniculate outputs were to primary nucleus of the ventral thalamus had inputs fromvisual cortex (V1), which projected broadly to visual cortex and the visual thalamus, projectingother visual and nonvisual areas of cortex, via inhibitory neurons into the visual thalamusincluding somatosensory and auditory fields (Crabtree & Killackey, 1989).(Wang & Burkhalter, 2007). Major visual targetswere the second visual area, V2, along the lateral The Somatosensory Systemborder of V1, and a small visual temporal region Early mammals retained from reptilian ances-caudolateral to V2. Cortex on the medial border tors most of the afferent systems that broughtof V1, a limbic or retrosplenial field, now iden- sensory information into the central nervoustified in primates as prostriata, also received V1 system (Kaas, 2007b). Obviously, new sensoryinputs (Lyon, 2007). These areas also distributed opportunities arose with the evolution of bodyvisual information to other regions of cortex, hair. Thus, early mammals had receptor affer-including somatosensory, auditory, motor, ents sensitive to touch, hair movement, vibra-frontal, limbic, and multisensory fields. Finally, tion, muscle and joint movement, temperature,the visual thalamus included a small pulvinar, and painful stimuli. Longer sensory hairsusually identified in nonprimates as the lateral (vibrissae) evolved to detect objects at short dis-posterior nucleus or complex. Different parts of tances from the skin via receptors around thethis small pulvinar received inputs from the base of each hair. In early mammals, afferents

SENSORY AND MOTOR SYSTEMS IN PRIMATES 179from the face, facial vibrissae, and mouth were of the ipsilateral mouth, the tongue and teeth,especially important, as they are in most mam- were also represented via the ipsilateral inputsmals today. to VPM. Sensory afferents from the skin and deeper Primary somatosensory cortex of early mam-tissue entered the spinal cord or brainstem, mals activated three or four adjoining fields.where they terminated in specific layers of the Cortex along the rostral and caudal margins ofdorsal horn of the spinal cord or in sensory brain- S1 were activated via topographically orderedstem nuclei (Fig. 10.2). The large fiber afferents projections in mediolateral sequences so thatrelated to tactile or muscle spindle receptors also they formed narrow somatosensory representa-sent branches that coursed in the brainstem or tions on the margins of S1. Here these areas aredorsal columns of the spinal cord to terminate in called the rostral (SR) and caudal (CR) somato-a complex of nuclei, the trigeminal–dorsal sensory areas (Fig. 10.1). In primates, these bor-column complex, at the junction of the cervical dering bands of cortex constitute area 3a, with aspinal cord and brainstem. The traditional gra- major involvement in proprioception, and areacile, cuneate, and trigeminal nuclei for tactile 1, a secondary tactile area. One or two areasinputs formed a medial to lateral sequence of lateral to S1 also received topographically orga-subnuclei of a single functional unit that system- nized projections from S1. One of these areas,atically represents the body from the hindlimb to the second somatosensory area (S2), has beenthe forelimb to the face and mouth. Neurons in described in all adequately explored mammals,this dorsal column–trigeminal nucleus projected and was certainly present in early mammals.to the ventroposterior nucleus of the contralateral Another adjoining area, the parietal ventralthalamus. Neurons representing at least part of area (PV), has been more recently identified inthe mouth, the tongue, and teeth projected to the mammals, and the comparative evidence indi-ipsilateral ventroposterior nucleus as well cates that it exists in a broad range of mammals,(Bombardieri et al., 1975). As in present-day and most probably was present in early mam-mammals, the ventroposterior nucleus (VP) mals. S2 and PV represent the contralateral bodyrepresented the body from hindlimb (and tail) as two small mirror images of each other. Bothto the mouth in a lateromedial sequence. In early S2 and PV received activating inputs from themammals, and most mammals today, many or ventroposterior nucleus, as well as from S1.most of the tactile inputs were from the face and These somatosensory areas distributed tomouth, and a large subnucleus of VP, distin- adjoining insular and cingulate and frontalguished as the ventroposterior medial ‘‘nucleus’’ areas of cortex. A small fringe of more posterior(VPM), represented the face and mouth. The cortex was all that could be considered posteriorremaining, more lateral part of VP, representing parietal cortex, and this region received somato-the rest of the body, is typically called the ventro- sensory and other sensory inputs, and had con-posterior lateral ‘‘nucleus’’ (VPL). Functionally, nections with frontal cortex. Connections alsoVPM and VPL are subdivisions of VP. reached perirhinal cortex and the hippocampus In all mammals, VP projects to primary (Fig. 10.2).somatosensory cortex (S1). In primates, this In addition to cortical representations of tac-cortex is distinguished as area 3b of anterior tile receptors, muscle spindle and joint receptorparietal cortex, and it is appropriate to use the information reached the somatosensory tha-anatomical term 3b for S1 in other mammals, lamus from separate subnuclei in the dorsalalthough various combinations of architectonic column–trigeminal complex. In primates, theterms for S1 are in current use. Comparative relayed proprioceptive information terminatesevidence indicates that S1 of early mammals in a separate nucleus of the contralateral tha-represented the contralateral body surface from lamus, the ventroposterior superior nucleustail to tongue in a mediolateral sequence in par- (VPS), but this nucleus has not been commonlyietal cortex, with the limbs facing forward (ros- distinguished in nonprimates. Yet, the availabletrally or anteriorly). As in mammals today, part evidence suggests that the proprioceptive inputs

Hippocampus Perirhinal PV S2 Cortex SR S1 SC RT VSP VP Thalamus Face Hand Foot VPI Brainstem Complex Hindbrain Face Hand Foot Receptors: Muscle spindle RA cutaneous Spinothalamic: SA cutaneous Touch, Pain, Temperature Other Spinal CordFigure 10.2 A diagram of the organization of the somatosensory system of early mammals. Processingstarts with receptors and afferents of the skin, muscles, and joints. Rapidly (RA) and slowly (SA) adaptingreceptors in the skin mediate aspects of touch. Muscle spindle receptors help mediate proprioception. Otherreceptors include those that mediate pain, temperature, and aspects of touch and movement. Afferentsterminate in the trigeminal–dorsal column complex where the face, hand, and foot are represented in alateromedial sequence, and muscle spindle inputs terminate in separate nuclei. Afferents also activate spinalcord neurons that project to the contralateral thalamus. The somatosensory thalamus includes theventroposterior nucleus for SA and RA inputs, and usually unnamed adjoining groups of cellscorresponding to the ventroposterior superior nucleus (VPS) of primates for muscle spindle inputs andthe ventroposterior inferior nucleus (VPI) of primates for spinothalamic inputs. The somatosensorysegment of the reticular nucleus (RT) interacts with cortex and these nuclei. Somatosensory cortexincludes the primary area, S1, with adjoining rostral somatosensory (SR) and caudal somatosensory (SC)areas, the second somatosensory area (S2), and the parietal ventral area (PV). 180

SENSORY AND MOTOR SYSTEMS IN PRIMATES 181to the thalamus of early mammals was likely to Tastehave been at least partly segregated in the dor- The gustatory sense is mediated by a specializedsorostral part of VP, becoming more distinctly part of the somatosensory system. As in present-separated later with the evolution of primates, day mammals, primary taste afferents from thecarnivores, and perhaps other lines of descent. taste buds on the tongue terminated in theHere, a VPS is shown as present in early mam- nucleus of the solitary tract, a nucleus thatmals (Fig. 10.2), although it was probably poorly receives various viscerosensory inputs over itsdifferentiated from VP. long extent through most of the medulla. One Another source of information to the somato- of the functions of the nucleus is to project tosensory thalamus was from second-order neu- neurons in the adjacent reticular formation,rons that form the spinothalamic tract for the where neurons in turn project to motor nucleibody and the functionally equivalent part of the involved in reflexes concerned with the accep-trigeminothalamic tract for the face and mouth. tance (licking) or rejection (tongue protrusion)Much of the information relayed in this pathway of palatable or aversive tastes. The nucleus of theis nociceptive and thermoreceptive, but tactile solitary tract also projected rostrally to the para-and proprioceptive information is included as brachial nucleus that in turn relayed to the lat-well (Dostrovsky & Craig, 2008). These thalamic eral hypothalamus and amygdala. Theinputs, representing the contralateral half of the parabrachial nucleus also provided taste, tactile,body, terminated in and around the ventropos- and viscerosensory information to the gustatoryterior nucleus. In primates, they largely terminate nucleus of the somatosensory thalamus, the par-in the ventroposterior inferior nucleus (VPI) and vocellular ventroposterior medial nucleuscell-poor septa that extend from this nucleus well (VPMpc). This nucleus projected to cortex justinto the ventroposterior nucleus to isolate sub- ventral to the mouth and face representations ofnuclei. However, VPI has not been distinguished S1, possibly including both the granular cortexin most mammals, although a VPI is well devel- belonging to the S1 tongue representation andoped in raccoons, and it can be identified in cats the dysgranular cortex corresponding to a repre-and a few other mammals. Here a VPI nucleus is sentation of taste and tactile inputs just ventralshown for early mammals (Fig. 10.2), although it to the S1 tongue representation and dorsal to themay not have been histologically distinct from rhinal fissure (Kosar et al., 1986; Norgren &VP. Probably, most mammals have a spinotha- Wolf, 1975). Thus, there may have been at leastlamic pathway that terminates largely on small two targets of VPMpc in early mammals, one inneurons on the ventral margin of VP and scat- the tongue representation in primary somato-tered within VP. These small cells in VPI and VP sensory cortex (see Remple et al., 2003) and oneappear to project to the superficial layers of S1, S2, in adjacent insular cortex. However, the repre-PV, and the rostral and caudal somatosensory sentation of the tongue in S1 is usually notbelt areas (e.g., Penny et al., 1982). Their major considered to be part of gustatory cortex.role may be to modulate neurons in these areas Gustatory cortex of insular cortex is thought toand signal stimulus intensity. Other neurons in project directly or indirectly to a higher-orderthis spinothalamic pathway terminated in poorly multisensory processing zone in orbitofrontaldefined groups of cells ventral and caudal to VP, cortex (Pritchard & Norgren, 2004).sometimes called the posterior group, whereinformation about temperature and pain is Pain and Temperaturerelayed to cortex caudal to S2 and PV in insularcortex. In addition, somatosensory visceral infor- A number of ascending systems that carry noci-mation in spinothalamic pathways was relayed ceptive and temperature information have beenvia a basal ventromedial thalamic nucleus to described in mammals (see for reviewpart of insular cortex, and from the parafasciculus Dostrovsky & Craig, 2008; Lima, 2008). Fornucleus to the striatum (Craig, 2002; Dostrovsky our purposes here, the relevant pathways are& Craig, 2008). those that reach the dorsal thalamus and are

182 PRIMATE NEUROETHOLOGYrelayed to cortex, as the cortex is thought to be the cochlear nuclei. These neurons providenecessary for consciously appreciating the information about sound intensity and fre-nature of painful stimuli and temperature quency. Disparities in the information relayedchange. Modifications and elaborations of noci- by the afferents from each cochlea allow centralceptive and temperature systems at the thalamic circuits to extract information about soundand cortical levels in primates allowed these sys- location. Auditory processing starts in the threetems to become involved in more behaviors divisions or ‘‘nuclei’’ of the cochlear nuclear(Craig, 2007). Limited comparative evidence complex, which relay to nuclei of higher levelssuggests that in all mammals, pain and tempera- in the brainstem of both sides. These include theture information is relayed from peripheral nuclei of the superior olivary complex, the nucleiafferents by trigemino- and spinothalamic proof the lateral lemniscus, and subdivisions of thejections to and around the ventroposterior inferior colliculus of the midbrain. The divisionsnucleus of the thalamus, the poorly defined pos- of the inferior colliculus project to the medialterior complex or nucleus, the intralaminar geniculate complex, where neurons relay tonuclei, and other targets, such as hypothalamus, auditory cortex. The ventral nucleus of theamygdala, and septal nuclei. Much of the infor- medial geniculate complex (MGv) projects tomation is relayed from thalamic targets to the auditory core of auditory cortex, the primaryinsular cortex just ventral to somatosensory area or areas, while other divisions, the dorsalcortex (S1 and S2), somatosensory cortex, and nucleus (MGd) and medial or magnocellularanterior cingulate cortex. Craig (2007) proposes nucleus (MGm), project more broadly to sec-that the primordial role of insular cortex was to ondary auditory and multisensory areas ofparticipate in the sensing of noxious and tem- cortex.perature stimuli and to modulate and control Most investigated mammals have an auditorybrainstem homeostatic integration sites, core of two to three primary or primary-likeincluding those associated with the autonomic areas, one of which (but not always the samenervous system. Somatosensory cortex probably one) has been identified as primary auditoryplayed a role in sensing these stimuli as well, but cortex, A1 (Fig. 10.1). The comparative evidencepossibly in terms of the intensities of noxious suggests that early mammals had a core of atstimuli, rather than the painful aspects. Inputs to least an anterior auditory field, AAF, and ancingulate cortex were probably indirect, but A1, surrounded by a narrow belt of several sec-important in limbic motor functions. ondary auditory fields. Each secondary field in turn involved, via connections, other regions ofThe Auditory and Vestibular Systems cortex in auditory and multisensory processing. Core areas were tonotopically organized, andThe auditory systems of mammals share a AAF and A1 were distinguished by havingnumber of components that likely were retained mirror-image or reversed patterns of tonotopicfrom an early mammal ancestor (Carr & Edds- organizations. In present-day mammals, theWalton, 2008; Kaas & Hackett, 2008). All tonotopic organizations of secondary auditorydepend on a peripheral auditory system that cortex are less precise, when they are present,includes an external ear that is generally mobile and often difficult to reveal.and a short canal ending at the tympanic mem- The sense of balance is mediated by the ves-brane. The presence in the middle ear of a chain tibular system (see for review Graf, 2007), whichof three small bones that transmit vibrations is not well understood at thalamic and corticalfrom the tympanic membrane to the sound levels of processing, partly because the systemwindow of the cochlea of the inner ear is a can be difficult to study and partly because wecharacteristic of mammals. The cochlea is a are largely unaware of its sensory functions incomplicated organ that allows the hair cells to postural control, reflexes, and the perception ofbe stimulated and activate the afferents of the self-movement. The organ for balance is part ofauditory nerve that terminate on the neurons of the inner ear, and it consists of semicircular

SENSORY AND MOTOR SYSTEMS IN PRIMATES 183canals and otoliths of the labyrinth. Hair cells in corticospinal projection system, and therebythe labyrinth are stimulated by the movement of has a more direct impact on motor behavior.fluid in the canals and in the otoliths. Afferents Most of the evidence for a second motor areain the vestibular nerve innervate vestibular in placental mammals comes from rats, wherenuclei, which provide inputs to a number of there is evidence for a second forelimb represen-control systems, such as those controlling eye tation rostral to M1 that has cortical and sub-movement reflexes. Vestibular information also cortical connections that differ from those of M1is relayed to the thalamus, and then to cortex, and are more similar to those of premotor cortexbut this network is poorly understood in most and the supplementary motor area of monkeysmammals. The thalamic and cortical levels of (Rouiller et al., 1993). Tree shrews, which areprocessing, which are largely multisensory, one of the closest living relatives of primates,have been more investigated in primates. have a strip of cortex along the rostral border of M1 that can be either considered aMotor Systems second parallel motor representation, M2, or possibly part of M1 (Remple et al., 2006).Judging from somewhat limited comparative Microstimulation of M2 required higher cur-evidence (see for review Beck et al., 1996; Nudo rents than M1 for evoked movements, and M2& Frost, 2007), early mammals did not have a had few corticospinal neurons, which were den-region of motor cortex that was distinct from sely distributed across M1, area 3a, and S1.somatosensory cortex. While more studies are Connection patterns suggest that tree shrewsneeded, motor cortex apparently did not emerge also have a motor area on the medial wall ofas a separate area or areas until eutherian mam- the cerebral hemisphere, possibly a cingulatemals evolved some 140 million years ago (Kaas, motor area. Thus, tree shrews appear to2007b). Judging from results from members of resemble primates more closely than rats inthe other two major branches of mammalian motor cortex organization. Studies of otherevolution, the monotremes and the marsupials, mammals would be useful in determining howprimary somatosensory cortex of early mam- motor cortex varies and is similar acrossmals received both somatosensory information mammalian taxa.from the ventroposterior nucleus of the tha-lamus and motor-related information from the Olfactory Systemcerebellum via projections from the ventrolat-eral thalamic nucleus, which is considered part Structures for processing olfactory informationof the motor thalamus. This early somatosensory were proportionately large and important forcortex influenced motor behavior via projec- early mammals. Olfactory receptor cells connecttions to the basal ganglia and other subcortical directly to the olfactory bulb, which in earlytargets, but the corticospinal pathway was rather mammals had a surface area as large as all ofpoorly developed (Nudo & Frost, 2007; Nudo & neocortex, as in tenrecs today (Fig. 10.1). TheMasterton, 1988). Comparative studies of pla- olfactory bulb, in turn, projected to a hugecental mammals, in contrast, indicate that most expanse of olfactory (piriform) cortex, whichor all have at least one distinct motor area, pri- had connections to frontal cortex via the dorsalmary motor cortex (M1), and likely a second thalamus (Wilson, 2008). In proportion to themotor area (M2), which may be homologous rest of the brain, the olfactory bulb and cortexto either premotor cortex or the supplementary are small in primates, especially in anthropoidmotor area of primates. In placental mammals, primates, where vision has become so importantM1 (Fig. 10.1) and S1 are distinguished by for the recognition of conspecifics, other ani-inputs from the ventral lateral nucleus of the mals, and food, and smell has become lessmotor thalamus and the ventroposterior nucleus important, especially in humans. Mammalsof the somatosensory thalamus, respectively. also have an accessory olfactory system withIn addition, M1 has a more developed inputs from the vomeronasal organ that is

184 PRIMATE NEUROETHOLOGYinvolved in pheromone detection and other che- eyes, a laminated lateral geniculate nucleusmosensory functions. In chimpanzees and and a number of pulvinar nuclei in the visualhumans, the vomeronasal organ and the rest of thalamus, and a visual cortex with a large pri-the accessory olfactory system are degenerate mary area and a number of additional visual(Bhatnagar & Smith, 2007) and nonfunctional. areas (Kaas, 2003; Kaas & Collins, 2004;Thus, males no longer sniff the urine of females Kremers, 2005). Early primates were nocturnal,to determine the state of their reproductive and many of the prosimian primates remaincycle. nocturnal, but the early anthropoid primates were diurnal, and all have remained diurnal except owl monkeys, which reverted to noc-SENSORY AND MOTOR SYSTEMS turnal life. Thus, all anthropoid primates,IN PRIMATES except owl monkeys, are specialized for diurnalThe sensory and motor systems of early mam- vision. This includes an emphasis on cone-mals have been modified and expanded in many rather than rod-mediated vision, a fovea inways in subsequent lines of mammalian evolu- the retina for detailed vision, and a great invest-tion. The altered auditory cortex of echolocating ment in the parvocellular pathway from thebats serves as one well-known example. Yet, the retina to the lateral geniculate nucleus, as thisgreatest changes are likely to be found in the pathway is devoted to the detailed, color visioncortical components of sensory and motor sys- that is used in object recognition. In macaquetems in those mammals with large brains and monkeys, and probably most other anthropoidgreatly expanded cortex, and humans are excep- primates, 80% of the retinal ganglion cells aretional in this regard. However, all primate brains devoted to the parvocellular (P) pathwayare exceptional in that they have many more (Weller & Kaas, 1989). The retina of primatesneurons than rodent brains of the same size also has two other types of outputs that are(Herculano-Houzel et al., 2007). Primates, in named after the layers they project to in theaddition, have a cortex that is subdivided into lateral geniculate nucleus (LGN); the magno-an unusually large number of cortical areas, and cellular or M-cell pathway and the koniocel-the number of areas appears to be greater in lular or K-cell pathway, each accounting forthose anthropoid primates with larger brains. about 10% of the retina’s output. The M-cellIn addition, many of their cortical areas are subsystem is specialized for the detection ofinvolved in sensory processing and motor changes in contrast, such as that caused by abehavior. blinking light or a moving object. The konio- Here, we consider sensory and motor sys- cellular system is not well understood, but ittems, especially at the cortical level, in members contains information from the blue (S) conesof the major branches of the primate radiation, and thereby is involved in color visionthe prosimian primates, New and Old World (Casagrande & Xu, 2004).monkeys, and apes and humans. The interesting The LGN of all primates is divided into twotarsiers, with a highly differentiated and pairs of parvocellular layers, one with inputsexpanded visual system (Collins et al., 2005a), from the ipsilateral eye and one with inputsare not considered here, as little is known about from the contralateral eye (Kaas et al., 1978).their other sensory systems or their motor In anthropoid primates other than owl mon-system. Tarsiers have evolved to be highly spe- keys, these layers are thick and they are subdi-cialized as visual predators of insects and small vided, producing four or more sublayers in thevertebrates. part of the nucleus devoted to central vision. The thinner magnocellular layers are also paired, one for each eye. The LGN of nocturnalThe Visual Systems of Primates prosimian primates has two well-developedAll primates are characterized by a well-devel- koniocellular layers, one for each eye, but theoped visual system, including forward-facing small cells of this system are not so distinct in

SENSORY AND MOTOR SYSTEMS IN PRIMATES 185anthropoid primates, and they are generally the contralateral eye (Fig. 10.3). This remarkableconsidered to be between layers (interlaminar modification of the visual system appears to becells) rather than forming layers. However, in related to an emphasis on frontal vision, but thenocturnal owl monkeys, a thick koniocellular implication of this change for the behavior inregion is found. Thus, the koniocellular early primates is not completely clear. Anotherpathway seems to be more important in noc- major modification is the huge projection to theturnal primates. Unlike some mammals, such superior colliculus from the many visual areas ofas cats and other carnivores, the LGN of pri- cortex (Collins et al., 2005b).mates projects almost exclusively (but not com- The pulvinar of primates contains inferior,pletely so) to a large primary visual area, V1. lateral, and medial divisions (Kaas & Huerta, The retinal K and M cells (but not the P cells 1988). The medial division has mixed functions,in anthropoid primates) also project to the judging from its cortical connections, but thesuperior colliculus, which in turn projects to lateral division of two nuclei and the inferiorthe LGN and parts of the visual pulvinar (Kaas division of four nuclei are all visual (Kaas && Huerta, 1988). The superior colliculus of all Lyon, 2007). The superior colliculus providesprimates is specialized for frontal, binocular K-cell and M-cell information to the pulvinarvision, and unlike other mammals, each colli- and K-cell information to the LGN. Most of theculus represents only the contralateral visual other activating inputs to pulvinar nuclei arehemifield, rather than the whole visual field of from visual cortex. Thus, the pulvinar complexFigure 10.3 The unique type of retinotopic organization of the superior colliculus that is shared by allprimates but differs from all studied nonprimates. The circles at the top represent the two eyes, while thelarge circle below represents a surface view of the left superior colliculus (SC). Arrows on the back of the eyeindicate the temporal part of the retina that projects to the ipsilateral SC in mammals (white arrows).In primates, this projection is largely overlapped by a projection from the larger nasal retina from thecontralateral eye (black arrow). The line of decussation (LD) marks the dividing point for these twoprojections. The LD runs vertically through the fovea. In nonprimates both the temporal segment of theretina of the contralateral eye (longer black arrow) and the temporal retina (short black arrow) project tothe contralateral superior colliculus, creating a representation of the complete retina. This extendedrepresentation may aid in the detection of movement and predators from all parts of the visual field,while the primate SC is more devoted to frontal vision.

186 PRIMATE NEUROETHOLOGYrelays superior colliculus information to visual of the other outputs of V1 are to a third visualcortex, and distributes visual cortex information area, V3, and the middle temporal visualback to visual cortex. Some of the possible func- area, MT.tions of such cortical-thalamic-cortical loops are Two of the classes of cell bands (modules) indiscussed by Sherman and Guillery (2005). The V2 project to the dorsolateral visual area (DL),larger nuclei of the pulvinar complex project to also known as V4. DL/V4 is a critically impor-early visual areas in the cortical processing hier- tant visual area in the so-called ventral stream ofarchy, those with mixed visual functions or an visual processing areas devoted to object visionemphasis on object vision, while a cluster of (Ungerleider & Mishkin, 1982). DL/V4 projectsinferior pulvinar nuclei, including those with in turn to inferior temporal cortex (IT), whichdense superior colliculus inputs, project to a consists of several visual areas. Parts of IT cortexcluster of temporal lobe visual areas concerned feed into memory-related areas that connectwith visual motion and using vision for guiding with the hippocampus, and to the prefrontalmotor behavior (Kaas & Lyon, 2007). cortex of the frontal lobe for ‘‘working All primates have a greatly expanded cortical memory’’ and other functions. Overall, a hugevisual system. This elaboration is greater in amount of cortex is devoted to this object visionanthropoid primates than in prosimians. cortical stream, even in prosimians (Fig. 10.4),However, many of the cortical visual areas but much more in anthropoid primatesthat characterize anthropoid primates are pre- (Fig. 10.5).sent in prosimian galagos (Fig. 10.4), indicating The third type of module in V2 projects tothat these visual areas emerged early in primate MT, as some classes of neurons in V1 do directlyor preprimate evolution. Tree shrews, which (Casagrande & Kaas, 1994). MT contains orderlyare the closest living relative of primates that arrangements of clusters of cells sensitive to sti-have been studied, also devote much of their mulus orientation and direction of movementcortex to visual processing (see Remple et al., (see Xu et al., 2004). MT is one of a number of2006), suggesting that some of the specializa- visual areas that appear to be unique to primatestions of visual cortex in primates predate pri- and collectively have the major role of furthermates and are shared by tree shrews. All analyzing visual inputs to extract informationprimates have a large primary visual area, V1, about object and global motion, which is thenthat has an orderly internal organization so that sent to subdivisions of posterior parietal cortexcells are grouped by the particular orientation (see Figs. 10.4 and 10.5). Thus, area MT is inter-of visual stimuli, such as lines, that best activate connected with bordering areas MST, MTc, andthem. V1 is divided into a number of pinwheels FST. FST has dorsal and ventral divisions withof such clusters of cells, which together repre- the dorsal division, FSTd, having interconnec-sent a given region of visual space and all orien- tions with MT and the ventral division, FSTv,tations. Tree shrews and carnivores also have having connections with MTc (Kaas & Morel,such pinwheels of orientation selective cells, but 1993). These areas are also associated with band-rodents do not (Van Hooser et al., 2005), sug- like portions of V3, the caudal division of DLgesting that tree shrews and primates shared an (DLc), and the dorsomedial area, DM. Areasancestor with orientation pinwheels, while car- MT, MST, and FSTd all project to portions ofnivores evolved them independently. The main posterior parietal cortex. Together, these areas,outputs of V1 are to the second visual area, V2, including those in posterior parietal cortex, con-which is divided into a series of repeats of three stitute most of the dorsal stream of visual pro-types of band-like clusters of cells, two of which cessing that is concerned with visually guidingare devoted to different aspects of object vision motor behavior. It is in posterior parietal cortexand the third part of a processing stream for where transformations occur from analyzingvisual guidance of motor control (Casagrande sensory information to informing sensorimotor& Kaas, 1994; Roe, 2003). This specialization of programs that induce ethologically relevantV2 is apparently unique to primates. Most behaviors.

SENSORY AND MOTOR SYSTEMS IN PRIMATES 187 Sensorimotor 3b 1 3a S1 DM V3 SMA PMD PP FE M1 Visual V2 V1 F T PMV MS DL MT A1 S2 PV T R AB FS MTc V3 Galago 0 5 IT mm VisualFigure 10.4 A dorsolateral view of the brain of a prosimian galago with major sensory and motor areasindicated. Visual areas include primary (V1), secondary (V2), and third (V3) visual areas, as well as thedorsolateral (DL), middle temporal (MT), fundal superior temporal (FST), MT crescent (MTc), medialsuperior temporal (MST), and dorsomedial (DM) visual areas. Much of posterior parietal (PP) and inferiortemporal (IT) cortex contains additional visual areas. Auditory cortex includes primary auditory cortex(A1), the rostral auditory area (R), areas of the auditory belt (AB), and areas of the parabelt (not shown).Somatosensory cortex includes primary somatosensory cortex (S1 or area 3b), adjoining somatosensoryareas 1 and 3a, the second area (S2), and the parietal ventral area (PV). Motor areas include the primary area(M1), dorsal (PMD) and ventral (PMV) premotor areas, a frontal eye field (FEF), supplementary motorcortex (SMA), and rostral and caudal cingulate motor areas (CMAr and CMAc) that are not shown. Dots inV1 and bars in V2 signify types of modular organization (see text).Posterior Parietal Cortex and the primates, but recent studies in prosimianSensorimotor Transformation galagos have revealed a lot about the organization of posterior parietal cortex (Fig. 10.6). MostMost mammals have very little cortex that can be notably, posterior parietal cortex is divided intocalled posterior parietal cortex (e.g., Fig. 10.1),but this region of the brain has greatly expanded rostral (PPr) and caudal (PPc) zones, with thein all primates (Figs. 10.4 and 10.5). The expan- rostral zone dominated by somatosensory inputssion may have preceded the emergence of pri- from higher-order somatosensory areas (S2, PV,mates, as tree shrews, close relatives of primates, area 1) and the caudal zone dominated by visualhave a strip of posterior parietal cortex that is inputs from higher-order visual areas (V2, V3,somewhat enlarged, and has basic features of DM, MT, MTc, MST, DLr). The caudal zone,primate posterior parietal cortex (Remple et al., PPc, projects to the rostral zone, PPr, so that2007). Thus, posterior parietal cortex in tree both visual and somatosensory inputs reachshrews receives both visual inputs from higher- PPr. Rostral PP is organized into a mediolateralorder visual areas and somatosensory inputs series of functionally distinct sensorimotorfrom higher-order somatosensory areas, while zones that were revealed by electrical stimulationprojecting to motor cortex. with microelectrodes. As described by Graziano Prosimian primates have proportionately less and coworkers for motor cortex and even part ofposterior parietal cortex than anthropoid posterior parietal cortex in macaque monkeys

188 PRIMATE NEUROETHOLOGY CC PS SMAm E-SMA SMAd M D PM PP DM 3a 3b -1 r 1 2 FV FEF M c -1 VPP M PMV S-1 V3 V2 V1 PR PV S-2 - - - - - DLr DLc VS CM C A-I MST MT Owl Monkey RM RT R PL M RT AL FSTd MT DLr DLc LRT Aud. c + V1 P.B. FSTv V2 + V3 + M1 + S1 V2 V1 ITc A1 MT ITr ITp ITm ERFigure 10.5 A lateral view of the brain of an owl monkey showing subdivisions of sensory and motorcortex (below) and a view of the cortex after it has been removed and flattened. See Figure 10.4 and text forabbreviations. Plus and minus notations in visual areas indicate representations of upper (+) and lower (–)visual hemifields. Rostral (DLr) and caudal (DLc) divisions of DL (V4) are shown, as well as caudal (ITc),rostral (ITr), medial (ITm), and polar (ITp) divisions of inferior temporal cortex. Dashed lines indicateopened fissures. M, medial visual area; VPP, ventral posterior parietal area; RI, retroinsular cortex; CC,corpus callosum.(see Graziano, 2006; Graziano, this volume), portion of posterior parietal cortex, eye move-half-second trains of electrical pulses, which are ments, defensive face movements, and aggressivelonger than those typically used for cortical face movements were evoked from separate cor-stimulation, can evoke complex movements. tical territories (Stepniewska et al., 2005). TheIn galagos, such half-second trains of brief elec- existence of these different subregions for dif-trical pulses evoked hindlimb and forelimb ferent complex, ethologically relevant move-movements when medial sites in PPr were sti- ments suggests that a number of corticalmulated (hindlimb and forelimb in Fig. 10.6C). networks exist for specific types of functionallyThese movements, evoked in anesthetized important movements. As similar movementsgalagos, resembled climbing movements. can be evoked from motor and premotorMore laterally in PPr, defensive forelimb move- cortex, it appears that posterior parietal modulesments, reaching movements, and hand-to- contain circuits that use visual and somatosen-mouth movements could be evoked from sepa- sory information to activate and modulate out-rate regions of cortex. Finally, in the most lateral puts. These modules dictate specific categories of

SENSORY AND MOTOR SYSTEMS IN PRIMATES 189 useful movements, which are then mediated via patterns, and cortical architecture, that do not projections to motor and premotor cortex, closely reflect their organization described here where the movement patterns are likely adjusted for posterior parietal cortex of galagos and New and refined. Aspects of the final movement World monkeys. While there is not complete patterns would also depend on motor nuclei agreement on areas and names for areas, circuits in the brainstem and spinal cord. In macaque areas include ventral (VIP), medial support of this proposal, our ongoing experi- (MIP), lateral (LIP), and anterior (AIP) areas ments in galagos and New World monkeys indi- of the intraparietal sulcus, as well as a number cate that any block of neural activity in primary of other areas (see Lewis & Van Essen, 2000). motor cortex, M1, abolishes the motor behavior Yet, when half-second trains of electrical pulses evoked from PPr. were applied to VIP, defensive movements were While posterior parietal cortex is unlikely to evoked (Cooke et al., 2003), and eye move- be organized in the same way in all primates, ments have been elicited by electrically stimu- our experiments have produced results very lating LIP (Kurylo & Skavenski, 1991; Thier & similar to those from galagos in New World Andersen, 1998). Other parts of posterior par- owl and squirrel monkeys. The arrangements ietal cortex in macaques appear to be involved of areas and functional subregions may be in reaching and grasping (Calton et al., 2002; somewhat different in posterior parietal cortex Snyder et al., 2000). Thus, the organization of of Old World macaque monkeys, where current posterior parietal cortex in Old World macaque proposals include a number of areas, defined monkeys may not be so different than in New by responses to sensory inputs, connection World monkeys and prosimians. SomeA C Hindlimb 3 a + forelimb Forelimb PM IPS (reach.) M1 S1 FSa Forelimb FSp V2 V1 hand to mouth IPS LS MT S Somatosensory ST Forelimb Forelimb cortex + face Face defense . Face ve moB (aggr.) Eye M1 S1 FSp PM FEF V2 V1 LS MT 1 mmFigure 10.6 The organization of posterior parietal cortex in prosimian galagos. See Figure 10.4 and textfor abbreviations. (A) A dorsolateral view of the brain with visual, somatosensory, motor, and posteriorparietal areas outlined. (B) Arrows indicate visual areas projecting to the caudal half of posterior parietalcortex and somatosensory and motor areas projecting to the rostral half. (C) An enlarged view of posteriorparietal cortex showing regions where electrical stimulation evoked different types of ethologically relevantcomplex movements. From Stepniewska, I., Fang, P. C., & Kaas, J. H. (2005). Microstimulation revealsspecialized subregions for different complex movements in posterior parietal cortex of prosimian galagos.Proceedings of the National Academy of Sciences, 102, 4878–4883. Used with permission.

190 PRIMATE NEUROETHOLOGYinvestigators, using functional magnetic reso- primary area, S1, which is clearly homologousnance imaging, have results that suggest simila- with area 3b of anthropoid primates, and narrowrities between macaques and the more extensive strips of somatosensory cortex bordering S1 (3b)posterior parietal cortex of humans (e.g., rostrally and caudally (Fig. 10.4). Area 3b getsSwisher et al., 2007). inputs from VP in prosimians and all other primates. The more rostral somatosensory strip (SR) gets input from VPS, and is involved inThe Somatosensory System of Primates proprioception in all primates and in at leastWhile vision was obviously very important to some other mammals. This strip is clearly areaearly primates, most early primates were small 3a of anthropoid primates. The identity of theand adapted to a nocturnal lifestyle that included caudal somatosensory strip (SC) is less clear. It isfeeding in the fine branches of bushes and trees in the position of area 1 of anthropoid primates,on insects, small vertebrates, fruits, and leaves but unlike area 1, SC does not respond well to(Ross & Martin, 2007). This lifestyle required light touch on the skin. While area 1 gets denseunusual sensorimotor abilities as these primates projections from VP, the projections are sparseneeded to hold on to moving branches while in SC. As a further difference, there is no evi-reaching for food. According to Whishaw dence for an area 2 just caudal to SC, as an area(2003), visual guidance of hand movements is 2 with inputs from VPS is caudal to area 1 inone of the most distinguishing features of pri- anthropoid primates. Possibly an area like SCmates. One of the reasons for reaching for food, differentiated into area 1 of anthropoid pri-rather than grasping it with their mouth, was to mates, or perhaps SC differentiated into bothprotect the large, forward-facing eyes. As an area 1 and area 2. Here we tentatively identifyadaptation for greater hand use, primates have SC as area 1. Areas 3a, 3b, 1, and 2 of anthropoidlarge concentrations of low-threshold mechan- primates (Fig. 10.5), including humans, containoreceptors in the glabrous skin of the hand, parallel representations of the contralateral halfespecially of the Meissner corpuscles, subserving of the body, from hindlimb to tongue in a med-the rapidly adapting type 1 afferents with small iolateral sequence. Together these fields inter-receptive fields and sensitivity to stimulus connect with somatosensory areas of lateralchange (see for review Kaas, 2004). An enlarged (insular) parietal cortex, posterior parietalrepresentation of the glabrous skin of the hand is cortex, and motor cortex.found in somatosensory nuclei and cortical areas Lateral somatosensory of the upper bank ofof primates, especially Old World monkeys, the lateral fissure and the insula contain addi-apes, and humans. tional somatosensory areas, including S2 and In the thalamus, the ventroposterior PV of other mammals, the ventral somatosen-complex of primates is well differentiated into sory area (VS), the parietal rostral area (PR),a ventroposterior inferior nucleus (VPI) with and likely others. The organization across pri-spinothalamic inputs, a ventroposterior nuc- mate taxa is not well understood, but differ-leus (VP) with inputs from cutaneous mechan- ences are likely given the large extent oforeceptors, and a ventroposterior superior insular cortex in some anthropoid primates.nucleus (VPS) with inputs from muscle spindle Pathways through lateral parietal cortex arereceptors (Kaas, 2007). An anterior pulvinar thought to be important in the recognition of(PA) can be identified, and it has connections objects by touch (Murray & Mishkin, 1984) andwith areas of somatosensory cortex. The non- form the functional equivalent of the ventralprimary homolog of PA is not obvious, but PA visual stream of processing. Posterior parietalpossibly corresponds to the posterior nucleus of cortex, as discussed previously, also forms anrodents. important part of the somatosensory system, Anterior parietal cortex organization varies constituting much of the dorsal steam of soma-across primates (Qi et al., 2008). In prosimian tosensory processing for guiding reaching andprimates, three areas can be distinguished: a other actions.

SENSORY AND MOTOR SYSTEMS IN PRIMATES 191Gustatory Cortex and subcortical connections, such as those to the inferior colliculus. With this note of caution,The organization of the taste or gustatory system present understandings of the organization ofis not well understood in any mammals. Thus, auditory cortex in primates are outlined.modifications in primates are not clear. The All anthropoid primates appear to have astandard view for primates is that the thalamic strip of auditory cortex that has the characteris-taste nucleus, VPMpc, projects to both the tics of a primary sensory field (Kaas & Hackett,tongue representation of ‘‘S1’’ and to a large 2008). This strip of cortex, generally called theprimary gustatory region in the cortex of the auditory core, consists of three auditory areas,lateral sulcus (Fig. 10.7A). The gustatory area, distinguished by their differing patterns of tono-G, in turn projects (apparently not directly) to topic organization (Fig. 10.5). The core has theorbitofrontal cortex, where hedonic or pleasur- well-differentiated layer 4 and other architec-able aspects of taste are processed with other tonic characteristics of primary sensory areas,types of relevant information (see for review as well as activating inputs from a thalamicKaas et al., 2006). More current evidence indi- relay nucleus, the ventral nucleus of the medialcates that VPMpc projects to the tongue repre- geniculate complex, MGv. Neurons in the coresentation of area 3b, and possibly area 1, and that respond well to pure tones, and neurons in dif-corticocortical connections implicate tongue ferent locations across the three primary areasrepresentations in areas 3a and 1 in processing respond best, or at the lowest sound intensity, totaste (Iyengar et al., 2007). While an area G may tones of different frequencies. The auditory coreexist, another possibility is that the tongue por- in monkeys is in cortex of the lower bank of thetions of several areas of the cortex of the lateral lateral sulcus, where it forms an elongated cau-sulcus are involved in taste (Fig. 10.7B). dorostral strip. The so-called primary area, A1, isPain and Temperature the caudal-most area. The most caudal neurons in A1 respond to tones of the highest frequency,According to Craig (2003, 2007), primates differ and neurons at progressively more rostral loca-from other mammals in having two specific tions in A1 respond best to tones of progressivelyregions of the thalamus that have differentiated lower frequencies. Neurons in bands runningas sites for the termination of nociceptive infor- perpendicular to this caudorostral frequencymation from the spinal cord and brainstem, the gradient respond best to tones of roughly theposterior part of the ventral medial nucleus same frequency. These bands or rows of neurons(VMpo) and the ventral caudal part of the constitute the lines of isorepresentation for tonesmedial dorsal nucleus (MDvc). VMpo provides in A1. The pattern of tonotopic representationprojections in turn to a representation of painful in A1 reverses for the rostral auditory area,stimuli in the dorsal portion of insular cortex, R, and again for the rostrotemporal auditorywhere other representations of body sensations, area, RT. The core has been histologically iden-including temperature, may exist. Another pro- tified in a number of primates, including maca-jection is to part of area 3a for uncertain func- ques, chimpanzees, and humans (Hackett et al.,tions. MDvc projects to anterior cingulate cortex 2001). In prosimian galagos, only A1 and R ofto motivate behavioral responses. the core have been identified (Fig. 10.4). The three core areas seem very much alike, but aAuditory Cortex in Primates presumption is that they have at least somewhatPossibly due to limited study, the subcortical different functional roles. One hypothesis is thatauditory system of primates is thought to be they contribute differently to dorsal and ventralhighly similar to those in other mammalian streams of auditory processing, with the dorsaltaxa. This cannot be quite true, as the greatly stream more concerned with locating sounds inexpanded cortical auditory system of primates space and the ventral stream involved in dedu-would be reflected by changes in the thalamus, as cing the meanings of the sounds (seecortical areas have thalamic interconnections Rauschecker & Tian, 2000).

S1 A S1 tongue H G OBF Taste cortex Thalamus VPMpc T5 Somatosensory T4 Sole Foot Foot SMA Foot T3 T2 T1 Posterior B Parietal PMD Trunk Trunk Cortex Trunk M1 3a Arm 1 2 FEF 3b Hand D5 Hand Hand D4 Hand V2 V1 Visual D3 Palm D2 CM D1 Face PMV Face Face th Face ee c.T Face A1 c.Teeth ngu e S2 c. To h R l c. et lt ra Te Be Tongue VSc us o i ips PV d lc mp ue Aud Au Teeth ng RT Parabelt To Su Te Teeth ips i PR VSr r ue rio ng ue To ng pe To Su lateral sulcus with insula opened Owl MonkeyFigure 10.7 Proposed locations of cortical taste areas shown on a lateral view of the brain of a squirrelmonkey. (A) Early investigators did not consider the architectonic fields 3a, 3b, 1, and 2 of anterior parietalcortex to be separate fields, and grouped them together in S1, where the lateral part represents the tongue andgets input from the parvocellular ventral posterior medial nucleus (VPMpc). However, the main projections ofVPMpc were thought to be to the gustatory area, G, at the rostral end of the central sulcus. Area G was thoughtto project to a hedonistic (H) taste region in orbitofrontal cortex (OBF). (B) A contemporary view ofsomatosensory cortex organization in monkeys. The region of G appears to contain parts of severalsomatosensory areas (also see Fig. 10.5), suggesting that tongue representations in several fields maycontribute to the processing of taste information. Note that the tongue is represented twice in area 3b. c,contralateral. Modified from Kaas, J. H., Qi, H-X., & Iyengar, S. (2006). Cortical network for representing theteeth and tongue in primates. Anatomical Records Part A, 288A, 182–190. Used with permission. 192

SENSORY AND MOTOR SYSTEMS IN PRIMATES 193 The auditory core is surrounded by a ‘‘belt’’ The parabelt projections are to regions ofof adjacent auditory areas that at least partly cortex that we define as the fourth level of cor-depend on inputs from the core for activation. tical auditory processing. Areas of the fourthThe cortical connections of the core, at least in level are diverse and distributed across the tem-monkeys, are almost completely with the belt. poral, parietal, and frontal lobes. One of theseThe number of areas in the belt is somewhat regions is cortex of the upper bank of theuncertain; the proposed number is eight, with superior temporal cortex in monkeys. Thistwo areas on the caudal end of A1 and pairs of appears to be a region where neurons respondinner and outer areas on each side of A1, R, and to auditory and visual stimuli (see Cusick, 1997).RT. Presently, there is evidence that at least three Neurons in this and other bisensory fields mayof the auditory belt areas are tonotopically orga- function to localize a sound to a visual objectnized, although this is difficult to determine as (the so-called ventriloquist effect) (seeneurons in the belt areas generally respond Romanski & Ghazanfar, this volume). Othermuch better to complex sounds than tones projections are to rostroventral parts of the(Rauschecker et al., 1995). One of the belt superior temporal cortex. This cortex relaysareas, the caudomedial area (CM), has many auditory information to orbitofrontal cortexneurons that respond to light touch (Fu et al., that is involved in evaluating the rewarding2003), while another area, the middle lateral belt value of stimuli (Rolls, 2004). Projections of(ML), is influenced by vision (Ghazanfar et al., the parabelt to cortex of the temporal-parietal2005), demonstrating a surprising substrate junction and adjoining parietal cortex are tofor bisensory integration at a very early level of multisensory regions (visual somatosensory,auditory processing (see Romanski & Ghazanfar, auditory) where neurons project to the frontalthis volume). Auditory belt areas connect eye fields (Huerta et al., 1987), perhaps to helpbroadly to core areas, other belt areas, the adja- direct the eyes toward sounds of interest. Parietalcent parabelt region, and even more distant cor- lobe multisensory areas also project to premotortical regions, such as prefrontal cortex where areas of the frontal lobe, where they help guideneurons responsive to visual and auditory sti- motor behavior. Finally, parabelt projectionsmuli are found (Romanski & Goldman-Rakic, to prefrontal granular cortex (Romanski &2002). Thalamic inputs are from dorsal (MGd) Goldman-Rakic, 2002) may be involved inand medial (MGm) divisions of the medial gen- working memory for auditory signals. Auditoryiculate complex, and other thalamic nuclei, sug- and visual parts of prefrontal cortex project togesting multisensory or broader auditory premotor areas of frontal cortex, thus providingfunctions. The more rostral belt areas appear to another source of sensory guidance of motorbe more involved in a ventral stream for sound behavior.identification, with the more caudal belt areas Humans, of course, differ from other pri-more concerned with sound localization. mates in that the left cerebral hemisphere is The auditory parabelt constitutes a third level usually specialized for language. However, anof auditory cortical processing in primates. The asymmetry between the extent and shape of theparabelt in monkeys occupies the part of the lateral sulcus between the left and right hemi-superior temporal gyrus that adjoins the lateral spheres has been described, not only in humans,belt (Fig. 10.5). The parabelt region gets dense but also, from skull endocasts, in the brains ofinputs from the auditory belt areas, but practically our extinct ancestors, suggesting that left hemi-no input from the core. Connections from belt sphere auditory specialization preceded theareas are most dense with nearer portions of the emergence of language (Galaburda et al., 1978).parabelt, suggesting that the parabelt has func- In humans, a region of cortex near the auditorytional divisions. Hackett and colleagues (1998) core is enlarged in the left hemisphere, and thisdivided the parabelt into rostral (RPB) and region, as part of Wernicke’s area, appears tocaudal (CPB) regions with most dense connec- be important in language. In the frontal lobe,tions with rostral or caudal belt areas, respectively. a premotor region is enlarged in the left

194 PRIMATE NEUROETHOLOGYhemisphere, constituting the main part of Primary motor cortex (M1) contains aBroca’s speech area (Foundas et al., 1996). mosaic of small regions, each devoted to a spe-These may be only some of the ways in which cific movement. These small regions are dis-the human auditory system differs from that of bursed within a larger, gross somatotopicmonkeys. For further discussion on the issue of pattern that progresses from hindlimb, trunk,cerebral asymmetries, see Hopkins in this forelimb, and face to tongue in a mediolateralvolume. sequence across frontal cortex (Donoghue et al., 1992; Gould et al., 1986; Huang et al., 1988;Vestibular Cortex Preuss et al., 1996). The gross pattern of somatotopic organization is similar across individualsMultisensory integration in the vestibular of the same species, but the mosaic pattern variessystem occurs as early as in the vestibular in detail. M1 receives inputs from other frontalnuclei of the brainstem, and cortical areas with motor areas, including PMD, PMV, SMA, andneurons sensitive to vestibular system activation cingulate areas; some of the somatosensoryare multisensory areas (Guldin & Grusser, ¨ areas including areas 3a, 1, 2, S2, and PV; and1998). Probably the main cortical vestibular posterior parietal cortex (see Fang et al., 2005;region in monkeys, the closest to a primary Stepniewska et al., 1993). Other importantvestibular area, occupies the medial part of ret- inputs are from the posterior ventrolateralroinsular cortex (Fig. 10.5). This region where nucleus (VLp) of the thalamus, which receiveshalf of the neurons respond to vestibular stimuli projections from the cerebellar nucleiis called the parietal insular vestibular cortex (e.g., Stepniewska et al., 2003). M1 provides the(PIVC). A more posterior region of the cortex majority of projections to brainstem and spinalof the lateral fissure, the visual posterior sylvian cord motor circuits, and the projection to thearea (VPS), is involved in visuomotor reflexes, upper spinal cord circuits that control the digitsand has neurons that are vestibularly activated is enlarged in Old World monkeys and in theor modulated. A third vestibular zone, the part highly dexterous New World Cebus monkeysof somatosensory area 3a where neck muscles are compared to most New World monkeys andrepresented, also has neurons activated by ves- prosimian primates (Nudo & Frost, 2007). Duetibular stimulation. Parts of area 7 and area to this more direct pathway, movements can be2 have also been implicated in this extended, evoked at lower current levels from M1 thanmultisensory vestibular cortical system, and from other frontal motor areas, although stimu-these vestibular cortical areas have a broader lation thresholds from PMV can be nearly asinfluence via connections with other cortical low. Brief trains of electrical pulses in M1 havefields. long been known to evoke simple movements, such as the extension of a digit. More recently, Graziano (this volume; Graziano et al., 2002)Cortical Motor Areas in Primates and coworkers have shown that larger, half-Motor cortex has expanded and increased in second trains of electrical pulses can evokenumber of areas in primates (Kaas, 2007b). more complex, behaviorally relevant move-Even prosimian galagos have an enlarged pri- ments, such as hand-to-mouth movements,mary area (M1), a dorsal (PMD) and ventral from M1 of macaque monkeys. In our unpub-(PMV) premotor area, a supplementary motor lished studies led by Iwona Stepniewska, com-area (SMA), a presupplementary motor area plex movements were evoked from M1 of(pre-SMA), a frontal eye field (FEF), and at galagos and squirrel monkeys. Complex move-least rostral (CMAr) and caudal (CMAc) cingu- ments have also been evoked from premotorlate motor areas (Figs. 10.4 and 10.6). These cortex, as well as from posterior parietal cortexsame areas are found in anthropoid primates (see previous section), but inactivation of M1(Fig. 10.5), where there is evidence for an abolishes or greatly modifies these movementincrease in the number of premotor fields. patterns, suggesting that M1 is the final cortical

SENSORY AND MOTOR SYSTEMS IN PRIMATES 195target in the circuits for such behaviors. in nonmotor tasks, and having a lack of projec-However, the circuits addressed by the subcor- tions to the spinal cord. Connections with SMAtical projections of M1 must be important in are not dense and direct projections to M1 areorganizing aspects of the movement patterns. sparse or absent (Luppino et al., 1993; see Akkal et All primates have dorsal and ventral divisions al., 2007, for review).of the classical premotor cortex. Mainly forelimb The cingulate motor areas are in frontaland mouth and face movements are evoked cortex of the medial wall of the cerebral hemi-from PMV, while both forelimb and hindlimb sphere (Fig. 10.4). In macaques, three cingulatemovements are evoked from PMD. Thresholds motor areas have been proposed, on the dorsalfor evoking movements in PMD are higher than (CMAr and CMAd) and ventral (CMAv) banksthose for M1, while those for PMV can be higher of the cingulate sulcus (Picard & Strick, 1996, foror similar (e.g., Preuss et al., 1996). Although review). These areas differ somewhat in connec-both of these areas project to the spinal cord and tions, but collectively they include those withbrainstem, much of their influence on motor prefrontal cortex, M1, the spinal cord, and par-behavior may depend on their projections to ietal cortex. Microstimulation of CMAd andM1. Both PMV and PMD receive inputs from CMAv evoke movements in patterns that sug-frontal cortex, posterior parietal cortex, and gest that these fields have somatotopic organiza-higher-order somatosensory fields. PMD tions. CMAv is less responsive to stimulation.appears to be divided into two fields in New CMAr is thought to not control movementsand Old World monkeys, and two or three divi- directly, but signal errors, reinforcement, andsions of PMV have been proposed for Old World conflict, and thereby be involved in supervisorymonkeys (e.g., Matelli et al., 1985). Thus, the control (Schall et al., 2002).number of premotor areas appears to be greater Finally, all primates appear to have a frontalin macaque monkeys, and this is likely the case eye field (FEF) and perhaps a supplementary eyefor apes and humans, where even more pre- field (SEF). Microstimulation of these areas pro-motor fields may exist. Broca’s area in the left duces eye movements, and both fields project tocerebral hemisphere of humans may be an ela- the deeper layers of the superior colliculusboration of one or more of the PMV fields (see (Huerta & Kaas, 1990; Huerta et al., 1986). ThePreuss et al., 1996). FEF has cortical connections with SEF and the The supplementary motor area, SMA, has SMA, prefrontal cortex, visual areas of the tem-been described in prosimian galagos (Fig. 10.4), poral lobe, and posterior parietal visuomotorNew World monkeys (Fig. 10.5), Old World areas (Huerta et al., 1987). Neurons in FEFmonkeys, and humans (see Tanji, 1994; Wu respond to visual stimuli or control eye move-et al., 2000, for reviews). SMA is located just ments in a decision-making process (Schall et al.,dorsal to the hindlimb representation in M1, 2002). The connections of the SEF are moreand it represents the hindlimb, forelimb, and extensive with cortical areas and subcorticalface in a caudorostral sequence (e.g., Gould et structures related to prefrontal and skeleto-al., 1986). SMA has been implicated in generating motor functions (Huerta & Kaas, 1990).movement sequences and in bimanual coordina-tion of movements (Tanji, 1994). SMA has dense CONCLUSIONSprojections to M1, while having some projectionsto the spinal cord. Inputs include those from This review outlines major features of the orga-posterior parietal cortex, pre-SMA, and cingulate nizations of sensory and motor systems inmotor cortex (Luppino et al., 1993). In addition, primates in comparison with the likely organiza-SMA receives inputs from both the basal ganglia tions of these systems in the nonprimate ances-and the cerebellum relayed through the motor tors of primates. The major premise of thethalamus (Akkal et al., 2007). The more rostrally review is that sensory and motor systems havelocated pre-SMA differs from SMA, being densely changed in many ways with or before the emer-connected with prefrontal cortex, being involved gence of early primates, and more variation

196 PRIMATE NEUROETHOLOGYsubsequently occurred in the different lines of parallel pathways to emerge, thereby addingprimate evolution. In general, prosimian galagos functions and abilities.seem to have changed the least, but the sensory In brief, primate brains differ from otherand motor systems of all primates are distinctly brains by maintaining a similar level of neuronaldifferent from those of other mammals. density as brains increase in size. Primate brains Alterations of sensory systems have been also differ from nonprimate brains in the waysrevealed most extensively in neocortex, which their sensory and motor systems are organized.of course has expanded greatly in all primates, Finally, the organizations of these systems varybut especially humans. Alterations have been of within and across the major branches of primatetwo main types: those within areas common to evolution, probably much more than in anyall or most mammals, and the addition of areas other mammalian order.that seem to be unique to primates, and even tospecific lines of primate evolution. This reviewgives a global overview of the evolution of sen- REFERENCESsory and motor areas in primates, but much has Akkal, D., Dum, R. P., & Strick, P. L. (2007).been left out, most notably the implications of Supplementary motor area andthe huge increases in brain and especially neo- presupplementary motor area: Targets of basalcortex size in some primates, especially apes and ganglia and cerebellar output. Journal ofhumans. A larger brain could mean larger sen- Neuroscience, 27, 10659–10673.sory and motor areas, and at least the more easily Basbaum, A. I., Kaneko, A., Shepherd, G. M., & Westheimer, G. (Eds.). (2008). The Senses:identified sensory areas, such as V1, S1, and the A Comprehensive Reference. London: Elsevier.auditory core are larger in larger brains. Yet, Beck, P. D., Pospichal, M. W., & Kaas, J. H. (1996).cortex has expanded more in the larger ape and Topography, architecture, and connections ofhuman brains than the primary sensory areas, somatosensory cortex in opossums: Evidenceindicating that the larger brains also likely have for five somatosensory areas. Journal ofmore cortical areas. In addition, the functions of Comparative Neurology, 366, 109–133.cortical areas relate to their sizes (Kaas, 2000), in Berson, D. M. (2008). Retinal ganglion cell typespart because cortical neurons do not vary much and their central projections. In: R. H. Maslan &in size so that large areas have more neurons but T. Albright (Eds.), The Senses: A Comprehensiveless widespread intrinsic connections. Thus, Reference (vol 1, pp. 491–519). London:large sensory areas are not well suited for global Elsevier. Bhatnagar, K. P., & Smith, T. D. (2007). Theintegration of sensory inputs from across the vomeronasal organ and its evolutionary loss inreceptor sheet. As primates with larger brains catarrhine primates. In: J. H. Kaas & T. M.and larger cortical areas evolved, the internal Preuss (Eds.), Evolution of Nervous Systemsorganizations and functions of cortical areas (vol 4, pp. 142–152). London: Elsevier.changed as areas increased in size so that they Bombardieri, R. A., Johnson, J. I., & Campos, G. B.were less involved in global integration. Other, (1975). Species differences in mechanosensorysmaller sensory areas evolved to take over roles projections from the mouth to the ventrobasalin global integration. Thus, the other major thalamus. Journal of Comparative Neurology,change that seems to have occurred in the evolu- 163, 41–64.tion of primate taxa with larger brains is an Calton, J. L., Dickinson, A. R., & Snyder, L. H.increase in the number of areas, including the (2002). Non-spatial, motor-specific activation in posterior parietal cortex. Naturenumber in sensory and motor systems. This Neuroscience, 5, 580–588.potentially increases the steps in serial proces- Carr, C. E., & Edds-Walton, P. L. (2008).sing, and it is the repetition of the local proces- Vertebrate auditory pathways. In: P. Dallos &sing within columns of cells from area to area D. Oertel (Eds.), The Senses: A Comprehensivethat allows complex outcomes from computa- Reference (vol 3). London: Elsevier.tionally simple steps. Adding cortical areas also Casagrande, V. A., & Kaas, J. H. (1994). Theincreases the potential for functionally distinct afferent, intrinsic, and efferent connections of

SENSORY AND MOTOR SYSTEMS IN PRIMATES 197 primary visual cortex in primates. In: A. Peters Fang, P-C., Stepniewska, I., & Kaas, J. H. (2005). & K. Rockland K (Eds.), Cerebral Cortex (vol 10, Ipsilateral cortical connections of motor, pp. 201–259). New York: Plenum Press. premotor, frontal eye, and posterior parietalCasagrande, V. A., & Xu, X. (2004). Parallel visual fields in a prosimian primate, Otolemur pathways: A comparative perspective. In: L. M. garnetti. Journal of Comparative Neurology, Chalupa & J. S. Werner (Eds.). The Visual 490, 305–333. Neurosciences (pp. 494–506). Cambridge, MA: Foundas, A. L., Leonard, C. M., Gilmore, R. L., MIT Press. Fennell, E. B., & Heilman, K. M. (1996). ParsCollins, C. E., Hendrickson, A., & Kaas, J. H. triangularis asymmetry and language (2005a). Overview of the visual system of dominance. Proceedings of the National tarsius. Anatomical Record, 287A, 1013–1025. Academy of Sciences, 93, 719–722.Collins, C. E., Lyon, D. C., & Kaas, J. H. (2005b). Fu, K. M., Johnston, T. A., Shah, A. S., Arnold, L. Distribution across cortical areas of neurons L., Smiley, J. F., Hackett, T. A., et al. (2003). projecting to the superior colliculus in New Auditory cortical neurons respond to World monkeys. Anatomical Record, 285A, somatosensory stimulation. Journal of 619–627. Neuroscience, 23, 7510–7515.Cooke, D. F., Taylor, C. R., Moore, T., & Graziano, Galaburda, A. M., LeMay, M., Kemper, T. L., & M. S. A. (2003). Complex movements evoked Geschwind, N. (1978). Right-left asymmetries by microstimulation of the ventral intraparietal in the brain. Science, 199, 852–856. area. Proceedings of the National Academy of Ghazanfar, A. A., Maier, J. X., Hoffman, K. L., & Sciences, 100, 6163–6168. Logothetis, N. K. (2005). MultisensoryCrabtree, J. W., & Killackey, H. P. (1989). The integration of dynamic faces and voices in topographic organization and axis of rhesus monkey auditory cortex. Journal of projection within the visual sector of the Neuroscience, 25, 5004–5012. rabbit’s thalamic reticular nucleus. European Gould, H. J., Cusick, C. G., Pons, T. P., & Kaas, J. H. Journal of Neuroscience, 1, 94–109. (1986). The relationship of corpus callosumCraig, A. D. (2002). How do you feel? connections to electrical stimulation maps of Interoception: The sense of the physiological motor, supplementary motor, and the frontal condition of the body. Nature Reviews eye fields in owl monkeys. Journal of Neuroscience, 3, 655–666. Comparative Neurology, 247, 297–325.Craig, A. D. (2003). A new view of pain as a Graf, W. M. (2007). Vestibular system. In: J. H. homeostatic emotion. Trends in Neuroscience, Kaas & L. A. Krubitzer (Eds.), Evolution of 26, 303–307. Nervous Systems (vol 3, pp. 341–371). London:Craig, A. D. (2007). Evolution of pain pathways. In: Elsevier. J. H. Kaas & L. A. Krubitzer (Eds.), Evolution of Graziano, M. S. A. (2006). The organization of Nervous Systems (vol 3, pp. 227–235). London: behavioral repertoire in motor cortex. Annual Elsevier. Review of Neuroscience, 29, 105–134.Cusick, C. G. (1997). The superior temporal Graziano, M. S. A., Taylor, C. S. R., & Moore, T. polysensory region in monkeys. In: K. S. (2002). Complex movements evoked by Rockland (Ed.), Cerebral cortex: Extrastriate microstimulation of precentral cortex. Neuron, cortex in primates (pp. 4356–4463). New York: 34, 841–851. Plenum Press. ¨ Guldin, W. O., & Grusser, O. J. (1998). Is there aDonoghue, J. P., Leibovic, S., & Sanes, J. N. (1992). vestibular cortex? Trends in Neuroscience, 21, Organization of the forelimb area in squirrel 254–259. monkey motor cortex: representation of digit, Hackett, T. A., Preuss, T. M., & Kaas, J. H. (2001). wrist, and elbow muscles. Experimental Brain Architectonic identification of the core region Research, 89, 1–19. in auditory cortex of macaques, chimpanzees,Dostrovsky, J. O., & Craig, A. D. (2008). The and humans. Journal of Comparative Neurology, thalamus and nociceptive processing. In: M. C. 441, 197–222. Bushnell & A. I. Bashaum (Eds.), The Senses: A Hackett, T. A., Stepniewska, I., & Kaas, J. H. (1998). Comprehensive Reference (vol 5, pp. 635–668). Subdivisions of auditory cortex and ipsilateral London: Elsevier. cortical connections of the parabelt auditory

198 PRIMATE NEUROETHOLOGY cortex in macaque monkeys. Journal of Preuss (Eds.), Evolution of Nervous Systems (vol Comparative Neurology, 394, 475–495. 4, pp. 35–57). London: Elsevier.Herculano-Houzel, S., Collins, C. E., Wong, P., & Kaas, J. H., & Collins, C. E. (2004). The Primate Kaas, J. H. (2007). Cellular scaling rules for Visual System. Boca Raton: CRC Press. primate brains. Proceedings of the National Kaas, J. H., Guillery, R. W., & Allman, J. M. (1972). Academy of Sciences, 104, 3562–3567. Some principles of organization in the dorsalHuang, C. S., Sirisko, M. A., Hiraba, H., & Murray, lateral geniculate nucleus. Brain Behavior and G. M. (1988). Organization of the primate face Evolution, 6, 253–299. motor cortex as served by intracortical Kaas, J. H., & Hackett, T. A. (2008). The functional microstimulation and electrophysiological neuroanatomy of the auditory cortex. In: P. identification of afferent inputs and Dallos & D. Oertel (Eds)., The Senses: corticobulbar or projections. Journal of A Comprehensive Reference (vol 3, pp. Neurophysiology, 59, 796–818. 765–780). London: Elsevier.Huerta, M. F., & Kaas, J. H. (1990). Supplementary Kaas, J. H., & Huerta, M. F. (1988). The subcortical eye field as defined by intracortical visual system of primates. In: H. P. Steklis (Ed.), microstimulation: Connections in macaques. Comparative Primate Biology (vol 4, pp. Journal of Comparative Neurology, 293, 299–330. 327–391). New York: Alan R. Liss.Huerta, M. F., Krubitzer, L. A., & Kaas, J. H. (1986). Kaas, J. H., Huerta, M. F., Weber, J. T., & Harting, Frontal eye field as defined by intracortical J. K. (1978). Patterns of retinal terminations microstimulation in squirrel monkeys, owl and laminar organization of the lateral monkeys, and macaque monkeys: I. geniculate nucleus of primates. Journal of Subcortical connections. Journal of Comparative Neurology, 182, 517–554. Comparative Neurology, 253, 415–439. Kaas, J. H., & Lyon, D. C. (2007). PulvinarHuerta, M. F., Krubitzer, L. A., & Kaas, J. H. (1987). contributions to the dorsal and ventral streams Frontal eye field as defined by intracortical of visual processing in primates. Brain Research microstimulation in squirrel monkeys, owl Reviews, 55, 285–296. monkeys, and macaque monkeys. II. Cortical Kaas, J. H., & Morel, A. (1993). Connections of connections. Journal of Comparative visual areas of the upper temporal lobe of owl Neurology, 265, 332–361. monkeys: the MT crescent and dorsal andIyengar, S., Qi, H-X., Jain, N., & Kaas, J. H. (2007). ventral subdivisions of FST. Journal of Cortical and thalamic connections of the Neuroscience, 13, 534–546. representations of the teeth and tongue in Kaas, J. H., & Preuss, T. M. (2008). Human brain somatosensory cortex of New World monkeys. evolution. In: L. R. Squire (Ed.), Fundamental Journal of Comparative Neurology, 501, 95–120. Neuroscience (3rd ed., pp. 1027–1035). SanKaas, J. H. (2000). Why is brain size so important: Diego: Elsevier. Design problems and solutions as neocortex Kaas, J.H. The somatosensory thalamus and gets bigger or smaller. Brain and Mind, 1, 7–23. associated pathways. In: E. Gardner & J. H.Kaas, J. H. (2003). The evolution of the visual Kaas (Eds.), The Senses: A Comprehensive system in primates. In: L. M. Chalupa & J. S. Reference, (vol 6, 117–141) London: Elsevier. Werner (Eds.), The Visual Neurosciences Kaas, J. H., Qi, H-X., & Iyengar, S. (2006). Cortical (pp. 1563–1572). Cambridge, MA: MIT Press. network for representing the teeth and tongueKaas, J. H. (2004). Somatosensory system. In: G. in primates. Anatomical Record, 288A, 182–190. Paxinos & J. K. Mai (Eds.), The Human Nervous Kosar, E., Grill, H. J., & Norgren, R. (1986). System (2nd ed., pp. 1059–1092). London: Gustatory cortex in the rat. I Physiological Elsevier. properties and cytoarchitecture. BrainKaas, J. H. (2007a). Reconstructing the Research, 379, 329–341. organization of the forebrain of the first Kremers, J. (2005). The Structure, Function and mammals. In: J. H. Kaas & L. A. Krubitzer Evolution of the Primate Visual System. (Eds.), Evolution of Nervous Systems (vol 3, pp. Chichester, UK: John Wiley and Sons. 27–48). London: Elsevier. ¨ Krubitzer, L.A., Kunzle, H., and Kaas, J. (1997).Kaas, J. H. (2007b). The evolution of sensory and The organization of sensory cortex in a motor systems in primates. In: J. H. Kaas & T. M. Madagascan insectivore, the tenrec (Echinaps

SENSORY AND MOTOR SYSTEMS IN PRIMATES 199 telfairi). Journal of Comparative Neurology, 379, Preuss, T. M., Stepniewska, I., & Kaas, J. H. (1996). 399–414. Movement representation in the dorsal andKurylo, D. D., & Skavenski, A. A. (1991). Eye ventral premotor areas of owl monkeys: movements elicited by electrical stimulation of A microstimulation study. Journal of area PG in the monkey. Journal of Comparative Neurology, 371, 649–676. Neurophysiology, 65, 1243–1253. Pritchard, T. C., & Norgren, R. (2004). GustatoryLewis, J. W., & Van Essen, D. C. (2000). Mapping system. In: G. Paxinos & J. K. Mai (Eds.), The of architectonic subdivisions in the macaque Human Nervous System (pp. 1171–1198). monkey, with emphasis on parieto-occipital Amsterdam: Elsevier Press. cortex. Journal of Comparative Neurology, 428, Qi, H-X., Preuss, T. M., & Kaas, J. H. (2008). 79–111. Somatosensory areas of the cerebral cortex:Lima, D. (2008). Ascending pathways: Anatomy Architectonic characteristics and modular and physiology. In: A. I. Basbaum & M. C. organization. In:E. Gardner & J. H. Kaas Bushnell (Eds.), The Senses: A Comprehensive (Eds.) The Senses: A Comprehensive Reference Reference (vol 5, pp. 477–526). London: (vol 6, pp. 143–169). London: Elsevier. Elsevier. Rauschecker, J. P., & Tian, B. (2000). MechanismsLuppino, G., Matelli, M., Camarda, R., & and streams for processing of "what" and Rizzolatti, G. (1993). Corticocortical "where" in auditory cortex. Proceedings of the connections of area F3 (SMA-proper) and area National Academy of Sciences, 97, 11800–11806. F6 (pre-SMA) in the macaque monkey. Journal Rauschecker, J. P., Tian, B., & Hauser, M. (1995). of Comparative Neurology, 338, 114–140. Processing of complex sounds in the macaqueLyon, D. C. (2007). The evolution of visual cortex nonprimary auditory cortex. Science, 268, and visual systems. In: J. H. Kaas & L. A. 111–114. Krubitzer (Eds.), Evolution of Nervous Systems Remple, M. S., Henry, E. C., & Catania, K. C. (vol 3, pp. 267–306). London: Elsevier. (2003). Organization of somatosensory cortexMatelli, M., Luppino, G., & Rizzolatti, G. (1985). in the laboratory rat (Rattus norvegicus): Patterns of cytochrome oxidase activity in the Evidence for two lateral areas joined at the frontal agranular cortex of macaque monkey. representation of the teeth. Journal of Behavioral Brain Research, 18, 125–137. Comparative Neurology, 467, 105–118.Murray, E. A., & Mishkin, M. (1984). Relative Remple, M. S., Reed, J. L., Stepniewska, I., & Kaas, contributions of S-II and area 5 to tactile J. H. (2006). The organization of frontoparietal discriminations in monkeys. Behavioral Brain cortex in the tree shrew (Tupaia belangeri): Research, 11, 67–83. I. Architecture, microelectrode maps andNorgren, R., & Wolf, G. (1975). Projections of corticospinal connections. Journal of thalamic gustatory and lingual areas in the rat. Comparative Neurology, 497, 1–154. Brain Research, 92, 123–129. Remple, M. S., Reed, J. L., Stepniewska, I., Lyon, D.Nudo, R. J., & Frost, S. B. (2007). The evolution of C., & Kaas, J. H. (2007). The organization of motor cortex and motor systems. In: J. H. Kaas frontoparietal cortex in the tree shrew (Tupaia & L. A. Krubitzer (Eds.), Evolution of Nervous belangeri): II Connectional evidence for a Systems (vol 3, pp. 373–395). London: Elsevier. frontal-posterior parietal network. Journal ofNudo, R. J., & Masterton, R. B. (1988). Descending Comparative Neurology, 501, 121–149. pathways to the spinal cord: A comparative Roe, A. W. (2003). Modular complexity of area V2 study of 22 mammals. Journal of Comparative in the macaque monkey. In: C. Collins & J. H. Neurology, 277, 53–79. Kaas (Eds.), The Primate Visual System (pp.Penny, G. R., Itoh, K., & Diamond, I. T. (1982). 109–138). New York: CRC Press. Cells of different sizes in the ventral nuclei Rolls, E. T. (2004). Convergence of sensory systems project to different layers of the somatic cortex in the orbitofrontal cortex in primates and in the cat. Brain Research, 242, 55–65. brain design for emotion. Anatomical Record,Picard, N. & Strick, P.L. (1996). Motor areas of the 281A, 1212–1225. medial wall: a review of their location and Romanski, L. M., & Goldman-Rakic, P. S. (2002). functional activation. Cerebral Cortex, 6, An auditory domain in primate prefrontal 342–353. cortex. Nature Neuroscience, 5, 15–16.

200 PRIMATE NEUROETHOLOGYRoss, C. F., & Martin, R. D. (2007). The role of vision Thier, P., & Andersen, R. A. (1998). Electrical in the origin and evolution of primates. In: J. H. microstimulation distinguishes distinct Kaas & T. M. Preuss (Eds.), Evolution of Nervous saccade-related areas in the posterior parietal Systems (vol 4, pp. 59–78). London: Elsevier. cortex. Journal of Neurophysiology, 80,Rouiller, E. M., Moret, V., & Liang, F. (1993). 1713–1735. Comparison of the connectional properties of Ungerleider, L. G., & Mishkin, M. (1982). Two the two forelimb areas of the rat sensorimotor cortical visual systems. In: D. G. Ingle, M. A. cortex: Support for the presence of a premotor Goodale, & R. J. Q. Mansfield (Eds.), Analysis of or supplementary motor cortical area. Visual Behavior (pp. 549–586). Cambridge, Somatosensory & Motor Research, 10, 269–289. MA: MIT Press.Schall, J. D., Stuphorn, V., & Brown, J. W. (2002). Van Hooser, S., Heimel, J., Chung, S., Nelson, S., & Monitoring and control of action by the frontal Toth, L. (2005). Orientation selectivity without lobes. Neuron, 36, 309–322. orientation maps in visual cortex of a highlySherman, S. M., & Guillery, R. W. (2005). Exploring visual mammal. Journal of Neuroscience, 25, the Thalamus and its Role in Cortical Function. 19–28. Cambridge, MA: MIT Press. Wang, Q., & Burkhalter, A. (2007). Area map ofSnyder, L. H., Batista, A. P., & Andersen, R. A. mouse visual cortex. Journal of Comparative (2000). Intention-related activity in the Neurology, 502, 339–357. posterior parietal cortex: a review. Vision Weller, R. E., & Kaas, J. H. (1989). Parameters Research, 40, 1433–1441. affecting the loss of ganglion cells of the retinaStepniewska, I., Fang, P. C., & Kaas, J. H. (2005). following ablations of striate cortex in primates. Microstimulation reveals specialized subregions Visual Neuroscience, 3, 327–349. for different complex movements in posterior Whishaw, I. Q. (2003). Did a change in sensory parietal cortex of prosimian galagos. control of skilled movements stimulate the Proceedings of the National Academy of evolution of the primate frontal cortex? Sciences, 102, 4878–4883. Behavioral Brain Research, 146, 31–41.Stepniewska, I., Preuss, T. M., & Kaas, J. H. (1993). Wilson, D. A. (2008). Olfactory cortex. In: S. Architectonics, somatotopic organization, and Firestein & G. K. Beauchamp (Eds.), The ipsilateral cortical connections of the primary Senses: A Comprehensive Reference (vol 4, pp. motor area (M1) of owl monkeys. Journal of 687–706). London: Elsevier. Comparative Neurology, 330, 238–271. Wu, CW-H., Bichot, N. P., & Kaas, J. H. (2000).Stepniewska, I., Sakai, S. T., Qi, H-X., & Kaas, J. H. Converging evidence from microstimulation, (2003). Somatosensory input to the architecture, and connections for multiple ventrolateral thalamic region (VL) in the motor areas in the frontal and cingulate cortex macaque monkey: A potential substrate for of prosimian primates. Journal of Comparative Parkinsonian tremor. Journal of Comparative Neurology, 423, 140–177. Neurology, 455, 378–395. Xu, X., Collins, C. E., Kaskan, P. M., Khaytin, I.,Swisher, J. D., Halko, M. A., Merabet, L. B., Kaas, J. H., & Casagrande, V. A. (2004). Optical McMains, S. A., & Somers, D. C. (2007). imaging of visually evoked responses in Visual topography of human intraparietal prosimian primates reveals conserved features sulcus. Journal of Neuroscience, 27, 5326–5337. of the middle temporal visual area. ProceedingsTanji, J. (1994). The supplementary motor area in the of the National Academy of Sciences, 101, cerebral cortex. Neuroscience Research, 19, 251–268. 2566–2571.

CHAPTER 11Vision: A Neuroethological PerspectiveBenjamin Y. HaydenINTRODUCTION convergence effectively doubles the number of photoreceptors devoted to a given location.Primates are fundamentally visual creatures In contrast to many other mammals, most pri-(Le Gros Clark, 1959). It is estimated that up to mates have no tapetum lucidum. This reflectivehalf of the surface area of the macaque cerebral sheet behind the retina gives photoreceptors acortex, and one third of the human cerebral second chance to catch any photons that passcortex, is specialized for visual processing into the eyes, but reduces acuity by scattering(Drury et al., 1996; Felleman & Van Essen, them. The tapetum is the reason feline eyes glow1991). Consistent with this elaboration of but human eyes do not. The absence of theneural hardware, primates have greater visual tapetum is likely an adaptation to demands foracuity than almost all other mammals and greater acuity coupled with a diurnal lifestylebirds (Kirk & Kay, 2004). Primates can classify (Kirk & Kay, 2004). The high visual acuityimages faster and more accurately than the best found in primates is also facilitated by the pre-computers. Given the close parallels in primate sence of a retinal fovea, a centrally located spe-brain evolution and specialization of the primate cialization of the retina made up entirely of conevisual system, it has been argued that increasing photoreceptors and possessing especially highdemands for visual processing were the domi- visual acuity. As a consequence, most primatesnant force driving the evolution of the primate move their eyes several times a second to bringbrain (Barton, 1998). Although the visual system the fovea into register with an area of interest inis often seen as a marvel of elegant engineering the visible scene. The speed and efficiency with(Purves et al., 2008; Wandell, 1995), like any which we can shift gaze contributes to ourother evolved trait it reflects the outcome of a acuity.long series of compromises and competingdemands. This chapter focuses on how the Orbital Convergence and Stereoscopicvisual system works, with an eye on how Visionevolutionary demands shaped visual systemfunction. The eyes of primates exhibit an extreme degree of orbital convergence, causing the visual fields of the two eyes to overlap substantially (Barton,EVOLUTIONARY INFLUENCES 2004; Le Gros Clark, 1959). Convergent visionON OUR VISUAL SYSTEM improves discrimination at the cost of reducedThe remarkable visual acuity found among pri- visual field size (Barton, 2004). The reduction inmates depends on several anatomical adapta- the size of visual field may reflect reducedtions. Primates have unusually large eyes and demands for avoiding predators (relative topupils (Kirk & Kay, 2004) that point in the ungulates, for example) and increased demandssame direction (orbital convergence). Orbital for locating animal prey. Convergent vision has 201

202 PRIMATE NEUROETHOLOGYother costs as well: Information from the two Apes, including humans, and Old Worldeyes must be combined, thus increasing compu- monkeys are predominantly trichromatstational demands on the cortical visual system, (Surridge et al., 2003). New World monkeyspossibly demanding increased brain size exhibit great heterogeneity in color vision, both(Barton, 2004). between and within species. Many are dichro- In addition to enhanced visual acuity, orbital mats, most of which typically cannot discrimi-convergence permits more accurate calculations nate red from green (just like 8% of humanof object distance. This ability, known as stereo- males) (Jacobs, 1993, 1995). Most other mam-scopic (three-dimensional) vision, is beneficial mals have dichromatic color vision (Surridgefor arboreal primates, especially during locomo- et al., 2003). Despite the obvious benefits oftion and hunting in the terminal canopy color vision, the simple physical presence of(Cartmill, 1970, 1974; Jones, 1916; Le Gros extra receptors reduces the density of rods, thusClark, 1970; Smith, 1912). reducing sensitivity to light and motion. Finally, orbital convergence facilitates Physical space on the surface of the retina isnight vision by effectively doubling the limited, so any increase in the number of conesnumber of photoreceptors looking at a given demands a reduction in the number of rods.segment of the world, suggesting that orbital Therefore, our superior color vision comes atconvergence may be an adaptation to sco- the expense of a reduction in visual acuitytopic (low-light) visual conditions confronted (Osorio & Vorobyev, 1996, 2008).by ancestral nocturnal primates (Ross et al., Trichromatic vision permits enhanced discri-2005). It is argued that, in general, visual mination of colors and thus more efficientacuity can be improved by either restricting search for fruit and leaves, which could benefitpupil diameter or aligning the eyes. Reducing both folivorous and frugivorous primatespupil diameter is especially detrimental to (Smith et al., 2003; Sumner & Mollon, 2000).nocturnal animals because it reduces the The relative preponderance of folivory (overamount of light entering the eyes. Therefore, frugivory) among trichromatic Old World mon-the fact that primates have convergent orbits keys and New World howler monkeys suggestsbut large pupils suggests that the ancestral that the demand for leaves, not fruit, has beenprimate was nocturnal, and that orbital con- the dominant factor driving color visionvergence represents an adaptation to noc- (Surridge et al., 2003). The relative importanceturnality (Allmann, 1977; Pettigrew, 1978). of color in sexual signals in Old World monkeys (e.g., mandrills) is likely to be a by-product, not a cause of trichromacy (Dixson, 2000).Trichromatic Vision Alternatively, the close match between theThe human visual system, like that of most other color coding used by our cones and the colorsOld World haplorhine primates, contains three of primate faces may reflect the outcome oftypes of color-detecting photoreceptor cells, demands for rapid assessment of the emotionalknown as cones, that selectively detect red, state of conspecifics (Changizi et al., 2006).green, and blue wavelengths of light (approxi- Finally, at night, when colors are less vibrantmately 430 nm, 530 nm, and 560 nm). Across the and the reduced light increases the demand forretina, cones compete for space with rods, which acuity, the benefits of color vision are reduced,are color-insensitive, but which provide greater and the costs increased. Thus, nocturnal owlsensitivity to light and facilitate motion detec- monkeys (the New World Aotus monkey) lacktion. Cones are especially dense at the fovea, the color vision.portion of the retina with the greatest visual Why are cones, the source of rich coloracuity. It is the contributions of these three vision, so dense at the fovea? A hint comestypes of photoreceptor cells, and their from a comparison of the retinas of primatesinteractions, that provide us with our rich with those of carnivores such as wolvescolor vision. (Mech & Boitani, 2003), large cats, and ferrets

VISION: A NEUROETHOLOGICAL PERSPECTIVE 203(Calderone & Jacobs, 2003). These animals have cortical areas (Felleman & Van Essen, 1991;similar retinal structure to primates, with foveas Wandell, 1995). Visual areas are defined by aand photoreceptor density gradients. However, combination of anatomical features and func-cats and dogs have a much greater density of tional features. Functionally, they are discreterods across the entire retina (about three to cortical regions containing retinotopic maps offour times) and have rods in the fovea, whereas all visual space. Single neurons in these areasprimates have only cones in the fovea. Moreover, respond to visual patterns in restricted regionsfor these predators, the density of cones in the of the visual field; the portion of the visual fieldfovea is between 10% and 20% that of primates in which visual patterns can activate a neuron is(Mech & Boitani, 2003). Given that rods have known as the neuron’s receptive field. Adjacentbetter temporal resolution than cones, carni- neurons in each area tend to have adjacentvores have a corresponding greater temporal receptive fields, and these collectively form aacuity than primates. Thus, it appears that the full representation of the visual field. Neuronsfact that the primary foods of anthropoid pri- in higher-level visual areas do not necessarilymates, namely fruits and leaves, are immobile have receptive fields, and so are demarcatedallows the luxury of superior color vision. through architectonic differences, neural or hemodynamic responses to different types of stimuli, or even studies of homologous visualORGANIZATION OF THE areas in other animals (Felleman & Van Essen,VISUAL SYSTEM 1991; Purves et al., 2008; Wandell, 1995).The human and rhesus macaque visual systems The first stage of cortical visual processingare remarkably similar. It is because of these occurs in the primary visual cortex, V1. V1 pro-similarities that the rhesus monkey is one of jects directly to visual areas V2, V3, V4, MT, andthe most popular organism for understanding other areas, and these then project to the rest ofhuman vision. Indeed, much more is known the brain. The rhesus monkey brain containsabout the visual system of the rhesus monkey around 32 visual areas (Felleman & Van Essen,than about the visual system of the human. Even 1991), although few (if any) of these solelyless is known about the visual systems of other respond to visual stimulation. The functionalprimates. Consequently, the present discussion neuroanatomy of the human visual system iswill be biased toward information about visual not nearly as well understood, although manyprocessing in the rhesus monkey. structures appear to be homologous, especially All visual information enters through the eyes in the early visual areas (Van Essen, 2004).and is transduced into electrical signals byphotoreceptors in a six-layer sheet of neurons Two Visual Processing Streamscalled the retina. Visual information from acrossthe retina converges in the optic nerve, and The most well-known organizing principle fortravels from there to the rest of the brain. the visual system is called the ‘‘two-streamsAbout 90% of optic nerve neurons project via model’’ (Milner & Goodale, 1995; Ungerleider &the lateral geniculate nucleus (LGN) of the tha- Mishkin, 1982). According to this model, visuallamus to the visual cortex. (Most of the information travels along two divergent, hier-remaining neurons project to the superior colli- archically organized, processing pathways. Theseculus, a midbrain structure involved in simple pathways are known as the dorsal (‘‘where’’ orvisual processing and orienting movements of ‘‘how’’) stream and the ventral (‘‘what’’) stream.the eyes and head, and to the suprachiasmatic The dorsal stream is concerned with identifyingnucleus of the hypothalamus, which regulates the location of visual stimuli and their directioncircadian rhythms, and the pretectum, which of movement; the ventral stream identifyingregulates pupil size; Purves et al., 2008). object form, shape, and identity. The dorsal Within the cerebral cortex, visual informa- stream travels along the parietal lobe, fromtion is processed in a variety of specialized visual V1 to the thick stripes of V2, to MT (possibly

204 PRIMATE NEUROETHOLOGYthrough V3), to the lateral intraparietal cortex (Felleman & Van Essen, 1991). Of the approxi-(LIP), and then to the frontal eye fields (FEF). mately 30 visual areas, there are at least 300 (out ofThe ventral stream travels along the temporal 900 possible) connections, supporting the idealobe, from V1 to the thin stripes and interstripes that the visual system is, at best, a highly tangledof V2 to V4, to the posterior inferotemporal hierarchy. Moreover, visual areas tend to getcortex (PIT), and then to anterior inferotemporal smaller as the hierarchies progress, and thecortex (AIT), and, by some definitions, to the strength of connections between them getsperirhinal cortex (Merigan & Maunsell, 1993; weaker, challenging the idea that these streamsMurray & Bussey, 1999; Murray et al., 2007). serve as major conduits of all visual information Although the two-streams model is still the (Lennie, 1998).dominant framework for thinking about the In addition, the latencies of visual responsesorganization of the visual system, its status in sequential visual areas do not provide muchremains controversial (Felleman & Van Essen, evidence for a hierarchical sequence of proces-1991; Hegde & Felleman, 2007; Hilgetag et al., sing, especially in the dorsal stream (Schmolesky1996; Lennie, 1998; Merigan & Maunsell, 1993; et al., 1998; Vanni et al., 2004). Moreover, neu-Schiller, 1993; Ungerleider & Haxby, 1994). rons in dorsal stream areas have been shown toLesion studies often affect adjacent structures be selective to form and shape (Janssen et al.,and fibers passing through lesioned areas, compli- 2008; Lehky & Sereno, 2007; Peng et al., 2008;cating interpretations of their results (Murray & Sereno & Maunsell, 1998), while neurons in theMishkin, 1998). Moreover, many studies that have ventral stream are sensitive to visual motiondirectly compared dorsal and ventral stream (Schiller, 1993; Tolias et al., 2005). Finally, thelesions have reported effects inconsistent with extent to which areas within streams form hier-the idea of distinct processing pathways archies remains murky. For example, it is gen-(Merigan & Maunsell, 1993). More generally, a erally thought that there is a clear gradient ofcomplete map of the interconnections between complexity in the form of information repre-the visual areas does not reveal two clear pathways sented by neurons in the ventral stream areas. Figure 11.1 Organization of the visual system. The visual system is often thought of as an ordered set of areas arranged in a simple pair of functionally distinct hierarchies. Although this idea has some support, anatomical considerations hint only weakly at the idea of two streams.

VISION: A NEUROETHOLOGICAL PERSPECTIVE 205However, most studies do not directly compare up 20% of energy) and the metabolic and phy-responses of single neurons in different areas sical demands of a larger brain (Aiello &using the same experimental paradigm. When Wheller, 1995; Laughlin et al., 1998). From thethey do so, it becomes difficult to qualitatively perspective of information theory (Shannon &distinguish neuronal selectivity in different areas Weaver, 1963), transmission efficiency can bein the hierarchy (e.g., Chafee & Goldman-Rakic, increased by reducing redundancy in visual1998; Hegde & Van Essen, 2007). representations (Barlow, 1961; Field, 1987; Collectively, these findings suggest that the Rieke et al., 1997). Suppose, for example, that atwo-streams hypothesis is at best oversimplified, large red square appears in the visual field. Theand at worst misleading (Fig. 11.1). Although visual system can represent each point in thethere is no single dominant alternative hypoth- square, or it can represent the edges only.esis, future directions may emphasize the impor- Because the information inside the square istance of recurrent processing (Hegde & highly redundant, there is an opportunity forFelleman, 2007), predictive encoding (Rao & an efficient system to reduce metabolic costs byBallard, 1999), and distributed decision making not representing it. Many aspects of visual repre-(Lennie, 1998). Despite the lack of a clear expla- sentation can be thought of as serving thisnatory framework, a great deal of information is purpose.known about the response properties of indivi- Note that efficiency of neural coding is quitedual neurons in each of the visual areas. different from efficiency in image encoding as this term is used in computer science. Traditionally, a computer encoding algorithm,Processing in Early Visual Areas such as Huffman encoding, must not lose anyThe visual system begins at the photoreceptors, information when it compresses data (i.e., it iswith something akin to a pixel representation of lossless), whereas neural codes lose unimportantthe visual field, and ends with a categorical information (i.e., they are lossy; Simoncelli &representation of different types of images (i.e., Olshausen, 2001). A computer algorithm mustdog vs. cat). How does this transformation be invertible, while the neural code does nothappen? We lack the detailed knowledge of the have to be (Simoncelli & Olshausen, 2001).connectional and functional properties of visual A computer algorithm is generally assumed tocortex neurons needed to make strong theories be processed in a relatively noiseless environ-about how this process happens. Moreover, we ment, while cortical processing is very noisycannot get computers to perform most higher- (Rieke et al., 1997). Finally, the efficiency of alevel visual processes, such as image classifica- neural code is determined, in part, by the tra-tion and segmentation, so we do not have a good deoff between the costs of creating more neuronscomputational model to show how it could be and producing more spikes, an empiricallydone. This section will discuss some of the large defined set of parameters whose analogs areamount of information we do know about the typically ignored in computer codesproblem. (Olshausen & Field, 1997). In general, it is assumed that the goal of thevisual system is to represent relevant features of The Retina and Lateral Geniculatethe visual world with the greatest efficiency Nucleus(Atick, 1992; Attneave, 1954; Barlow, 1961;Laughlin et al., 1998). In other words, the struc- The retina is a flexible sheet of neurons, about 2ture of the visual system reflects the outcome of cm in diameter, that is attached to the back ofevolutionary processes that emphasize the need the orbits of the eyes. At its center is the fovea, afor high-quality extraction of visual information highly sensitive patch of photoreceptors withwith minimal wasted energy. The costs asso- especially high color sensitivity. Photoreceptorciated with vision probably involve two factors: cells in the retina consist of rods and cones. Thethe metabolic demands of spiking (which take retina is generally thought to serve as a simple

206 PRIMATE NEUROETHOLOGYphotoreceptor array. However, many of its metabolically efficient code (Olshausen & Field,operations, including the decrease in responses 1997; Zhao, 2004).when a stimulus appears close to a neuron’s Most V1 neurons respond only to visual sti-receptive field (center-surround inhibition) muli located within a small region of the visualand the transformation of signals from three field, the receptive field. They typically respondcolor inputs to opponent colors, reduce redun- most strongly to lines oriented at a particulardancy (Buschbaum & Gottschalk, 1983). Indeed, angle, known as the neuron’s peak orientationmany of the computational processes of the tuning. According to legend, Hubel and Wieselretina can be described in terms of redundancy (who won the Nobel prize in Physiology andreduction, given the constraints of the visual Medicine in 1978) were having no luck usingworld (Atick & Redlich, 1991, 1992). small dots painted onto microscope slides toUnderscoring the critical importance of redun- excite V1 neurons when one of them placed adancy reduction in the retina, consider that there cracked slide into the slide projector and as it fellare about 100 million photoreceptors and only 1 into place, they heard a loud burst of neuronalmillion optic nerve fibers (Thorpe et al., 2001). activity. Indeed, action potentials in V1 neurons The next step in visual processing is the LGN are elicited quite effectively by oriented linesof the thalamus. Commonly thought of as placed within their receptive fields. Theirnothing more than a relay linking the primary responses are somewhat suppressed by similarvisual cortex to the retina, the LGN instead stimuli just outside of their receptive field, aactually appears to perform some sophisticated phenomenon known as surround suppressioncomputational processing. Like the retina, the (Hubel & Wiesel, 1968).LGN recodes visual information into a less Response properties such as orientationredundant, more efficient form (Dan et al., tuning motivated the idea that visual neurons1996). The LGN therefore takes advantage of (and sensory neurons more generally) are ‘‘fea-redundancies in the temporal (Dong & Atick, ture detectors.’’ Indeed, it is generally believed1995) and spatial structure of natural images that V1 neurons respond most strongly to theand eliminates these. Such redundancy reduc- appearance of a bar in the receptive field of ation is known as image whitening. In fact, specified orientation and color (Hubel & Wiesel,similar principles are used in standard computer 1959, 1962; Neisser, 1967). It is more accurate,image compression algorithms, such as jpeg and however, to say that they represent not imagempeg. features, but the local Fourier energy within a restricted orientation and spatial frequency domain (Albrecht et al., 1980, 1982; De ValoisArea V1 et al., 1982). A V1 neuron can thus be describedV1, the first stop for all visual information as a band-pass filter (Campbell & Robson, 1968;entering the cortex, occupies 13% of the surface Campbell et al., 1969; Enroth-Cugell & Robson,area of the macaque cerebral cortex, making it 1966), and V1 as a whole performs somethingthe largest single cortical brain region (Van akin to a localized Fourier energy analysis (i.e., aEssen, 2004). Small lesions to V1 lead to sco- wavelet decomposition) of the retinal imagetomas (i.e., a small blind spot), while larger (Olshausen & Field, 1996, 1997). The populationlesions can cause near-complete to total blind- of V1 neurons therefore contains a completeness (Sprague et al., 1977). Interestingly, the representation of the local spectrotemporalnumber of neurons in V1 is about 100 times energy patterns of the visual field.larger than the number in the LGN (Wandell, Although it is the best-studied visual area, V1 is1995). This fact suggests that V1 ‘‘unpacks’’ far from completely understood. Indeed, one ambi-visual information that has been compressed, tious paper argues, with admirable precision, thatmaking it more accessible for subsequent pro- we understand about 15% of what there is to knowcessing. This unpacking process makes the visual about V1 (Olshausen & Field, 2005). For example,code more sparse—which is a more typical studies sample neurons with a bias toward

VISION: A NEUROETHOLOGICAL PERSPECTIVE 207neurons with large somas and predicted responses properties observed in V2 include relative (as(Olshausen & Field, 2005). Moreover, the results opposed to absolute) retinal disparity (Thomasof our single-unit studies and functional magnetic et al., 2002); stereoscopic edges, which can leadresonance imaging (fMRI) studies, our two best to depth information (von der Heydt et al.,methods, do not always agree (Maier et al., 2008). 2000); three-dimensional surface configura-Finally, some recent studies indicate that many V1 tions (Bakin et al., 2000); and second-orderneurons are tuned for non-Cartesian images edges (see Figure 1, Marcus & Van Essen,(spirals, concentric circles, etc.) (Mahon & De 2002). The majority of these properties haveValois, 2001) and hermite functions (Victor et not been properly tested in V1, however, so ital., 2006), challenging the simple spectrotemporal remains unclear whether these propertiesenergy models of V1 function (Mahon & De emerge within V2.Valois, 2001). It is quite difficult to show that a given response property emerges in a particular visual area, such as V2. It must first be shownArea V2 that the response property does not appear in V1V2 is the largest visual area after V1, and is nearly or even LGN. Second, it must be shown that thatas large as V1 itself. It occupies about 10% of the the property is not initially generated in a highersurface of the cerebral cortex (Van Essen, 2004). visual area, and then transmitted, via feedback,Receptive fields of V2 neurons are larger than back to V2. Such feedback connections are quitethose of V1 neurons at a given eccentricity, and fast, so it is nearly impossible to use timingeccentricities are greater, on average, than those information to identify the source of a brainin V1. The average latency of spiking responses signal. These possibilities are only compoundedto visual information is slightly longer by the diffuse and tangled nature of the inter-(Schmolesky et al., 1998). Beyond this, however, connections within the visual system—V2response properties of V2 neurons are remark- receives direct projections from at least nineably similar to those of V1. different visual areas, which could potentially Early studies reported that V2 neurons are support the formation of any representationprimarily distinguished from V1 neurons by pattern (Felleman & Van Essen, 1991).the fact that they respond to illusory contours,such as the imaginary line formed by two ver- Object Identification and Face Processingtices of a Kanisza Triangle (Peterhans & von derHeydt, 1991; von der Heydt et al., 1984). Such Most primates are highly social, and most behavecoding could reflect a possible elaboration of the as if they identify others based on their faces andmore veridical encoding patterns observed in bodies (Cheney & Seyfarth, 1992). Given theV1. However, subsequent work has shown that importance of faces to social behavior, it has beenillusory contour coding is also found in V1 hypothesized that specialized regions of the pri-(Grosof et al., 1993; Mahon & De Valois, 2001; mate brain mediate face processing. Single neuronsRamsden et al., 2001; Sheth et al., 1996), demon- in the monkey brain respond selectively when astrating that this property does not emerge in human or monkey face is presented (Desimone,V2. It may instead emerge through local circuit 1991; Hasselmo et al., 1989; Perrett et al., 1982;activity within V1. Tsao et al., 2006). Neurons in the superior tem- It is also thought that shape tuning in V2, poral sulcus (STS) analyze movable parts of thewhich includes small contours, angles, and face, such as the eyes or mouth (Hasselmo et al.,non-Cartesian stimuli, is more complex than 1989; Perrett et al., 1984, 1985, 1992), while inferiorthat observed in V1 (Hegde & Van Essen, temporal regions appear to represent stable face2000). However, direct comparisons are rarely properties, namely identity (Hasselmo et al., 1989;if ever made, and when they are, the differences Perrett et al., 1984, 1985). To a first approximationbetween V2 and V1 are weak and unsystematic these same regions are present in the human(Hegde & Van Essen, 2007). Other functional (Haxby et al., 2000b; Rolls, 2007; see Chapter 24).

208 PRIMATE NEUROETHOLOGY The neural processes that support the ability temporal pole that represent fully elaboratedto recognize and respond to facial identity concepts, irrespective of any particular stimulusremain a contentious subject of debate (Bukach configuration. This idea, known as the ‘‘grand-et al., 2006; Dekowska et al., 2008; McKone et al., mother cell’’ hypothesis, was popularized by2007; Yovel & Kanwisher, 2004). The majority of Lettvin, who developed it as a straw man toresearch on this question has been performed demonstrate the necessity of combinatorialusing fMRI. It is clear that a specific region of coding schemes (Barlow, 1995; see also Gross,the visual cortex in humans responds selectively 2002; Konorski, 1967). Indeed, the idea of theto faces (Kanwisher et al., 1997). This region, grandmother cell is often used as the basis forknown as the fusiform face area (FFA), in the reductio ad absurdum arguments for densetemporal lobe, is hemodynamically activated population coding, in which the majority cellswhen human subjects look at faces. A homolo- in a population contribute to representation bygous area in the monkey brain is also activated representing different aspects of images (Barlow,by (monkey) faces, and contains large patches of 1972; Kandel et al., 2000; Purves et al., 2008).single neurons that are highly face specific (Tsao In fact, the grandmother cell is one endpointet al., 2006; see Chapter 24). Interestingly, a along a continuum from dense to sparse codingseparate region responds to bodies (Downing strategies. Despite the intuitive appeal of denseet al., 2001). As in the monkey, it appears that coding strategies, much neural evidence sup-distinct face-selective cortical regions represent ports the idea that representations are quitestable features such as identity and variable fea- sparse, and may even approximate the ideal oftures such as facial expression (Gobbini & grandmother cell encoding (Logothetis &Haxby, 2007; Haxby et al., 2000a, 2002); Sheinberg, 1996; Perrett et al., 1992; Tanaka,mobile features are represented in the superior 1996; but see Quiroga et al., 2008). Indeed, atemporal sulcus, while the constant features are recent study of single neurons in the humanrepresented in the fusiform gyrus (FFA). medial temporal lobe indicates that the idea of Is the FFA specialized for recognizing faces, or is the grandmother cell may not be so far-fetchedit activated when we respond to any stimulus for (Kreiman et al., 2000; Quiroga et al., 2005).which we are experts? The issue is complex because Scientists placed electrodes into the medial tem-we are all experts at recognizing faces; those of us poral lobes of humans undergoing surgical treat-who are not are likely to have pathological condi- ment for epilepsy. They found that singletions, such as autism (Dalton et al., 2005; Schultz et isolated neurons responded to particular indivi-al., 2000). When bird experts are asked to identify duals, whether presented in photograph, carica-birds, their FFAs are selectively activated (Gauthier ture, or merely a written name. These neuronset al., 2000). The same is true for car experts and seem to represent the image of the person at theexperts trained to identify novel complex shapes highest level of abstraction (i.e., the ‘‘concept’’ ofwith no inherent ecological validity (‘‘greebles’’) the person). For example, one neuron in the left(Gauthier et al., 1999). However, upon closer posterior hippocampus responded to sevenexamination, it appears that trial-to-trial variations photographs of Jennifer Aniston, but not toin performance on such tasks are well predicted by any of 80 other photographs, including one ofactivity in the FFA only for identifying faces, Jennifer Aniston and Brad Pitt together. Anotherwhereas other types of judgments, such as car neuron responded to many images of the Sydneyidentification by car experts, are mediated by Opera House, and to a photo of a Baha’i Templeother brain areas in the ventral occipitotemporal that the subject erroneously believed wascortex (Grill-Spector et al., 2004). another view of the opera house. These results suggest that visual coding is sparse, and they are consistent with the ideaGrandmother Cells and Jennifer Aniston that sparse codes are more efficient and thusWhat is the endpoint of visual coding? One adaptive than dense codes (Olshausen & Field,extreme idea is that there are neurons at the 1996, 1997, 2004; Zhao, 2004). It remains

VISION: A NEUROETHOLOGICAL PERSPECTIVE 209unclear whether these neurons are tuned for adaptiveness (Geisler, 2008; Simoncelli &other stimuli that were not probed or whether Olshausen, 2001).they were indeed unique to individual faces and The visual system is highly adapted to theplaces (Quiroga et al., 2008). It also remains statistics of the visual world we inhabit. Thus, aunclear how the coding properties of these neu- greater understanding of statistical properties ofrons change over time, as more is learned or the visual world can shed light on the evolutionforgotten about an individual. of vision (Field, 1987; Geisler, 2008; Gibson, 1966). Although natural images are highly diverse, they have several consistent propertiesNATURAL VISION that make them suitable for study: although there is virtually no limit to the number ofNatural Images things we can see, the things we are likely to seeStandard methods of characterizing response come from a very small subset of all possibleproperties of visual areas involve presenting configurations. For one thing, colors and inten-simple synthetic stimuli in their response fields. sity of adjacent pixels are highly correlatedSuch stimuli typically include spots, oriented (Simoncelli & Olshausen, 2001). It is estimatedbars, drifting gratings, Gabor patches that the average amount of information carried(Fig. 11.2), white noise, and coherently moving by a single pixel in a color photograph image isdots. It is generally assumed that neuronal 1.4 bits (Kersten, 1987), whereas the amount ofresponses to these simple stimuli will predict information stored by a computer in an explicitthose to other, more complex, naturalistic sti- representation of the same image (i.e., a bitmap)muli. However, recent work calls these assump- is 8 bits in a black and white image and 24 bits intions into question, and instead, indicates the a color image. The extra bits come from redun-importance of characterizing neural responses dant information, and they provide a greatto natural stimuli for understanding visual pro- opportunity for redundancy reduction in visualcessing (David et al., 2004). Indeed, natural sti- encoding. Such reductions appear to bemuli are those that the visual system has evolved achieved, in part, by surround suppression andto process, and are the ones that were seen most related mechanisms (Barlow, 1961; Grigorescuof the time in the environment of evolutionary et al., 2003; Olshausen & Field, 1996).Figure 11.2 Visual stimuli. (A) Examples of visual stimuli used to probe neuronal response functions.Examples include, from left to right, oriented bars, oriented sinusoids, Gabor patches, non-Cartesiangratings, angled contours, and three examples of natural images. (B) Example of Kanisza triangle. Manypeople perceive that the edges of the triangle continue into the white space between the circles. Suchcontours are known as illusory contours. (C) Example of second-order edge. The transition between onepattern and the next forms a type of contour.

210 PRIMATE NEUROETHOLOGY Interestingly, there is little correlation in 1994). Although individual images are highlyluminance or contrast of sequentially fixated variable, sizeable populations of images havelocations in a natural scene (Frazor & Geisler, consistent statistical properties. This regularity2006). This suggests that one function of sac- can be exploited by the visual system (Rudermancades (rapid, reorienting movements of the eyes; & Bialek, 1994). This particular power spectrumsee later and Chapters 15 and 26) is to increase may reflect the size invariance of naturalthe efficiency by which information is harvested. images—objects in the visual world can appearThis is done by allowing quick shifts to more at many distances, sizes, and angles. Conse-informative locations, and to help the visual quently, natural images have a fractal character,system focus on the most informative points in and thus possess a Fourier spectrum with ana scene (Fig. 11.3) (Geisler et al., 2007; Mante inverse power law distribution (Fig. 11.4)et al., 2005). Further support for the idea that (Simoncelli & Olshausen, 2001). (Of course,saccade targets are chosen to provide maximal such distributions characterize white noise asinformation comes from the finding that sac- well.) Alternatively, natural images are repletecade endpoints have more information than with edges, which have a Fourier spectrum ofrandomly chosen pixels (Reinagel & Zador, 1/f2 (Simoncelli & Olshausen, 2001).1999). Indeed, the unique power spectrum of nat- Another important statistical property of ural images is quite distinct from that of randomnatural images is that they have a characteristic images (white noise). It is likely that this idio-power spectrum that is highly biased toward low syncratic power spectrum is exploited by thefrequencies (Field, 1987; Ruderman & Bialek, visual system, and is therefore reflected in the1994). The power spectrum of natural images tuning properties of neurons in the early visualfalls off as a function of 1/f2 (Ruderman & system. For example, the aggregate tuning prop-Bialek, 1994; Tolhurst et al., 1992). This finding erties of V1 neurons appear to represent theapplies to contrast as well (Ruderman & Bialek, independent components of natural imagesFigure 11.3 Example of natural image and naturalistic saccades. Dark lines represent paths betweenprogressive saccades.

VISION: A NEUROETHOLOGICAL PERSPECTIVE 211Figure 11.4 Example of natural image and associated power spectra. Three levels of magnification areshown. Power spectra are similar at each magnification level.(Bell & Sejnowski, 1997; Olshausen & Field, with points of interest in the scene, and are1996; van Hateren & Ruderman, 1998). The directed by both internal and external factorsuse of Gabor filters (spatially localized sine (Yarbus, 1967).wave gratings; see Fig. 11.2), which are locally Attempts to study the role of saccades on visuallimited in both the spatial and spectral domains, processing have recently begun. At the simplestmay represent an optimal compromise between level, visual images can be presented with thethe spatial and spectral domains (Daugman, same or similar temporal dynamics as saccades.1985). Indeed, theoretical studies support this Neuronal responses in visual areas have character-hypothesis, and demonstrate that a Gabor istic temporal dynamics that are obscured by rapidwavelet-style decomposition may represent the presentation of visual images (David et al., 2004).most efficient way to represent the visual world These results suggest that the rate of saccades is(Field, 1987). determined to be as slow as possible (to reduce motor costs) while still yielding maximal informa-Saccades tion. Despite the added noise associated withJust as the visual system has not evolved to uncertainty in eye position and uncontrollableprocess simple synthetic images of the type nonclassical receptive field stimulation whengenerally used to characterize neuronal recep- experimental animals are free to move their eyes,tive fields, it also has not evolved to perform visual tuning and patterns of attentional modula-simple laboratory tasks, most of which require tion can be measured under free-viewing condi-stable gaze for unnaturally long periods. tions (David et al., 2008; Mazer & Gallant, 2003).Natural vision is distinguished from vision as These studies suggest that neuronal response prop-it is studied in the lab in several ways. The most erties are preserved during free-viewing and pointcharacteristic feature of natural vision is the to improvements in our understanding of visualoccurrence of saccades, the natural scanning processing that can be achieved by incorporatingmovements of the eyes that occur about three natural eye movements into future studies ofto four times a second. Saccades align the fovea visual processing.

212 PRIMATE NEUROETHOLOGYAttention excites the neuron (David et al., 2008). For example, if one were searching for Waldo, neu-Visual attention can be defined as the selection rons that did not normally respond mostof part of the visual world, whether in mind strongly to horizontal red and white lines(covert attention) or by gaze direction (overt would change their tuning to become Waldoattention) for enhanced scrutiny. Attention can detectors. It appears that the effects of attentionselect a location (spatial attention); a feature, are not limited to a single form, but insteadsuch as all the green items in a scene (feature- depend on task demands and the type of visualbased attention); or an object (object-based processing performed within an area. These stu-attention). Attention may either be directed dies confirm the validity of models of neuronalby external events, such as a flash of light (exo- responses for natural images and natural eyegenously cued attention), or inwardly, such as movements; nonetheless, it remains unclearby one’s long-term goals (endogenously how robust these findings will be in even moredirected attention; Egeth & Yantis, 1997). natural tasks.Behaviorally, attention enhances responsive- There are several important remaining ques-ness to visual stimuli, including improving tions in vision and attention. First, the exactdetection and discrimination thresholds patterns of attentional modulation remain(Desimone & Duncan, 1995; Egeth & Yantis, uncharacterized. Second, how does the micro-1997; Maunsell & Treue, 2006; Pashler, 1999; circuitry of the brain support attentional pro-Reynolds & Chelazzi, 2004). cesses? For example, it is possible that different Importantly, attention and vision are not dis- neuronal subtypes mediate different aspects ofcrete processes but instead are intertwined. In attention. Third, there is a large discontinuityother words, attention does not affect complete between blood oxygen level–dependentrepresentations formed at the end of visual pro- (BOLD)/fMRI results and single-unit phy-cessing, but influences the way visual informa- siology results. For example, spatial attentiontion is processed at all levels of cortical has large effects on the BOLD signal in V1, butprocessing (Mehta et al., 2000; Reynolds & negligible effects on single units (Yoshor et al.,Chelazzi, 2004; Treue, 2001), and may even 2007). What is the reason for this discrepancy?alter responses in the LGN (O’Connor et al., These are just a sampling of the major issues in2002). Spatial attention enhances responses of visual attention.neurons whose receptive fields match the loca-tion of the attended stimulus (McAdams & Working MemoryMaunsell, 1999; Moran & Desimone, 1985).Analogously, feature-based attention enhances Working memory may be defined as the mentalresponses of neurons whose tuning curves maintenance of task-relevant information forinclude the attended features (Hayden & use in guiding subsequent behavior (seeGallant, 2005; McAdams & Maunsell, 2000; Chapter 18). Sometimes called the mentalTreue & Martinez Trujillo, 1999). sketchpad, it is distinguished from long-term Nonetheless, the precise pattern of enhance- memory by its duration—typically a few sec-ment associated with attentional modulation is onds versus up to 80 years. It is thought thatdisputed. In some situations, attention can working memory is mediated by persistentincrease the gain of visual neurons, essentially changes in responses of individual neurons orturning up their volume (David et al., 2008; neuronal populations in specialized brain areasMcAdams & Maunsell, 1999; Treue & Martinez (Goldman-Rakic, 1995). In contrast, long-termTrujillo, 1999). In other situations, attention memory is thought to be mediated by mole-appears to enhance neuronal contrast response cular changes in specific neurons (Purves etfunctions (Reynolds et al., 2000). More power- al., 2008).fully, attention can alter a neuron’s tuning func- Incoming visual stimuli typically activatetion, changing the stimulus that most strongly single neurons in visual cortex briefly, from 50

VISION: A NEUROETHOLOGICAL PERSPECTIVE 213to 500 ms. This phasic response is thought to be Decision Makingidentical whether the information is remem- Ultimately, the goal of vision is to guide beha-bered or not. However, when information is vior (Schall, 2001). However, not much ismaintained across a delay in working memory, known about how visual processing is used toresponses of single neurons in the prefrontal guide behavioral decisions. Nonetheless, thecortex are typically elevated throughout the information we have about the visual systemdelay. Prefrontal cortex neurons are selectively provides an excellent opportunity to study thetuned for specific remembered locations neural mechanisms of decision making. The(Funahashi et al., 1989) and nonspatial features most well-understood system for visual deci-(Miller et al., 1996; Rainer et al., 1998; Scalaidhe sion making is the motion detection system,et al., 1999). These persistent elevations are the middle temporal (MT)/lateral intraparietalthought to be the memory trace (Courtney et (LIP) circuit (Shadlen & Newsome, 2001). MTal., 1998; Desimone, 1996; Funahashi et al., 1989; responses are highly specific for motionalFuster, 1973; Fuster & Alexander, 1971; direction; an energy model shows thatGoldman-Rakic, 1995; Machens et al., 2005; responses are an approximately instanta-Miller et al., 1996). Consequently, it is com- neously linear function of motion energymonly believed that the visual cortex represents (Simoncelli & Heeger, 1998). Motion energyincoming visual information but does not parti- can be manipulated along with the proportioncipate in cognitive processes such as working of coherently moving dots in a random-dotmemory and decision making (Constantinidis stimulus. In a motion discrimination task with& Steinmetz, 1996; Desimone, 1996; Goldman- low coherence stimuli, it often takes up to 2Rakic, 1995; Miller et al., 1996; Moody et al., seconds, or more, to make an accurate deci-1998). sion. In such situations, there are single MT Thus, it is traditionally thought that vision neurons that classify motion as well as, orand working memory are discrete cognitive pro- even sometimes better than, the monkeycesses, mediated by discrete brain regions. whose behavior is under study (Britten et al.,However, some evidence points to the idea that 1992, 1993; Shadlen et al., 1996). From thisthe same neurons that subserve vision are reac- neurometric/psychometric match, it has beentivated to maintain information in working inferred that MT is the area that makes deci-memory (Pasternak & Greenlee, 2005; Postle et sions about moving visual stimuli.al., 2003). Neurons in many visual areas are When visual motion is reported with anactivated during memory epochs of working orienting saccade, responses of LIP neurons gra-memory tasks (Ferrera et al., 1994; Gnadt & dually rise as evidence is accumulated. MoreAndersen, 1988; Haenny et al., 1988; Maunsell specifically, LIP neurons have localized contra-et al., 1991; Mikami & Kubota, 1980; Miller & versive response fields. As evidence is accumu-Desimone, 1994; Pasternak & Greenlee, 2005; lated that a response must be made into theSuper et al., 2001; Zhou & Fuster, 1996). receptive field, neuronal activity rises, and asAlthough these changes may reflect cognitive evidence is accumulated that a response mustprocesses other than working memory (Bisley be made away from the receptive field, firinget al., 2004; Romo et al., 2002), there is very rates fall (Roitman & Shadlen, 2002; Shadlen &little distinction between neuronal response pat- Newsome, 1996, 2001). These properties of LIPterns that support delay modulation in visual do not depend on the stimulus—the same effectscortical areas and prefrontal cortex. Moreover, are observed for abstract stimuli that providedirect comparisons of the functions of temporal explicit probability information (Yang &cortex and prefrontal cortex suggest that pre- Shadlen, 2007) and for evidence that is accumu-frontal cortex mediates executive aspects of lated over several trials about the weighting of atask performance, while visual cortex stores sen- target based on learning rules. Moreover, thesesory information (Petrides, 2000; Lebedev et al., responses do seem to be specific to the output2004).

214 PRIMATE NEUROETHOLOGYmodality—when the task dictates a response Visual Awarenessreporting the judgment with an arm movement, Consider two photographs of the beach. Theyneuronal activity in the parietal reach region are nearly identical, except that in one, a palm(PRR), an arm movement analog of the LIP, tree in the background has been digitally erased.accumulates in the same way (Musallam et al., If the two photographs are flipped back and2004; Scherberger & Andersen, 2007). forth, the tree will appear and disappear. Collectively, these results suggest that LIP However, if a white frame is inserted betweenserves to represent the accumulated weight of the two images for a tenth of a second, you willevidence that a saccade must be made toward a see no change (Simons & Levin, 1997; Simons &particular location. However, LIP function is Rensink, 2005). Even if you do detect the change,more subtle. In a situation in which the it does not pop out the way it does when theamount of reward associated with a particular images flip immediately from one to the other.response is manipulated but the evidence is not Our inability to detect surprisingly large changesmanipulated (because there is only one option), when a brief blank divides two presentations of afiring rates of LIP neurons depend on saccade similar scene is known as change blindnessvalue (Platt & Glimcher, 1999). In fact, when the (Simons & Levin, 1997; Simons & Rensink,reward is reduced in a stochastic manner, thus 2005). In fact, the blank period is not needed.providing a large reward only half the time, Even small, task-irrelevant, distracting exo-neuronal activity covaries with the economic genous cues far from the location of the changeexpected value of the target (Platt & Glimcher, can lead to change blindness (O’Regan et al.,1999). Furthermore, this pattern is obtained 1999). Moreover, change blindness is not limitedindependent of the modality of the reward; LIP to laboratory situations. Carefully constructedneurons are sensitive to both fluid and social real-world situations reveal clear limits to ourrewards, such as the opportunity to view faces moment-to-moment awareness of our visualof attractive conspecifics (Klein et al., 2008). world (Simons & Levin, 1998). Although the MT/LIP circuit is the most The fact that even the slightest interruption iswell-understood circuit for visual decision enough to erase any record of the world suggestsmaking, it is not the only one. When a saccade that our internal representation of the world istarget is selected in a visual detection task, highly fragile. Although very little is knownresponses of neurons in the FEF gradually about the mechanisms underlying change blind-evolve toward a threshold. Variability in the ness (but see Beck et al., 2001, for a discussion oftimes for single neurons to reach this threshold the neural correlates of change blindness), theis correlated with variability in saccade onset phenomenon suggests that our representation oftimes in the task, suggesting that this area plays the visual world is not nearly as rich as wea critical role in the mechanisms of visual deci- believe. Indeed, one natural interpretation ofsion making (Schall & Thompson, 1999). The the phenomenon of change blindness is that itresponses of FEF neurons do not predict a sac- reveals how little information is available to con-cade with certainty though—when a briefly pre- scious awareness at a time. Even though we feelsented stimulus is not perceived because another like we perceive and are aware of much of thestimulus appears immediately afterward (back- multitude of stimuli in our visual world, instead,ward masking), responses of FEF neurons are our awareness appears to be ruthlessly efficient,somewhat enhanced, but do not reach the only keeping track of the most important orthreshold. This finding suggests that FEF does salient features of a scene.not mediate consciousness, but instead encodesa preconscious decision variable—a hypotheticalconstruct that represents the likelihood that a CONCLUSIONsaccade will be made. The decision variable gra-dually rises over time, and when it hits a We are highly visual creatures. Vision dominatesthreshold, the decision is triggered. our mental life so much that, in English, we

VISION: A NEUROETHOLOGICAL PERSPECTIVE 215express comprehension by saying ‘‘I see.’’ Attneave, F. (1954). Some informational aspects ofPrimate behavior is controlled to a large extent visual perception. Psychological Review, 61,by what can be seen. Primate vision is deter- 183–193.mined, in large part, by the accumulated forces Bakin, J. S., Nakayama, K., & Gilbert, C. D. (2000).of natural selection. These forces have deter- Visual responses in monkey areas V1 and V2 to three-dimensional surface configurations.mined the way our eyes are situated in our Journal of Neuroscience, 20, 8188–8198.skulls, the construction of our retina, and struc- Barlow, H. B. (1961). Possible principlestures of our brain. It is clear that a richer under- underlying the transformation of sensorystanding of the statistical properties of our visual messages. In: W. A. Rosenblith (Ed.), Sensoryworld can provide insights into how the visual communication. Cambridge, MA: MIT Press.system is constructed. By a similar logic, a richer Barlow, H. B. (1972). Single units and sensation: aunderstanding of the forces that have shaped neuron doctrine for perceptual psychology?evolution can provide information about how Perception, 1, 371–394.we see the world. Much remains to be deter- Barlow, H. B. (1995). The neuron in perception. In:mined. Evolutionary pressures on our visual M. S. Gazzaniga (Ed.), The cognitivesystems must be more fully explored. The neurosciences (pp. 415–434). Cambridge, MA: MIT Press.neural processes allowing rapid and accurate Barton, R. A. (1998). Visual specialization anddetection and discrimination of behaviorally brain evolution in primates. Proceedings ofrelevant stimuli must be identified. Such infor- Biological Sciences, 265, 1933–1937.mation will provide a more complete picture of Barton, R. A. (2004). From the cover: Binocularityhow behavior affects, and is affected by, our and brain evolution in primates. Proceedings ofvisual systems. the National Academy of Sciences, 101, 10113–10115. Beck, D. M., Rees, G., Frith, C. D., & Lavie, N.REFERENCES (2001). Neural correlates of change detectionAiello, L. C., & Wheller, P. (1995). The expensive and change blindness. Nature Neuroscience, 4, tissue hypothesis: The brain and the digestive 645–650. system in human and primate evolution. Bell, A. J., & Sejnowski, T. J. (1997). The Current Anthropology, 36, 199–221. ‘‘independent components’’ of naturalAlbrecht, D. G., De Valois, R. L., & Thorell, L. G. scenes are edge filters. Vision Research, 37, (1980). Visual cortical neurons: Are bars or 3327–3338. gratings the optimal stimuli? Science, 207, Bisley, J. W., Zaksas, D., Droll, J. A., & Pasternak, T. 88–90. (2004). Activity of neurons in cortical area MTAlbrecht, D. G., De Valois, R. L., & Thorell, L. G. during a memory for motion task. Journal of (1982). Receptive fields and the optimal Neurophysiology, 91, 286–300. stimulus. Science, 216, 205. Britten, K. H., Shadlen, M. N., Newsome, W. T., &Allmann, J. (1977). Evolution of the visual system in Movshon, J. A. (1992). The analysis of visual early primates. In: J. M. Sprague & A. N. Epstein motion: A comparison of neuronal and (Eds.), Progress in psychobiology and physiological psychophysical performance. Journal of psychology. New York: Academic. Neuroscience, 12, 4745–4765.Atick, J. J. (1992). Could information theory Britten, K. H., Shadlen, M. N., Newsome, W. T., & provide an ecological theory of sensory Movshon, J. A. (1993). Responses of neurons in processing? Network: Computation in Neural macaque MT to stochastic motion signals. Systems, 3, 213–251. Vision Neuroscience, 10, 1157–1169.Atick, J. J., & Redlich, A. N. (1991). Predicting Bukach, C. M., Gauthier, I., & Tarr, M. J. (2006). ganglion and simple cell receptive field Beyond faces and modularity: The power of an organizations. International Journal of Neural expertise framework. Trends in Cognitive Systems, 1, 305–315. Science, 10, 159–166.Atick, J. J., & Redlich, A. N. (1992). What does the Buschbaum, G., & Gottschalk, A. (1983). retina know about natural scenes? Neural Trichromacy, opponent colours coding and Computation, 4, 196–210. optimum colour information transmission in

216 PRIMATE NEUROETHOLOGY the retina. Proceedings of the Royal Society of spectral tuning of V4 neurons during natural London B, 220, 89–113. vision. Neuron, 59, 509–521.Calderone, J. B., & Jacobs, G. H. (2003). Spectral David, S. V., Vinje, W. E., & Gallant, J. L. (2004). properties and retinal distribution of ferret Natural stimulus statistics alter the receptive cones. Vision Neuroscience, 20, 11–17. field structure of v1 neurons. Journal ofCampbell, F. W., Cooper, G. F., Robson, J. G., & Neuroscience, 24, 6991–7006. Sachs, M. B. (1969). The spatial selectivity of De Valois, R. L., Albrecht, D. G., & Thorell, L. G. visual cells of the cat and the squirrel monkey. (1982). Spatial frequency selectivity of cells in Journal of Physiology, 204, 120–132. macaque visual cortex. Vision Research, 22,Campbell, F. W., & Robson, J. G. (1968). 545–559. Application of Fourier analysis to the visibility Dekowska, M., Kuniecki, M., & Jaskowski, P. of gratings. Journal of Physiology, 197, 551–566. (2008). Facing facts: Neuronal mechanisms ofCartmill, M. (1970). The orbits of arboreal mammals: face perception. Acta Neurobiology Experiement A reassessment of the arboreal theory of primate (Wars), 68, 229–252. evolution. Chicago: University of Chicago. Desimone, R. (1991). Face-selective cells in theCartmill, M. (1974). Rethinking primate origins. temporal cortex of monkeys. Journal of Science, 184, 436–443. Cognitive Neuroscience, 3, 1–8.Chafee, M. V., & Goldman-Rakic, P. S. (1998). Desimone, R. (1996). Neural mechanisms for Matching patterns of activity in primate visual memory and their role in attention. prefrontal area 8a and parietal area 7ip Proceedings of the National Academy of neurons during a spatial working memory Sciences, 93, 13494–13499. task. Journal of Neurophysiology, 79, 2919–2940. Desimone, R., & Duncan, J. (1995). NeuralChangizi, M. A., Zhang, Q., & Shimojo, S. (2006). mechanisms of selective visual attention. Bare skin, blood and the evolution of primate Annual Review of Neuroscience, 18, 193–222. color vision. Biology Letters, 2, 217–221. Dixson, A. F. (2000). Primate sexuality. London:Cheney, D. L., & Seyfarth, R. M. (1992). How Oxford University Press. monkeys see the world. Chicago: University of Dong, D. W., & Atick, J. J. (1995). Statistics of Chicago Press. natural time-varying images. Network:Constantinidis, C., & Steinmetz, M. A. (1996). Computation in Neural Systems, 6, 345. Neuronal activity in posterior parietal area 7a Downing, P. E., Jiang, Y., Shuman, M., & during the delay periods of a spatial memory Kanwisher, N. (2001). A cortical area selective task. Journal of Neurophysiology, 76, 1352–1355. for visual processing of the human body.Courtney, S. M., Petit, L., Maisog, J. M., Science, 293, 2470–2473. Ungerleider, L. G., & Haxby, J. V. (1998). An Drury, H. A., Van Essen, D. C., Anderson, C. H., area specialized for spatial working memory in Lee, C. W., Coogan, T. A., & Lewis, J. W. (1996). human frontal cortex. Science, 279, 1347–1351. Computerized mappings of the cerebral cortex:Dalton, K. M., Nacewicz, B. M., Johnstone, T., A multiresolution flattening method and a Schaefer, H. S., Gernsbacher, M. A., surface-based coordinate system. Journal of Goldsmith, H. H., et al. (2005). Gaze fixation Cognitive Neuroscience, 8, 1–28. and the neural circuitry of face processing in Egeth, H. E., & Yantis, S. (1997) Visual attention: autism. Nature Neuroscience, 8, 519–526. Control, representation, and time course.Dan, Y., Atick, J. J., & Reid, R. C. (1996). Efficient Annual Review of Psychology, 48, 269–297. coding of natural scenes in the lateral geniculate Enroth-Cugell, C., & Robson, J. G. (1966). The nucleus: Experimental test of a computational contrast sensitivity of retinal ganglion cells of theory. Journal of Neuroscience, 16, 3351–3362. the cat. Journal of Physiology, 187, 517–552.Daugman, J. G. (1985). Uncertainty relation for Felleman, D. J., & Van Essen, D. C. (1991). resolution in space, spatial frequency, and Distributed hierarchical processing in the orientation optimized by two-dimensional primate cerebral cortex. Cerebral Cortex, 1, 1–47. visual cortical filters. Journal of the Optical Ferrera, V. P., Rudolph, K. K., & Maunsell, J. H. Society of America A, 2, 1160–1169. (1994). Responses of neurons in the parietalDavid, S. V., Hayden, B. Y., Mazer, J. A., & Gallant, and temporal visual pathways during a motion J. L. (2008). Attention to stimulus features shifts task. Journal of Neuroscience, 14, 6171–6186.

VISION: A NEUROETHOLOGICAL PERSPECTIVE 217Field, D. J. (1987). Relations between the statistics Goldman-Rakic, P. S. (1995). Cellular basis of of natural images and the response properties of working memory. Neuron, 14, 477–485. cortical cells. Journal of the Optical Society of Grigorescu, C., Petkov, N., & Westenberg, M. A. America A, 4, 2379–2394. (2003). Contour detection based onFrazor, R. A., & Geisler, W. S. (2006). Local nonclassical receptive field inhibition. IEEE luminance and contrast in natural images. Transactions of Image Processes, 12, 729–739. Vision Research, 46, 1585–1598. Grill-Spector, K., Knouf, N., & Kanwisher, N.Funahashi, S., Bruce, C. J., & Goldman-Rakic, P. S. (2004). The fusiform face area subserves face (1989). Mnemonic coding of visual space in the perception, not generic within-category monkey’s dorsolateral prefrontal cortex. Journal identification. Nature Neuroscience, 7, 555–562. of Neurophysiology, 61, 331–349. Grosof, D. H., Shapley, R. M., & Hawken, M. J.Fuster, J. M. (1973). Unit activity in prefrontal (1993). Macaque V1 neurons can signal cortex during delayed-response performance: ‘illusory’ contours. Nature, 365, 550–552. neuronal correlates of transient memory. Gross, C. G. (2002). Genealogy of the Journal of Neurophysiology, 36, 61–78. ‘‘grandmother cell’’. Neuroscientist, 8, 512–518.Fuster, J. M., & Alexander, G. E. (1971). Neuron Haenny, P. E., Maunsell, J. H., & Schiller, P. H. activity related to short-term memory. Science, (1988). State dependent activity in monkey 173, 652–654. visual cortex. II. Retinal and extraretinal factorsGauthier, I., Skudlarski, P., Gore, J. C., & in V4. Experimental Brain Research, 69, 245–259. Anderson, A. W. (2000). Expertise for cars and Hasselmo, M. E., Rolls, E. T., & Baylis, G. C. (1989). birds recruits brain areas involved in face The role of expression and identity in the face- recognition. Nature Neuroscience, 3, 191–197. selective responses of neurons in the temporalGauthier, I., Tarr, M. J., Anderson, A. W., visual cortex of the monkey. Behavioral Brain Skudlarski, P., & Gore, J. C. (1999). Activation Research, 32, 203–218. of the middle fusiform ‘face area’ increases with Haxby, J. V., Hoffman, E. A., & Gobbini, M. I. expertise in recognizing novel objects. Nature (2000a). The distributed human neural system Neuroscience, 2, 568–573. for face perception. Trends in Cognitive Science,Geisler, W. S. (2008). Visual perception and the 4, 223–233. statistical properties of natural scenes. Annual Haxby, J. V., Hoffman, E. A., & Gobbini, M. I. Review of Psychology, 59, 167–192. (2002). Human neural systems for faceGeisler, W. S., Albrecht, D. G., & Crane, A. M. recognition and social communication. (2007). Responses of neurons in primary Biological Psychiatry, 51, 59–67. visual cortex to transient changes in local Haxby, J. V., Ishai, I. I., Chao, L. L., contrast and luminance. Journal of Ungerleider, L. G., & Martin, I. I. (2000b). Neuroscience, 27, 5063–5067. Object-form topology in the ventral temporalGibson, J. J. (1966). The perception of the visual lobe: Response to I. Gauthier (2000). Trends in world. Boston: Houghton-Mifflin. Cognitive Science, 4, 3–4.Gnadt, J. W., & Andersen, R. A. (1988). Memory Hayden, B. Y., & Gallant, J. L. (2005). Time course related motor planning activity in posterior of attention reveals different mechanisms for parietal cortex of macaque. Experimental Brain spatial and feature-based attention in area v4. Research, 70, 216–220. Neuron, 47, 637–643.Gobbini, M. I., & Haxby, J. V. (2007). Neural Hegde, J., & Felleman, D. J. (2007). Reappraising the systems for recognition of familiar faces. functional implications of the primate visual Neuropsychologia, 45, 32–41. anatomical hierarchy. Neuroscientist, 13, 416–421.Gold, J. I., & Shadlen, M. N. (2001). Neural Hegde, J., & Van Essen, D. C. (2000). Selectivity for computations that underlie decisions about complex shapes in primate visual area V2. sensory stimuli. Trends in Cognitive Science, 5, Journal of Neuroscience, 20, RC61. 10–16. Hegde, J., & Van Essen, D. C. (2007).Gold, J. I., & Shadlen, M. N. (2002). Banburismus A comparative study of shape representation and the brain: Decoding the relationship in macaque visual areas v2 and v4. Cerebral between sensory stimuli, decisions, and Cortex, 17, 1100–1116. reward. Neuron, 36, 299–308.

218 PRIMATE NEUROETHOLOGYHilgetag, C. C., O’Neill, M. A., & Young, M. P. Kreiman, G., Koch, C., & Fried, I. (2000). (1996). Indeterminate organization of the Category-specific visual responses of single visual system. Science, 271, 776–777. neurons in the human medial temporal lobe.Horwitz, G. D., Batista, A. P., & Newsome, W. T. Nature Neuroscience, 3, 946–953. (2004). Representation of an abstract perceptual Laughlin, S. B., de Ruyter van Steveninck, R. R., & decision in macaque superior colliculus. Journal Anderson, J. C. (1998). The metabolic cost of of Neurophysiology, 91, 2281–2296. neural information. Nature Neuroscience, 1, 36–41.Hubel, D. H., & Wiesel, T. N. (1959). Receptive Le Gros Clark, W. E. (1959). The antecedents of fields of single neurones in the cat’s striate man. New York: Harper. cortex. Journal of Physiology, 148, 574–591. Le Gros Clark, W. E. (1970). History of the primates.Hubel, D. H., & Wiesel, T. N. (1962). Receptive London: British Museum of Natural History. fields, binocular interaction and functional Lebedev, M. A., Messinger, A., Kralik, J. D., & Wise, architecture in the cat’s visual cortex. Journal S. P. (2004). Representation of attended versus of Physiology, 160, 106–154. remembered locations in prefrontal cortex.Hubel, D. H., & Wiesel, T. N. (1968). Receptive fields PLoS Biology, 2, e365. and functional architecture of monkey striate Lehky, S. R., & Sereno, A. B. (2007). Comparison of cortex. Journal of Physiology, 195, 215–243. shape encoding in primate dorsal and ventralJacobs, G. H. (1993). The distribution and nature visual pathways. Journal of Neurophysiology, 97, of colour vision among the mammals. Biological 307–319. Review of the Cambridge Philosophical Society, Lennie, P. (1998). Single units and visual cortical 68, 413–471. organization. Perception, 27, 889–935.Jacobs, G. H. (1995). Variations in primate colour Logothetis, N. K., & Sheinberg, D. L. (1996). Visual vision: Mechanisms and utility. Evolutionary object recognition. Annual Review of Anthropology, 3, 196–205. Neuroscience, 19, 577–621.Janssen, P., Srivastava, S., Ombelet, S., & Orban, G. Machens, C. K., Romo, R., & Brody, C. D. (2005). A. (2008). Coding of shape and position in Flexible control of mutual inhibition: A neural macaque lateral intraparietal area. Journal of model of two-interval discrimination. Science, Neuroscience, 28, 6679–6690. 307, 1121–1124.Jones, F. W. (1916). Arboreal man. London: Mahon, L. E., & De Valois, R. L. (2001). Edward Arnold. Cartesian and non-Cartesian responses inKandel, E., Schwartz, J., & Jessell, T. (2000). LGN, V1, and V2 cells. Vision Neuroscience, Principles of neural science (4th ed.). New York: 18, 973–981. McGraw-Hill. Maier, A., Wilke, M., Aura, C., Zhu, C., Ye, F. Q., &Kanwisher, N., McDermott, J., & Chun, M. M. Leopold, D. A. (2008). Divergence of fMRI and (1997). The fusiform face area: A module in neural signals in V1 during perceptual human extrastriate cortex specialized for face suppression in the awake monkey. Nature perception. Journal of Neuroscience, 17, Neuroscience, 11, 1193–1200. 4302–4311. Mante, V., Frazor, R. A., Bonin, V., Geisler, W. S., &Kersten, D. (1987). Predictability and redundancy Carandini, M. (2005). Independence of of natural images. Journal of the Optical Society luminance and contrast in natural scenes and of America A, 4, 2395–2400. in the early visual system. Nature Neuroscience,Kirk, E. C., & Kay, R. F. (2004). The evolution of 8, 1690–1697. high visual acuity in the Anthropoidea. In: Marcus, D. S., & Van Essen, D. C. (2002). Scene C. F. Ross & R. F. Kay (Eds.), Anthropoid segmentation and attention in primate cortical origins: New visions. New York: Springer. areas V1 and V2. Journal of Neurophysiology, 88,Klein, J. T., Deaner, R. O., & Platt, M. L. (2008). 2648–2658. Neural correlates of social target value in Maunsell, J. H., Sclar, G., Nealey, T. A., & DePriest, macaque parietal cortex. Current Biology, 18, D. D. (1991). Extraretinal representations in 419–424. area V4 in the macaque monkey. VisionKonorski, J. (1967). Integrative activity of the brain: Neuroscience, 7, 561–573. An interdisciplinary approach. Chicago: Maunsell, J. H., & Treue, S. (2006). Feature-based University of Chicago Press. attention in visual cortex. Trends in Neuroscience, 29, 317–322.

VISION: A NEUROETHOLOGICAL PERSPECTIVE 219Mazer, J. A., & Gallant, J. L. (2003). Goal-related primates and rodents. Annual Review of activity in V4 during free viewing visual search. Neuroscience, 30, 99–122. Evidence for a ventral stream visual salience Murray, E. A., & Mishkin, M. (1998). Object map. Neuron, 40, 1241–1250. recognition and location memory in monkeysMcAdams, C. J., & Maunsell, J. H. (1999). with excitotoxic lesions of the amygdala and Effects of attention on orientation-tuning hippocampus. Journal of Neuroscience, 18, functions of single neurons in macaque 6568–6582. cortical area V4. Journal of Neuroscience, 19, Musallam, S., Corneil, B. D., Greger, B., 431–441. Scherberger, H., & Andersen, R. A. (2004).McAdams, C. J., & Maunsell, J. H. (2000). Cognitive control signals for neural Attention to both space and feature modulates prosthetics. Science, 305, 258–262. neuronal responses in macaque area V4. Journal Neisser, U. (1967). Cognitive psychology. New York: of Neurophysiology, 83, 1751–1755. Appleton-Century-Crofts.McKone, E., Kanwisher, N., & Duchaine, B. C. O’Connor, D. H., Fukui, M. M., Pinsk, M. A., & (2007). Can generic expertise explain special Kastner, S. (2002). Attention modulates processing for faces? Trends in Cognitive responses in the human lateral geniculate Science, 11, 8–15. nucleus. Nature Neuroscience, 5, 1203–1209.Mech, L. D., & Boitani, L. (2003). Wolves: Behavior, O’Regan, J. K., Rensink, R. A., & Clark, J. J. (1999). ecology, and conversation. Chicago: University of Change-blindness as a result of ‘mudsplashes’. Chicago Press. Nature, 398, 34.Mehta, A. D., Ulbert, I., & Schroeder, C. E. (2000). Olshausen, B. A., & Field, D. J. (1996). Emergence Intermodal selective attention in monkeys. I: of simple-cell receptive field properties by Distribution and timing of effects across visual learning a sparse code for natural images. areas. Cerebral Cortex, 10, 343–358. Nature, 381, 607–609.Merigan, W. H., & Maunsell, J. H. (1993). How Olshausen, B. A., & Field, D. J. (1997). Sparse parallel are the primate visual pathways? Annual coding with an overcomplete basis set: A Review of Neuroscience, 16, 369–402. strategy employed by V1? Vision Research, 37,Mikami, A., & Kubota, K. (1980). Inferotemporal 3311–3325. neuron activities and color discrimination with Olshausen, B. A., & Field, D. J. (2004). Sparse delay. Brain Research, 182, 65–78. coding of sensory inputs. Current Opinions inMiller, E. K., & Desimone, R. (1994). Parallel Neurobiology, 14, 481–487. neuronal mechanisms for short-term memory. Olshausen, B. A., & Field, D. J. (2005). How close Science, 263, 520–522. are we to understanding v1? NeuralMiller, E. K., Erickson, C. A., & Desimone, R. Computation, 17, 1665–1699. (1996). Neural mechanisms of visual working Osorio, D., & Vorobyev, M. (1996). Colour vision memory in prefrontal cortex of the macaque. as an adaptation to frugivory in primates. Journal of Neuroscience, 16, 5154–5167. Proceedings of Biological Sciences, 263, 593–599.Milner, A. D., & Goodale, M. A. (1995). The visual Osorio, D., & Vorobyev, M. (2008). A review of the brain in action. New York: Oxford. evolution of animal colour vision and visualMoody, S. L., Wise, S. P., di Pellegrino, G., & communication signals. Vision Research, 48, Zipser, D. (1998). A model that accounts for 2042–2051. activity in primate frontal cortex during a Pashler, H. E. (1999). The psychology of attention. delayed matching-to-sample task. Journal of Cambridge, MA: MIT Press. Neuroscience, 18, 399–410. Pasternak, T., & Greenlee, M. W. (2005). WorkingMoran, J., & Desimone, R. (1985). Selective memory in primate sensory systems. Nature attention gates visual processing in the Reviews Neuroscience, 6, 97–107. extrastriate cortex. Science, 229, 782–784. Peng, X., Sereno, M. E., Silva, A. K., Lehky, S. R., &Murray, E. A., & Bussey, T. J. (1999). Perceptual- Sereno, A. B. (2008). Shape selectivity in mnemonic functions of the perirhinal cortex. primate frontal eye field. Journal of Trends in Cognitive Science, 3, 142–151. Neurophysiology, 100, 796–814.Murray, E. A., Bussey, T. J., & Saksida, L. M. Perrett, D. I., Hietanen, J. K., Oram, M. W., & (2007). Visual perception and memory: A new Benson, P. J. (1992). Organization and view of medial temporal lobe function in functions of cells responsive to faces in the

220 PRIMATE NEUROETHOLOGY temporal cortex. Philosophical Transactions of Rainer, G., Asaad, W. F., & Miller, E. K. (1998). the Royal Society of London B Biological Memory fields of neurons in the primate Sciences, 335, 23–30. prefrontal cortex. Proceedings of the NationalPerrett, D. I., Rolls, E. T., & Caan, W. (1982). Visual Academy of Sciences, 95, 15008–15013. neurones responsive to faces in the monkey Ramsden, B. M., Hung, C. P., & Roe, A. W. (2001). temporal cortex. Experimental Brain Research, Real and illusory contour processing in area V1 47, 329–342. of the primate: A cortical balancing act. CerebralPerrett, D. I., Smith, P. A., Mistlin, A. J., Cortex, 11, 648–665. Chitty, A. J., Head, A. S., Potter, D. D., et al. Rao, R. P., & Ballard, D. H. (1999). Predictive (1985). Visual analysis of body movements by coding in the visual cortex: A functional neurones in the temporal cortex of the macaque interpretation of some extra-classical monkey: A preliminary report. Behavioral Brain receptive-field effects. Nature Neuroscience, 2, Research, 16, 153–170. 79–87.Perrett, D. I., Smith, P. A., Potter, D. D., Mistlin, Reinagel, P., & Zador, A. M. (1999). Natural scene A. J., Head, A. S., Milner, A. D., et al. (1984). statistics at the centre of gaze. Network, 10, Neurones responsive to faces in the temporal 341–350. cortex: Studies of functional organization, Reynolds, J. H., & Chelazzi, L. (2004). Attentional sensitivity to identity and relation to modulation of visual processing. Annual Review perception. Humam Neurobiology, 3, 197–208. of Neuroscience, 27, 611–647.Peterhans, E., & von der Heydt, R. (1991). Reynolds, J. H., Pasternak, T., & Desimone, R. Subjective contours—bridging the gap (2000). Attention increases sensitivity of V4 between psychophysics and physiology. Trends neurons. Neuron, 26, 703–714. in Neuroscience, 14, 112–119. Rieke, F., Warland, D., de Ruyter van Steveninck,Petrides, M. (2000). Dissociable roles of mid- R., & Bialek, W. (1997). Spikes: Exploring the dorsolateral prefrontal and anterior neural code. Cambridge, MA: MIT Press. inferotemporal cortex in visual working Roitman, J. D., & Shadlen, M. N. (2002). Response memory. Journal of Neuroscience, 20, 7496– of neurons in the lateral intraparietal area 7503. during a combined visual discriminationPettigrew, J. D. (1978). Comparison of the reaction time task. Journal of Neuroscience, 22, retinotopic organization of the visual world in 9475–9489. nocturnal and diurnal raptions, with a note on Rolls, E. T. (2007). The representation of the evolution of frontal vision. In: S. J. Cool & information about faces in the temporal and E. L. Smith (Eds.), Frontiers of visual science. frontal lobes. Neuropsychologia, 45, 124–143. New York: Springer-Verlag. Romo, R., Hernandez, A., Zainos, A., Lemus, L., &Platt, M. L., & Glimcher, P. W. (1999). Neural Brody, C. D. (2002). Neuronal correlates of correlates of decision variables in parietal decision-making in secondary somatosensory cortex. Nature, 400, 233–238. cortex. Nature Neuroscience, 5, 1217–1225.Postle, B. R., Druzgal, T. J., & D’Esposito, M. Ross, C. F., Hall, M. I., & Heesy, C. P. (2005). Were (2003). Seeking the neural substrates of visual basal primates nocturnal? Evidence from eye working memory storage. Cortex, 39, 927–946. and orbit shape. In: M. J. Ravosa &Purves, D., Augustine, G. J., Fitzpatrick, D., Hall, M. Dagosto (Eds.), Primate origins: W. C., LaMantia, A-S., McNamara, J. O., et al. Adaptations and evolution. New York, NY: (2008). Neuroscience (4th ed.). Sunderland, Springer. MA: Sinauer. Ruderman, D. L., & Bialek, W. (1994). Statistics ofQuiroga, R. Q., Kreiman, G., Koch, C., & Fried, I. natural images: Scaling in the woods. Physical (2008). Sparse but not ‘grandmother-cell’ Review Letters, 73, 814–817. coding in the medial temporal lobe. Trends in Scalaidhe, S. P., Wilson, F. A., & Goldman-Rakic, P. S. Cognitive Science, 12, 87–91. (1999). Face-selective neurons during passiveQuiroga, R. Q., Reddy, L., Kreiman, G., Koch, C., & viewing and working memory performance of Fried, I. (2005). Invariant visual representation rhesus monkeys: Evidence for intrinsic by single neurons in the human brain. Nature, specialization of neuronal coding. Cerebral 435, 1102–1107. Cortex, 9, 459–475.

VISION: A NEUROETHOLOGICAL PERSPECTIVE 221Schall, J. D. (2001). Neural basis of deciding, Simoncelli, E. P., & Olshausen, B. A. (2001). choosing and acting. Nature Review Natural image statistics and neural Neuroscience, 2, 33–42. representation. Annual Review of Neuroscience,Schall, J. D., & Thompson, K. G. (1999). Neural 24, 1193–1216. selection and control of visually guided eye Simons, D. J., & Levin, D. T. (1997). Change movements. Annual Review of Neuroscience, blindness. Trends in Cognitive Science, 1, 22, 241–259. 261–267.Scherberger, H., & Andersen, R. A. (2007). Target Simons, D. J., & Levin, D. T. (1998). Failures to selection signals for arm reaching in the detect changes to people during a real-world posterior parietal cortex. Journal of interaction. Psychonomic Bulletin and Review, Neuroscience, 27, 2001–2012. 5, 644–649.Schiller, P. H. (1993). The effects of V4 and middle Simons, D. J., & Rensink, R. A. (2005). Change temporal (MT) area lesions on visual blindness: Past, present, and future. Trends in performance in the rhesus monkey. Vision Cognitive Science, 9, 16–20. Neuroscience, 10, 717–746. Smith, A. C., Buchanan-Smith, H. M., Surridge, A.Schmolesky, M. T., Wang, Y., Hanes, D. P., K., Osorio, D., & Mundy, N. I. (2003). The effect Thompson, K. G., Leutgeb, S., Schall, J. D., of colour vision status on the detection and et al. (1998). Signal timing across the macaque selection of fruits by tamarins (Saguinus spp.). visual system. Journal of Neurophysiology, 79, Journal of Experimental Biology, 206, 3159– 3272–3278. 3165.Schultz, R. T., Gauthier, I., Klin, A., Fulbright, R. Smith, G. E. (1912). The evolution of man. In: K., Anderson, A. W., Volkmar, F., et al. (2000). Annual report. Washington, DC: Smithsonian Abnormal ventral temporal cortical activity Institution. during face discrimination among individuals Sprague, J. M., Levy, J., DiBerardino, A., & with autism and Asperger syndrome. Archives of Berlucchi, G. (1977). Visual cortical areas General Psychiatry, 57, 331–340. mediating form discrimination in the cat.Sereno, A. B., & Maunsell, J. H. (1998). Shape Journal of Comprehensive Neurology, 172, selectivity in primate lateral intraparietal 441–488. cortex. Nature, 395, 500–503. Sumner, P., & Mollon, J. D. (2000). ChromaticityShadlen, M. N., & Newsome, W. T. (1996). Motion as a signal of ripeness in fruits taken by perception: Seeing and deciding. Proceedings of primates. Journal of Experimental Biology, 203, the National Academy of Sciences, 93, 628–633. 1987–2000.Shadlen, M. N., & Newsome, W. T. (2001). Neural Super, H., Spekreijse, H., & Lamme, V. A. (2001). basis of a perceptual decision in the parietal A neural correlate of working memory in the cortex (area LIP) of the rhesus monkey primary visual cortex. Science, 293, monkey. Journal of Neurophysiology, 86, 120–124. 1916–1936. Surridge, A. K., Osorio, D., & Mundy, N. I. (2003).Shadlen, M. N., Britten, K. H., Newsome, W. T., & Evolution and selection of trichromatic vision Movshon, J. A. (1996). A computational in primates. Trends in Ecology and Evolution, 18, analysis of the relationship between neuronal 198–205. and behavioral responses to visual motion. Tanaka, K. (1996). Inferotemporal cortex and Journal of Neuroscience, 16, 1486–1510. object vision. Annual Review of Neuroscience,Shannon, C. E., & Weaver, W. (1963). The 19, 109–139. mathematical theory of communication. Thomas, O. M., Cumming, B. G., & Parker, A. J. Urbana, Chicago: University of Illinois Press. (2002). A specialization for relative disparity inSheth, B. R., Sharma, J., Rao, S. C., & Sur, M. V2. Nature Neuroscience, 5, 472–478. (1996). Orientation maps of subjective Thorpe, S., Delorme, A., & Van Rullen, R. (2001). contours in visual cortex. Science, 274, Spike-based strategies for rapid processing. 2110–2115. Neural Networks, 14, 715–725.Simoncelli, E. P., & Heeger, D. J. (1998). A model Tolhurst, D. J., Tadmor, Y., & Chao, T. (1992). of neuronal responses in visual area MT. Vision Amplitude spectra of natural images. Research, 38, 743–761. Ophthalmic Physiology Optometry, 12, 229–232.

222 PRIMATE NEUROETHOLOGYTolias, A. S., Keliris, G. A., Smirnakis, S. M., & of pattern onset responses in the human visual Logothetis, N. K. (2005). Neurons in macaque areas: an fMRI constrained VEP source analysis. area V4 acquire directional tuning after Neuroimage, 21, 801–817. adaptation to motion stimuli. Nature Victor, J. D., Mechler, F., Repucci, M. A., Purpura, Neuroscience, 8, 591–593. K. P., & Sharpee, T. (2006). Responses of V1Treue, S. (2001). Neural correlates of attention in neurons to two-dimensional hermite functions. primate visual cortex. Trends in Neuroscience, Journal of Neurophysiology, 95, 379–400. 24, 295–300. von der Heydt, R., Peterhans, E., & Baumgartner,Treue, S., Martinez Trujillo, J. C. (1999). Feature- G. (1984). Illusory contours and cortical neuron based attention influences motion processing responses. Science, 224, 1260–1262. gain in macaque visual cortex. Nature, 399, von der Heydt, R., Zhou, H., & Friedman, H. S. 575–579. (2000). Representation of stereoscopic edges inTsao, D. Y., Freiwald, W. A., Tootell, R. B., & monkey visual cortex. Vision Research, 40, Livingstone, M. S. (2006). A cortical region 1955–1967. consisting entirely of face-selective cells. Science, Wandell, B. A. (1995). Foundations of vision. 311, 670–674. Sunderland, MA: Sinauer.Ungerleider, L. G., & Mishkin, M. (1982). Two Yang, T., & Shadlen, M. N. (2007). Probabilistic cortical visual systems. In: D. J. Ingle, R. J. reasoning by neurons. Nature, 447, 1075–1080 W. Mansfield, & M. S. Goodale (Eds.), The Yarbus, A. L. (1967). Eye movements and vision. analysis of visual behavior. Cambridge, MA: New York: Plenum. MIT Press. Yoshor, D., Ghose, G. M., Bosking, W. H., Sun, P., &Ungerleider, L. G., & Haxby, J. V. (1994). ‘What’ Maunsell, J. H. (2007). Spatial attention does not and ‘where’ in the human brain. Current strongly modulate neuronal responses in early Opinions in Neurobiology, 4, 157–165. human visual cortex. Journal of Neuroscience, 27,Van Essen, D. C. (2004). Organization of visual 13205–13209. areas in macaque and human cerebral cortex. Yovel, G., & Kanwisher, N. (2004). Face perception: In: L. M. Chalupa & J. S. Werner (Eds.), The Domain specific, not process specific. Neuron, visual neurosciences. Cambridge, MA: MIT Press. 44, 889–898.van Hateren, J. H., & Ruderman, D. L. (1998). Zhao, L. (2004). Is sparse and distributed the Independent component analysis of natural coding goal of simple cells? Biological image sequences yields spatio-temporal filters Cybernetics, 91, 408–416. similar to simple cells in primary visual cortex. Zhou, Y. D., & Fuster, J. M. (1996). Mnemonic Proceedings of Biological Sciences, 265, 2315–2320. neuronal activity in somatosensory cortex.Vanni, S., Warnking, J., Dojat, M., Delon-Martin, Proceedings of the National Academy of C., Bullier, J., & Segebarth, C. (2004). Sequence Sciences, 93, 10533–10537.

CHAPTER 12Circuits of Visual AttentionTirin Moore, Robert J. Schafer and Behrad NoudoostPrimate vision is severely constrained by the fact DECIDING WHERE TO LOOKthat fine details in a visual scene can only be Numerous factors influence the decision ofresolved by the fovea, where acuity is greatest. where to shift the direction of gaze, includingThis tiny portion of each retina, which amounts factors that are both external and internal to theto less than half of 1 degree of visual angle, must viewer. Consider a driver, maneuvering a carbe moved around and positioned on behaviorally through traffic. Certain properties of the visualrelevant stimuli in order to facilitate visual per- world might draw her attention involuntarily: aception. Saccadic eye movements (saccades) flashy billboard, or a sputtering street lamp.reposition the direction of gaze (and the fovea) On the other hand, the driver can direct hersome three to five times per second and provide attention voluntarily, such as to scan streetthe means by which detailed visual information is signs to find a particular destination. Finally,accumulated during visual scanning. The ability the interaction between properties of the visualto move the eyes accurately and precisely among world and the driver’s own intentions addstargets of interest is crucial to adaptive behavior. another layer of complexity to the attention pro-Moreover, understanding how visual and sac- blem: While she might direct her gaze towardcadic mechanisms interact to choose each succes- restaurants along the side of the road if she issive target of saccades not only is central to hungry, she won’t if she has just eaten, despiteunderstanding the physiology of primate vision being presented with the same visual scene.but, as it turns out, is also important for under- One approach toward understanding howstanding the neural basis of visual attention. neural circuits direct attention has been to con-Attention is typically focused on targets of interest struct models that explain and potentially pre-by directly foveating them overtly with saccades. dict the type of scanning and search behaviorThat is, saccades are the typical way in which exhibited by primates. In creating one of the firstinformation is selectively processed and other models of visual attention, Treisman and Geladeinformation is ignored. But attention can be (1980) used human psychophysical data todirected to objects covertly as well (Sperling & demonstrate that very low-level features of aMelchner, 1978). For example, when direct eye visual scene, such as intensity, color, and orien-contact with another person of interest is avoided, tation, predict how easily a viewer finds a targetone’s attention can still be focused on him or her in an array of simple visual stimuli. When a(see Chapter 26). As we shall see, the neural target differs from all nontargets in one ormechanisms involved in mobilizing the fovea more feature dimensions (such as a red objectand the focus of attention together are also in an array of green, or a horizontal bar sur-involved in moving attention by itself. To begin, rounded by vertical ones), the target seems towe discuss the factors that determine which tar- ‘‘pop out’’ at the viewer, and the time requiredgets we attend to, either overtly or covertly. to find the target stays constant even if the 223

224 PRIMATE NEUROETHOLOGYnumber of distracters is greatly increased. This and behavior, Francolini and Egeth (1979)observation led Treisman and colleagues to suggested that the many low-level feature mapshypothesize that the visual system conducts par- might be combined with weights that can changeallel ‘‘preattentive processing’’ of the simple fea- according to the knowledge and intentions oftures of visual scenes, and then directs attention the viewer. The FeatureGate model of visualserially to each stimulus to bind the simple selection (Cave, 1999) used a related concept,features together. This model, known as the in which locations with features dissimilar tofeature-integration theory of attention, became the known target were subject to top-down inhi-the foundation of numerous subsequent bition. Similarly, Navalpakkam and Itti (2005)models of visual attention. used remembered representations of visual tar- To extend the feature-integration theory and gets to bias the map weights according to theexplain how it might be implemented in a bio- known features of a target, or to maximize dif-logically plausible model, Koch and Ullman ferences in saliency between targets and nontar-(1985) incorporated the concept of a ‘‘saliency gets (Navalpakkam & Itti, 2007).map’’ that encodes the conspicuity (‘‘salience’’)of every part of the visual scene. First, consistent NEURAL CIRCUITS FOR SHIFTINGwith the concept of preattentive processing, sev- OVERT ATTENTIONeral topographic ‘‘feature maps’’ of visual spacewere organized in parallel, one for each of a These models of bottom-up and top-downnumber of low-level visual features (color, influences on attention have found considerableorientation, direction of movement, disparity, success in describing the visual behavior of pri-etc.). Within each of these maps, a region was mates, but the question remains whether thesemarked as conspicuous if it differed along its models map onto circuits in the brain. To thefeature axis from its immediate neighborhood: extent that the models provide a conceptualFor example, in a color map, a bright red object description of how the visual and oculomotorsurrounded by green would be especially con- systems work, one would like to be able to iden-spicuous, and in a motion map, leftward motion tify brain regions with the properties of the low-surrounded by rightward motion would be as level feature maps, and perhaps with the feature-well. Finally, the conspicuous regions of each independent saliency map. (Note, however, thatfeature map were combined into a single some other models do not require separatefeature-independent saliency map of the image, feature maps [e.g., Li, 2002] or an explicitlyand a winner-take-all competition determined represented saliency map [e.g., Desimone &the most salient region. According to this model, Duncan, 1995].) Furthermore, one would likethe viewer’s gaze would then be directed to this to see pathways through which top-downpart of the scene. influences, such as prior expectations or motiva- Guidance of attention based on features of tion, can affect the representations in the featurethe image, or ‘‘bottom-up’’ guidance, captures maps, saliency map, or both. Neurophysiologicalone source of attentional signals, and is suc- studies have only begun to provide candidatecessful at mimicking how certain aspects of a neural substrates for many of these maps andvisual scene attract a viewer’s gaze. However, as connections.in the case of the driver in her car, attention can The primate visual system comprises severalalso be guided voluntarily, from the ‘‘top down.’’ distinct cortical areas, each with its ownSeveral models have maintained the concept of retinotopic map of visual space (Felleman &bottom-up processing of simple features, but Van Essen, 1991; Goodale & Milner, 1992;have added a role for ‘‘top-down’’ attention to Ungerleider & Mishkin, 1982; Van Essen et al.,influence the saliency map or the feature maps 1992). A single visual cortical neuron only repre-themselves (Wolfe, 1994). Noting that a subject’s sents information about stimuli that fall within aknowledge of task demands can modulate how limited part of the visual field, known as thedifferent visual features influence perception neuron’s receptive field (RF). Furthermore,

CIRCUITS OF VISUAL ATTENTION 225neurons in each cortical area are responsive to In particular, the frontal eye field (FEF) and lat-certain features of a stimulus: For example, neu- eral intraparietal area (LIP) in cortex and therons in monkey primary visual cortex (V1) are superior colliculus (SC) in the midbrain eachselective to the orientations of edges and in some appear to meet the requirements of saliencycases to directions of motion (Hubel & Wiesel, maps. The FEF, LIP, and SC are interconnected,1965, 1968); many neurons in area V4 are color and all receive extensive projections from areasselective (Schein & Desimone, 1990); and most throughout much of visual cortex. Despite thisneurons in the middle temporal visual area connectivity, neurons in saccade-related areas are(MT) are selective for the direction and speed not themselves selective for low-level visual fea-of visual motion (Dubner & Zeki, 1971; Zeki, tures. All three regions receive projections from1974). Despite the fact that most of these areas prefrontal cortex, which is known to be involvedare not as specialized as the low-level feature in representing top-down, task- or goal-relatedmaps in the models of attention (e.g., most information. The activity of neurons in thesecolor-selective V4 neurons are not selective for saccade-related structures also distinguishes thecolor alone, but often also for spatial frequency targets of a visual search from nontargets andand orientation of stimuli), they nonetheless represents the targets of impending shifts ofrepresent features of the visual scene in a retino- attention (Kusunoki et al., 2000; McPeek &topic manner, as described in the models. Keller, 2002; Robinson et al., 1995; Schall, 1995;Moreover, local inhibition between neurons Steinmetz & Constantinidis, 1995). Finally, thewith neighboring receptive fields in the same oculomotor structures are connected to brainvisual cortical area enhances the activity asso- regions that encode reward expectation and moti-ciated with local contrast within a feature’s vation: The caudate nucleus of the basal ganglia,dimension, as in several of the models. which is known to encode reward expectation To qualify as a saliency map, an area must and motivation, projects through the thalamussatisfy several requirements in terms of connec- to the FEF and LIP, and the SC receives projec-tivity and function (Fecteau & Munoz, 2007). tions from the caudate through the substantiaFirst, since salience can be driven by low-level nigra pars reticulata. Neurons in all three areasfeatures, a saliency map should have access to are strongly modulated by reward expectationthe numerous visual cortical areas that repre- (Ding & Hikosaka, 2006; Ikeda & Hikosaka,sent various features of visual stimuli. However, 2003; Platt & Glimcher, 1999; Sugrue et al., 2004).neurons participating in a saliency map repre-sentation should not themselves be featureselective: Instead, they should represent the fea- NEURAL CORRELATES OF OVERTtureless conspicuity of a region of visual space. AND COVERT ATTENTIONNext, a saliency map should have access to top-down signals and receive inputs from areas that Beginning in the 1980s and continuingrepresent motivation, expectation, and reward. throughout the 1990s, a large body of neurophy-In terms of function, if the saliency map is siological work established that the activity oflocalized to one or a handful of areas, then neurons within many visual cortical areas islesions to these regions should result in deficits modulated by selective attention. That is, thein shifting the focus of attention. Finally, in a magnitude of responses to visual stimulationbrain area representing a saliency map, the (i.e., number of action potentials elicited)activity of neurons should be related to the depends on whether an animal pays attentionlikelihood that the represented part of visual to that stimulus. For example, since a monkeyspace will be the target of attention. In the can be trained to repeatedly fixate and hold itsmonkey brain, an interconnected group of gaze on a spot within a stable display, the experi-areas fits these criteria: the network of menter can study the responses of single neuronsoculomotor structures, responsible for plan- within the animal’s visual cortex by presentingning and executing saccadic eye movements. visual stimuli on the display as the animal

226 PRIMATE NEUROETHOLOGYfixates. The experimenter can probe the the locus of attention (Cameron et al., 2002), theneuron’s response to stimuli presented within observation of attention-driven modulation inthe RF. Furthermore, the experimenter can monkey visual cortex provides a striking neuraltrain the monkey to covertly monitor the RF correlate of a fundamental cognitive function.stimulus (e.g., by requiring it to detect changes This correlate allows us to link the activity ofin that stimulus), and then compare the neu- single neurons within the monkey brain to aron’s responses to conditions in which the basic human mental faculty.monkey monitors a stimulus at some other loca- Before it had been established that coverttion (Fig. 12.1, left). In both cases, the monkey attention effects were widespread within visualremains fixated and thus the RF stimulus stays cortex, it had been observed that visually evokedconstant. (The monkey doesn’t do all of this neuronal responses in a number of brain struc-work for free of course, but is instead motivated tures are enhanced when an animal targets an RFby the drop of juice it receives at the end of each stimulus with a saccade. Wurtz and Goldbergtrial.) Using this basic experimental design, (1972) were the first to note this presaccadicnumerous investigators have found that the enhancement among neurons within the super-visually evoked responses of visual cortical neu- ficial layers of the SC, again a structure known torons are generally more robust when attention is play a role in the programming of visuallydirected to their RFs. The fact that the strength guided saccades. Subsequently, Mountcastleof visual representations is dynamic with respect and colleagues (1975) observed a similar effectto the locus of attention has altered the view of in posterior parietal cortex. Later studiesvisual cortical neurons as passive filters to one in demonstrated that the presaccadic visualwhich they play an active role in constructing enhancement is also observed among neuronsvisual representations according to behavioral in visual cortex, namely in area V4 (Fischer &demands. But most important, as it is known Boch, 1981), and in inferior temporal cortexthat human visual perception is heightened at (Chelazzi et al., 1993). Although there is a RF Stimulus RF Stimulus Eye Position Eye Position V4 Receptive Field Attend Saccade toNeural Ignore Neural Saccade awayResponse response Orientation OrientationFigure 12.1 Similarity of the effects of covert and overt attention on the visual responses of neurons invisual cortex. Top left: In studies of covert attention, monkeys are trained to attend (red searchlight icon) toa peripheral stimulus (yellow bar) positioned within a neuron’s receptive field (dotted circle) whileremaining fixated. The event plot at the top indicates the period over which the activity of the cell iscompared in the attended and unattended conditions (tick marks on stable eye position). Top right: Instudies of overt attention a monkey is trained to make saccades to stable visual stimuli after a delay and theeffects on the presaccadic responses are compared between the different conditions. The event plot at the topindicates the period during which the activity of the cell is compared (tick marks prior to time of saccade) inthe overtly attended (saccade to receptive field) and unattended (saccade away) conditions. The bottomplots (left and right) summarize the basic results obtained: Visual responses are enhanced when either typeof attention is directed to the receptive field stimulus, and the greatest enhancement corresponds to thestimulus that best drives the neuron.

CIRCUITS OF VISUAL ATTENTION 227striking parallel apparent between the presac- presented in the other half. The cues could becadic enhancement of visual responses presented in one of several locations in the cuedthroughout the visuo-oculomotor axis and the half of the visual field. The authors found thatattention-related modulation in visual cortex saccade trajectories were systematically deviated(Fig. 12.1, right), a connection between the two according to the location of the covertlyphenomena was ambiguous for many years. This attended (i.e., cued) location. Thus, the deploy-ambiguity was due primarily to the fact that the ment of covert attention can perturb saccadeinfluence of saccade preparation (overt atten- programming. This observation and similartion) on visual activity in some structures findings from other investigators are frequentlyappeared to be contingent upon a saccade actu- cited as evidence for what is commonly referredally being made. In other words, the covert and to as the premotor theory of attention, a modernovert attention effects seemed to occur in dif- incarnation of earlier motor theories of cognitionferent neural structures. For example, in earlier (see Moore et al., 2003, for review).experiments, there was little or no evidence that To address the role of saccade preparation indirecting covert attention to visual stimuli pre- the deployment of attention, Hoffman andsented to SC or FEF RFs had any effect on neu- Subramaniam (1995) instructed subjects toronal activity, whereas activity was found to be make saccades to a location while also detectingrobustly modulated by saccades (e.g. Goldberg & visual targets presented prior to the eye move-Bushnell, 1981). However, more recent studies ment. These authors found that target detectionhave indeed found modulation of SC and FEF was typically best at the location of plannedneuronal activity during covert attention saccades and that subjects were unable to com-(Ignashchenkova et al., 2004; Thompson et al., pletely dissociate the attended and saccade loca-2005). Thus, there is now good evidence that tions. These results suggest that the preparationvisual responses, whether exhibited by neurons of saccades to a location deploys attention tomore closely related to saccades or more closely that location. Deubel and Schneider (1996)related to coding the parameters of visual sti- observed similar results and concluded that amuli, are modulated both by covert and overt single mechanism must drive the selection ofattention. objects for perceptual processing and the information needed to drive the appropriate saccadic response.SACCADE PREPARATION AND Although these psychophysical studies pro-ATTENTION vide a sound rationale for the hypothesis thatA number of psychophysical studies have oculomotor and attentional circuits have arevealed evidence linking covert attention and common neural substrate, they cannot rule outthe preparation of saccades. Given that the visual the alternative view that these are two distinctand saccadic systems of the primate brain are and independent systems. The results fromheavily interconnected (Fig. 12.2), such links recording and lesion studies suggest that theshould not be surprising. Generally, this direction of spatial attention is caused byevidence suggests one or both of two possibili- neural activity within saccade-related structuresties: that visual spatial attention influences sac- (Moore et al., 2003). If this is the case, then itcade preparation or that saccade preparation should be possible to direct attention by per-influences the deployment of visual spatial turbing neural activity within these same struc-attention. Perhaps the most classic evidence of tures. Recent studies have used electricalthe influence of directed attention on saccades is microstimulation to test the hypothesis that sac-that provided by Sheliga and colleagues (1994). cade-related circuits play a causal role in drivingIn this study, the authors examined how covert attention to particular locations in the visualattention perturbs saccade trajectories. Subjects field (Fig. 12.3). While it had long been knownwere instructed to initiate saccades to a location that microstimulation of the FEF and the SCin one half of the visual field according to cues elicits saccades, Moore and Fallah (2001) were

Saccade Visual cortex mechanisms SEF V2 , V3 , V4 , MT , ... LIP FEF V1 dLGN Pulv ML MD SC bSGFigure 12.2 A schematic of the major pathways carrying visual information in from the retina and thenback to the eye muscles in the form of saccade commands. Several oculomotor structures involved inplanning and triggering visually guided eye movements, and their connections (gray arrows), are shown.Areas such as the frontal eye field (FEF), the lateral intraparietal area (LIP) and the superior colliculus (SC)(yellow boxes) are each involved in transforming visual information into saccade commands. Each of theseareas is connected to each other, and to visual cortex (dotted box with sample visual areas and V1), and theSC, supplementary eye field (SEF), and FEF also have projections directly to the brainstem saccade generator(bSG). Recent neurophysiological studies have uncovered a role of the FEF, lateral intraparietal area (LIP),and SC in the allocation of visual spatial attention. IML, internal medullary lamina of the thalamus; dLGN,dorsal lateral geniculate nucleus; MD, medial dorsal nucleus of the thalamus; Pulv, pulvinar nucleus. Frontal eye field Superior ColliculusFigure 12.3 Electrical microstimulation of sites within the frontal eye field (FEF) and the superiorcolliculus (SC) can elicit both shifts in gaze and in visual spatial attention. Left cartoon indicates thelocation of the FEF in the anterior bank of the arcuate sulcus in prefrontal cortex, and the location of the SCin the midbrain. Microstimulation of either structure has long been known to elicit saccades (yellow arrow,middle diagram) that are identical to voluntary ones. Recent studies demonstrate that microstimulationusing submovement stimulation currents increases covert attention (spotlight icon, right diagram) at thelocation to which suprathreshold currents shift gaze. The right diagram depicts the monkey performing acovert attention task in which it must ignore distracters (flashing squares) and respond to a dimming of aparticular target (as in Moore, T., & Fallah, M. (2001). Control of eye movements and spatial attention.Proceedings of the National Academy Sciences, 98, 1273–1276.) 228

CIRCUITS OF VISUAL ATTENTION 229the first to examine the effect of microstimula- experimenter is ultimately unable to measure thetion on visual perception. They found that when impact of microstimulation on monkeys’ per-they stimulated sites within the FEF using sub- ceptual experiences, there is probably no way ofthreshold currents (i.e., currents below that completely eliminating this possibility. How-needed to evoke a saccade), they could none- ever, there are a few key observations thattheless enhance monkeys’ abilities to detect appear to make that explanation unlikely. First,luminance changes in a visual target. Impor- as previously mentioned, there is a strict depen-tantly, the improvements observed with micro- dence of the enhancement effect on the syn-stimulation were dependent upon the target of chrony of FEF microstimulation and the targetattention being positioned at the location to event (Moore & Fallah, 2004). Specifically, whenwhich suprathreshold microstimulation would microstimulation is initiated more than 150 msshift the monkey’s gaze. Furthermore, the enhan- prior to the target event, there is no reliablecing effect of microstimulation depended on the improvement in performance, while the biggesttemporal synchrony of the microstimulation and improvement is seen when the microstimulationthe luminance change. Specifically, the more train temporally overlaps with the target event.temporal overlap, the bigger the improvement. This temporal dependence places severe limita-Thus, it appeared that by increasing the likeli- tions on the speed at which a perceptual experi-hood that the monkey would foveate a location ence could have indirectly shifted attention.in visual space, the experiments also enhanced the Cueing with visual stimuli typically requiresanimal’s ability to process visual events there. more than 50 ms between the cue and theSubsequently, two other studies reported similar target event to improve detection performanceenhancements in visual spatial attention (Posner & Cohen, 1984). Moreover, when thefollowing subthreshold microstimulation of the cue and the target event occur within less thanSC (Cavanaugh & Wurtz, 2004; Muller et al., 50 ms of each other, performance can actually be2005). In both cases, the performance-enhancing worse (Yeshurun & Carrasco, 1999), presumablyeffects of microstimulation were spatially depen- because of masking (Breitmeyer & Ganz, 1976).dent as in the FEF studies. A second set of observations that seems to rule In all of the aforementioned studies, in which against the hypothesis that attentional improve-electrical microstimulation of a saccade-related ments were indirectly due to microstimulation-structure produces spatially and temporally driven perceptual experiences is that the use ofspecific improvements in visual perception, it is artificial ‘‘phosphenes’’ have thus far failed toimportant to consider how direct the effects are. reproduce the effects of microstimulationAlthough one assumes that the results indicate (Cavanaugh & Wurtz, 2004; Muller et al.,that microstimulation directly strengthens the 2005). That is, simply replacing SC microstimu-deployment of attention at the target, and there- lation with an equally long presentation of afore reveals an underlying role of the excited Gaussian ‘‘blob’’ at the target location is notneurons (both at the site of stimulation and enough to produce the same improvementsamong neurons connected with neurons at the observed with microstimulation.site), it is important to entertain any other pos- A third study was specifically aimed at mea-sible explanations for the results. To date, the suring the perceptual impact of FEF microsti-alternative explanation most entertained is that mulation in monkeys. Murphey and Maunsellmicrostimulation of the FEF and/or the SC gen- (2008) trained monkeys to detect electricalerates a visual percept (or other ‘‘experience’’) at microstimulation of sites within the FEF usingthe corresponding point in space, for example, a currents similar to those used in the attention‘‘phosphene’’ (Cavanaugh & Wurtz, 2004; studies and across the range used to evoke sac-Moore & Fallah, 2004; Muller et al., 2005; cades. As in an earlier study conducted by theMurphey & Maunsell, 2008), and it is that same authors (Murphey & Maunsell, 2007), itpercept that in turn strengthens spatial attention was observed that monkeys could detect micro-at the corresponding point in space. Because the stimulation currents significantly below those

230 PRIMATE NEUROETHOLOGYneeded to evoke saccades. Monkeys could reli- performance in monkeys suggests that duringably report currents that ranged between 60% stimulation, visual representations of particularand 70% of the saccade-evoking current. This stimuli are enhanced above those of others.indicates that although subthreshold currents do As mentioned earlier, a wealth of evidence hasnot result in an observable change in behavior, it established that correlates of visual attention candoes not mean that the animal does not experi- be observed throughout much of macaque visualence them. What the animal experiences, how- cortex; visual responses to RF stimuli are typi-ever, is still very much an open question. cally enhanced when a monkey is trained toExperiments conducted in humans with surgi- covertly attend to them. Thus, one mightcally implanted electrodes above and near the expect that microstimulation of the FEF or thehuman homolog of the FEF seem to show that SC will likewise result in the enhancement of themicrostimulation at saccade-evoking sites does responses of visual cortical neurons to RF sti-not produce any visual experiences (Blanke et muli. Indeed, this was the hypothesis posed byal., 2000), only a sense that gaze is not under the Moore and Armstrong (2003), who paired FEFsubjects’ control (O. Blancke, personal commu- microstimulation with single-neuron recordingnication). However, by determining the current in extrastriate area V4. Specifically, they studiedmonkeys can detect, and by demonstrating that the impact of subthreshold microstimulation ofthat current can be predicted by the current FEF sites on the visual responses of V4 neuronsrequired to evoke a saccade (threshold current), with RFs that were either overlapping or non-the authors seemed to have validated the use of overlapping with the spatial location representedthe ‘‘50% of threshold’’ rule adopted in studies at the FEF site. They found that following veryof effects on attention (Cavanaugh & Wurtz, brief trains of FEF stimulation (20 to 50 ms),2004; Cavanaugh et al., 2006; Moore & Fallah, visual responses were enhanced at overlapping2001, 2004). This rule was originally adopted locations in the visual field, whereas they werebecause it was found that currents that exceeded suppressed at nonoverlapping locations. Fur-%75% of the movement threshold current thermore, this attention-like modulation wastended to impede attention performance rather greatest when there was another ‘‘distracter’’than improve it (see Fig. 2A, Moore & Fallah, stimulus presented outside of the V4 neuron’s2004). The combined results of the two FEF RF. A subsequent study also found that FEFstudies suggest that whatever the perceptual microstimulation can alter the structure of V4‘‘experience’’ induced by microstimulation, it is RFs and can bias V4 responses in favor of one oftypically absent with currents that are 50% of the two RF stimuli (Armstrong et al., 2006). Impor-movement threshold, and when it does occur it tantly, these changes in V4 responses are veryimpedes performance. Lastly, a recent study by similar to what has been observed in the sameElsley and colleagues (2007) suggests that area during voluntary attention (Armstrong &rather than visual ‘‘phosphenes,’’ subthreshold Moore, 2007) (Fig. 12.4). Together, these results‘‘experiences’’ might instead be neck muscle suggest that microstimulation of saccade-relatedproprioception. These authors report robust, structures drives covert attention and its corre-short-latency neck muscle twitches with FEF lates in visual cortex, and that indeed, as specu-microstimulation. Importantly, they found that lated by Latto and Cowey (1971), visual spatialthese neck muscle twitches frequently occurred attention seems to be the result of feedback fromeven in the absence of evoked saccades. This the oculomotor system onto visual corticalraises the possibility that even in the absence of areas, and this feedback influence may exist inmeasurable gaze shifts when delivering ‘‘suborder to modulate processing according to eyethreshold’’ currents to the FEF, proprioceptive movement plans.sensations via neck muscle twitches may none- In summary, recent neurophysiological worktheless be delivered. has directly addressed and found much evidence The effect of electrical microstimulation of of a role of saccade mechanisms in directingthe FEF and the SC on visual spatial attention covert visual spatial attention. This evidence

CIRCUITS OF VISUAL ATTENTION 231A B C Response Response Stim Control Overlap No Overlap Preferred Nonpreferred Receptive Field D E Visual Evoked stimulus Response Response saccades sp/s Eye position V Stim. E Stim. Aligned Misaligned Preferred Nonpreferred aligned alignedFigure 12.4 Modulation of visually driven responses of area V4 neurons with microstimulation of thefrontal eye field (FEF). Sites within the FEF were electrically stimulated while recording from neurons in areaV4. Cartoon shows a side view of the macaque brain. (A) Top: The locations of the FEF in the anterior bankof the arcuate sulcus and area V4 in the prelunate gyrus and below the inferior occipital sulcus are shown(shaded). Bottom left: The response of a V4 neuron when a visual stimulus (V Stim) appears in its receptivefield (RF) while the monkey maintains central fixation (dots at origin). Bottom right: Eight saccades evokedto a location overlapping the V4 RF following microstimulation of an FEF site (E stim). (B—E)Subthreshold microstimulation of the FEF increases the visual activity (‘‘Response’’) of V4 neurons withreceptive fields (circles) that overlap with the saccade represented at the stimulated FEF site (arrow fromcrosshairs), but decreases visual activity when the receptive fields and the saccades do not overlap (B). Thegray bar in each receptive field represents the visual stimulus used to drive the V4 response; in this example,the neuron prefers horizontal bars to vertical ones. The magnitude of enhancement is greatest for stimulithat evoke the largest response (C). The enhancement of V4 visual responses also depends on the spatialalignment of the receptive field stimulus and the FEF saccade vector within the RF (D). FEFmicrostimulation also biases the responses of neurons to two RF stimuli in favor of the one aligned withthe saccade vector (E).appears to support the conclusion that the the question of how the two processes interactneural mechanisms of covert and overt spatial during visually guided behavior. Specifically,attention are identical with respect to the influ- how does the brain select the target of visualence of the saccade plan on visual representa- attention, enhancing the perception of certaintions. Although it should be assumed that a features within the visual scene, and simulta-different set of mechanisms is in effect when neously use information about these sameattention is directed overtly (i.e., when planned features to specify an appropriate saccade plan?movements are triggered and shifts in gaze are Depending on the answer to this question, oneactually carried out) than when it is directed could imagine that perturbing neural activity incovertly, recent results suggest that those differ- an oculomotor region might have one of severalences are less important than the similarities. different effects: First, if visual selection and saccade specification are independent modules, it is possible that an experimental perturbation ofSimultaneous Control of Attention and neural activity could alter a saccade plan withoutSaccades noticeable effects on perception. Alternatively,The link between attention and saccades, and the the perturbation of oculomotor activity couldapparent dual role of saccade-related structures affect the perception of a visual target, presum-in controlling both of these phenomena, raises ably by modulating feedback signals to visual

232 PRIMATE NEUROETHOLOGYcortex. According to this possibility, the altered aperture, the endpoints of their saccades wereperception could be independent of a biased in the direction of the grating motion,concurrently developing saccade plan, and thus consistent with an ‘‘apparent position’’ illu-the perturbation might uncouple the link sion. This illusion, and the motion-inducedbetween perception and saccades. Finally, if per- bias of the saccades away from the center ofception and saccade selection are reciprocally the visual target, allowed the authors to sepa-linked, the perturbation might cause both a dif- rate the veridical position of the target inference in the perception of a target and an visual space from the perception of thealtered saccade plan that is consistent with the target, and thus the endpoints of the saccadesnew percept, rather than with the perturbation (Fig. 12.5).itself. Low-frequency electrical stimulation was Schafer and Moore (2007) tested these then delivered via a microelectrode to anpossible outcomes by using a paradigm in FEF site corresponding to the veridical positionwhich electrical stimulation of an FEF site of the center of the grating while the monkeypitted the potential perceptual effects of sti- planned and executed its targeting saccade. Themulation against the saccadic effects. When authors suggested at least two possiblemonkeys made voluntary saccades to a sinu- consequences of microstimulation: First, thesoidal grating drifting within a stationary effect of stimulation on saccade planning might Apparent positions Saccade of targets vectors A Spatial attention Target position Fixation spot Saccade command B Spatial attention Target position Saccade commandFigure 12.5 Interaction of attentional and saccadic signals during visually guided behavior. (A) Left:Projections from the frontal eye field (FEF) carry spatial information to visual cortex and saccade signalsthat ultimately redirect gaze, and visual target information is sent from visual cortex to the FEF. Middle:When a sinusoidal grating presented in the periphery drifts upward (green arrow) or downward (bluearrow), its apparent position is shifted in the direction of motion (green and blue dotted circles). Right:Voluntary saccades to the drifting gratings are biased in the direction of motion (green and blue lines),consistent with the apparent position illusion. (B) Left: Microstimulation of spatially alignedrepresentations in the FEF during preparation of saccades seems to enhance the interaction between theFEF and visual cortex (red lines). Middle: During stimulation the apparent position illusion is increased,and the gratings appear to be located farther from the veridical position. Right: Voluntary saccades to thedrifting gratings are influenced more strongly by grating motion, and are biased farther from the stimulationsite’s representative saccade vector (red line).

CIRCUITS OF VISUAL ATTENTION 233dominate any effects on attention and percep- the archistriatal gaze field (AGF), in the forebrain,tion, thereby influencing the monkey’s saccade which and is considered homologous with the FEFplan alone. This potential outcome would cause in primates. Microstimulation of sites within thethe saccade to land closer to the center of the part AGF evokes short-latency head movements of aof space represented by the FEF stimulation site given direction (Knudsen et al., 1995). Winkowskiand therefore toward the center of the grating, and Knudsen (2006) examined the influence ofthus eliminating the motion-induced saccade subthreshold microstimulation of AGF on audi-bias. Alternatively, if the attentional role of tory responses of tectal neurons and found thatthe FEF interacts with and informs the saccadic responses to stimuli in the space represented by therole, microstimulation could lead to an stimulated AGF site were enhanced. They alsoenhancement of the apparent position illusion found that AGF microstimulation sharpens theand a resulting increase in the motion-induced tuning of auditory receptive fields in the opticsaccade bias. The authors observed the latter: tectum and tends to shift spatial auditory tuningWhen voluntary saccades were paired with toward the auditory receptive field encoded by thelow-frequency stimulation, the effect of the AGF stimulation site (Winkowski & Knudsen,motion-induced illusion on saccade trajectories 2007). Since neurons in the optic tectum are mul-was enhanced, not decreased (Fig. 12.5). This timodal and exhibit visual responses, the authorsoutcome indicates that the attentional effects of were also able to demonstrate that AGF microsti-this FEF perturbation govern the simultaneously mulation enhances visual signals as well. Thus,planned saccades. More generally, the results evidence that gaze control mechanisms providesuggest that the feedback connections from the spatial attention-like biases in sensory processingFEF to visual cortex are integral both for appro- extends to the auditory domain, and to nonpri-priately perceiving the visual world and for pre- mate species.paring precise, target-guided saccades. The possibility that mechanisms controlling the deployment of visual spatial attention also influence other sensory modalities is notGAZE CONTROL AND entirely surprising. Psychophysical and electro-MULTISENSORY ATTENTION physiological studies have suggested that multi-In describing the evidence to date on the mechan- sensory integration occurs at very early stages ofisms within the primate brain that guide the selec- sensory processing, and perhaps before atten-tion of targets of interest, we have focused solely on tional selection is accomplished. Driver (1996)the visual modality. While this may be appropriate exploited the well-known ventriloquism illu-given the dominance of the visual modality in sion in which the speaker’s lips appear to ‘‘cap-primate species, it is nonetheless important to ture’’ a sound and translocate it when there is aconsider whether the control of visual attention spatial offset between the real auditory andis a special case. How might the findings of a role apparent visual source of the sound. Heof saccade mechanisms in visual attention showed that this cross-modal illusion cangeneralize to other sensory modalities? Could it enhance selective spatial attention to speechbe that biasing spatially directed movements sounds. It is based on the observation thatfacilitates other types of sensory processing at the under some circumstances, ventriloquism canintended location of action? Winkowski and apparently influence the ease of attentionalKnudsen (2006) provide evidence that the central selection. Participants had to repeat one of twonervous system uses a common gaze-related concurrent auditory messages, while they werestrategy for dynamically regulating sensory gain both played in mono from the same loud-across modalities, even in nonprimate species. speaker without any auditory cue that reliablyNeurons within the optic tectum of the barn owl distinguishes the relevant message from thecomprise a map of auditory space (Cohen concurrent distracting message. The distinctionand Knudsen, 1999). The owl’s gaze direction was by lip movements presented visibly, for theis controlled in part by neurons within relevant message only. The performance was

234 PRIMATE NEUROETHOLOGYimproved for recognizing the relevant message the particular classes of problems faced by anwhen this concurrent visual information was organism. One assumes that the need for morepresented at a slightly different location from ethological approaches extends to even the mostthe mono sound source. Driver attributed this complex and mysterious of functions within ouroutcome to ventriloquism arising for just the own primate brain.relevant sounds toward the concurrent lipmovements that matched them temporally inthe spatially offset condition. It was argued thatsuch visually determined illusory spatial separa- REFERENCEStion could only aid auditory selection if it arose Armstrong, K. M., Fitzgerald, J. K., & Moore, T.before that selection was complete, thus (2006). Changes in visual receptive fields withimplying that the cross-modal integration microstimulation of frontal cortex. Neuron, 50,underlying ventriloquism might be preattentive 791–798.in this particular sense. The idea was supported Armstrong, K. M., & Moore, T. (2007). Rapidlater by studies showing that the ventriloquism enhancement of visual cortical response discriminability by microstimulation of theeffect did not depend on the extent to which the frontal eye field. Proceedings of the Nationalvisual events were attended (Bertelson et al., Academy of Sciences, 104, 9499–9504.2000; Spence & Driver, 2000; Vroomen et al., Bertelson, P., Vroomen, J., de Gelder, B., & Driver, J.2001). Using event-related potentials in (2000). The ventriloquist effect does not dependhumans, Giard and Peronnet (1999) provided on the direction of deliberate visual attention.evidence that multisensory integration can take Perception & Psychophysics, 62, 321–332.place very early in the processing of auditory Blanke, O., Spinelli, L., Thut, G., Michel, C. M.,and visual information. Perrig, S., Landis, T., et al. (2000). Location of the human frontal eye field as defined by electrical cortical stimulation: Anatomical,SUMMARY REMARKS functional and electrophysiological characteristics. Neuroreport, 11, 1907–1913.For primates, the gathering of sensory informa- Breitmeyer, B. G., & Ganz, L. (1976). Implications oftion is an active process. This is particularly true sustained and transient channels for theories offor the dominant sense, vision, which requires visual pattern masking, saccadic suppression, andthe organism to sample discrete points of the informationprocessing.PsychologyReview,83,1–36.visual environment in a sequential fashion. Cameron, E. L., Tai, J. C., & Carrasco, M. (2002).This sampling is accomplished primarily by Covert attention affects the psychometric functionmoving the focus of gaze from item to item of contrast sensitivity. Vision Research, 42, 949–967.with head and eye movements, most notably Cavanaugh, J., Alvarez, B. D., & Wurtz, R. H.saccades. Neurophysiological studies have (2006). Enhanced performance with brainrecently revealed that the visual and saccadic stimulation: Attentional shift or visual cue?systems of the primate brain are not only heavily Journal of Neuroscience, 26, 11347–11358. Cavanaugh, J., & Wurtz, R. H. (2004). Subcorticalinterconnected but also that the influence of the modulation of attention counters changetwo processes on one another is bidirectional. blindness. Journal of Neuroscience, 24,Most notably, it appears that the faculty of atten- 11236–11243.tion, a fundamental cognitive function, emerges Cave, K. R. (1999). The FeatureGate model of visualfrom the reciprocal interaction of visual proces- selection. Psychology Research, 62, 182–194.sing and saccade preparation. Above its signifi- Chelazzi, L., Miller, E. K., Duncan, J., & Desimone,cance for understanding the neural basis of R. (1993). A neural basis for visual search incognition, this revelation has not only exposed inferior temporal cortex. Nature, 363, 345–347.the limitations of a strictly sensory or motor Cohen, Y. E., & Knudsen, E. I. (1999). Maps versusapproach to neural systems but also reaffirmed clusters: Different representations of auditorythe need for a more ethological approach to the space in the midbrain and forebrain. Trends in Neuroscience, 22, 128–135.question of how particular types of brains solve

CIRCUITS OF VISUAL ATTENTION 235Desimone, R., & Duncan, J. (1995). Neural Hubel, D. H., & Wiesel, T. N. (1968). Receptive fields mechanisms of selective visual attention. and functional architecture of monkey striate Annual Review of Neuroscience, 18, 193–222. cortex. Journal of Physiology, 195, 215–243.Deubel, H., & Schneider, W. X. (1996). Saccade Ignashchenkova, A., Dicke, P. W., Haarmeier, T., & target selection and object recognition: evidence Thier, P. (2004). Neuron-specific contribution of for a common attentional mechanism. Vision the superior colliculus to overt and covert shifts Research, 36, 1827–1837. of attention. Nature Neuroscience, 7, 56–64.Ding, L., & Hikosaka, O. (2006). Comparison of reward Ikeda, T., & Hikosaka, O. (2003). Reward-dependent modulationinthefrontaleyefieldandcaudateofthe gain and bias of visual responses in primate macaque. Journal of Neuroscience, 26, 6695–6703. superior colliculus. Neuron, 39, 693–700.Driver, J. (1996). Enhancement of selective Knudsen, E. I., Cohen, Y. E., & Masino, T. (1995). listening by illusory mislocation of speech Characterization of a forebrain gaze field sounds due to lip-reading. Nature, 381, 66–68. in the archistriatum of the barn owl:Dubner, R., & Zeki, S. M. (1971). Response properties Microstimulation and anatomical connections. and receptive fields of cells in an anatomically Journal of Neuroscience, 15, 5139–5151. defined region of the superior temporal sulcus in Koch, C., & Ullman, S. (1985). Shifts in selective the monkey. Brain Research, 35, 528–532. visual attention: Towards the underlying neuralElsley, J. K., Nagy, B., Cushing, S. L., & Corneil, B. D. circuitry. Human Neurobiology, 4, 219–227. (2007). Widespread presaccadic recruitment of Kusunoki, M., Gottlieb, J., & Goldberg, M. E. neck muscles by stimulation of the primate frontal (2000). The lateral intraparietal area as a eyefields.JournalofNeurophysiology,98,1333–1354. salience map: The representation of abruptFecteau, J. H., & Munoz, D. P. (2007). Warning onset, stimulus motion, and task relevance. signals influence motor processing. Journal of Vision Research, 40, 1459–1468. Neurophysiology, 97, 1600–1609. Latto, R., & Cowey, A. (1971). Visual field defectsFelleman, D. J., & Van Essen, D.C. (1991). after frontal eye-field lesions in monkeys. Brain Distributed hierarchical processing in the Research, 30, 1–24. primate cerebral cortex. Cerebral Cortex, 1, 1–47. Li, Z. (2002). A saliency map in primary visualFischer, B., & Boch, R. (1981). Enhanced activation cortex. Trends in Cognitive Science, 6, 9–16. of neurons in prelunate cortex before visually McPeek, R. M., & Keller, E. L. (2002). Saccade guided saccades of trained rhesus monkeys. target selection in the superior colliculus Experimental Brain Research, 44, 129–137. during a visual search task. Journal ofFrancolini, C. M., & Egeth, H.E. (1979). Perceptual Neurophysiology, 88, 2019–2034. selectivity is task dependent: The pop-out effect Moore, T., & Armstrong, K. M. (2003). Selective poops out. Perception & Psychophysics, 25, 99–110. gating of visual signals by microstimulation ofGiard, M. H., & Peronnet, F. (1999). Auditory-visual frontal cortex. Nature, 421, 370–373. integrationduringmultimodalobjectrecognitionin Moore, T., Armstrong, K. M., & Fallah, M. (2003). humans: a behavioral and electrophysiological Visuomotor origins of covert spatial attention. study.JournalofCognitiveNeuroscience,11,473–490. Neuron, 40, 671–683.Goldberg, M. E., & Bushnell, M. C. (1981). Moore, T., & Fallah, M. (2001). Control of eye Behavioral enhancement of visual responses in movements and spatial attention. Proceedings of monkey cerebral cortex. II. Modulation in the National Academy Sciences, 98, 1273–1276. frontal eye fields specifically related to saccades. Moore, T., & Fallah, M. (2004). Microstimulation Journal of Neurophysiology, 46, 773–787. of the frontal eye field and its effects on covertGoodale, M. A., & Milner, A. D. (1992). Separate spatial attention. Journal of Neurophysiology, visual pathways for perception and action. 91, 152–162. Trends in Neuroscience, 15, 20–25. Mountcastle, V. B., Lynch, J. C., Georgopoulos,Hoffman, J. E., & Subramaniam, B. (1995). The role A., Sakata, H., & Acuna, C. (1975). Posterior of visual attention in saccadic eye movements. parietal association cortex of the monkey: Perception & Psychophysics, 57, 787–795. Command functions for operations withinHubel, D. H., & Wiesel, T.N. (1965). Receptive extrapersonal space. Journal of fields and functional architecture in two Neurophysiology, 38, 871–908. nonstriated visual areas (18 and 19) of the cat. Muller, J. R., Philiastides, M. G., & Newsome, W. T. Journal of Neurophysiology, 28, 229–289. (2005). Microstimulation of the superior

236 PRIMATE NEUROETHOLOGY colliculus focuses attention without moving the Sugrue, L. P., Corrado, G. S., & Newsome, W. T. eyes. Proceedings of the National Academy (2004). Matching behavior and the Sciences, 102, 524–529. representation of value in the parietal cortex.Murphey, D. K., & Maunsell, J.H. (2007). Behavioral Science, 304, 1782–1787. detectionof electricalmicrosti-mulationin different Thompson, K. G., Biscoe, K. L., & Sato, T. R. cortical visual areas. Current Biology, 17, 862–867. (2005). Neuronal basis of covert spatialMurphey, D. K., & Maunsell, J. H. (2008). Electrical attention in the frontal eye field. Journal of microstimulation thresholds for behavioral Neuroscience, 25, 9479–9487. detection and saccades in monkey frontal eye Treisman, A.M. and Gelade, G. (1980). A feature- fields. Proceedings of the National Academy integration theory of attention. Cognitive Sciences, 105, 7315–7320. Psychology, 12, 97–136.Navalpakkam, V., & Itti, L. (2005). Modeling the Ungerleider, L. G. & Mishkin, M. (1982). Two cortical influence of task on attention. Vision Research, visual systems. In: D. G. Ingle, M. A. Goodale, & R. 45, 205–231. J. Q. Mansfield (Eds.), Analysis of visual behaviorNavalpakkam, V., & Itti, L. (2007). Search goal tunes (pp. 549–586). Cambridge, MA: MIT Press. visual features optimally. Neuron, 53, 605–617. Van Essen, D. C., Anderson, C. H., & Felleman, D.Platt, M.,L., & Glimcher, P. W. (1999). Neural J. (1992). Information processing in the primate correlates of decision variables in parietal visual system: An integrated systems cortex. Nature, 400, 233–238. perspective. Science, 255, 419–423.Robinson, D. L., Bowman, E. M., & Kertzman, C. Vroomen, J., Bertelson, P., & de Gelder, B. (1995). Covert orienting of attention in (2001). The ventriloquist effect does not macaques. II. Contributions of parietal cortex. depend on the direction of automatic visual Journal of Neurophysiology, 74, 698–712. attention. Perception & Psychophysics, 63,Schafer, R. J., & Moore, T. (2007). Attention 651–659. governs action in the primate frontal eye field. Winkowski, D. E., & Knudsen, E. I. (2006). Top- Neuron, 56, 541–551. down gain control of the auditory space map bySchall, J. D. (1995). Neural basis of saccade target gaze control circuitry in the barn owl. Nature, selection. Reviews Neuroscience, 6, 63–85. 439, 336–339.Schein, S. J., & Desimone, R. (1990). Spectral Winkowski, D. E., & Knudsen, E. I. (2007). Top- properties of V4 neurons in the macaque. down control of multimodal sensitivity in the Journal of Neuroscience, 10, 3369–3389. barn owl optic tectum. Journal of Neuroscience,Sheliga, B. M., Riggio, L., & Rizzolatti, G. 27, 13279–13291. (1994). Orienting of attention and eye Wolfe, J. M. (1994). Guided Search 2.0 A revised movements. Experimental Brain Research, model of visual search. Psychonomic Bulletin & 98, 507–522. Review, 1(2), 202–238.Spence, C., & Driver, J. (2000). Attracting attention Wurtz, R. H., & Goldberg, M. E. (1972). Activity to the illusory location of a sound: reflexive of superior colliculus in behaving monkey. 3. crossmodal orienting and ventriloquism. Cells discharging before eye movements. Neuroreport, 11, 2057–2061. Journal of Neurophysiology, 35, 575–586.Sperling, G., & Melchner, M. J. (1978). The Yeshurun, Y., & Carrasco, M. (1999). Spatial attention operating characteristic: Examples attention improves performance in spatial from visual search. Science, 202, 315–318. resolution tasks. Vision Research, 39, 293–306.Steinmetz, M. A., & Constantinidis, C. (1995). Zeki, S. M. (1974). Functional organization of a Neurophysiological evidence for a role of visual area in the posterior bank of the posterior parietal cortex in redirecting visual superior temporal sulcus of the rhesus attention. Cerebral Cortex, 5, 448–456. monkey. Journal of Physiology, 236, 549–573.

CHAPTER 13Vocalizations as Auditory Objects: Behavior andNeurophysiologyCory T. Miller and Yale E. CohenVocal communication is a fundamental compo- neurophysiological studies that examine vocali-nent of both human and nonhuman animal zation processing within this framework with anbehavior. Because of the behavioral significance emphasis on object and categorical processing.of vocalizations, it is likely that these acousticsignals have had a significant influence on theevolution of the primate auditory system OBJECTS(Ghazanfar & Hauser, 1999; Hauser, 1997; Objects are a core building block of our percep-Wordon & Galambos, 1972). Traditionally, it tual world. Yet, despite the significant amount ofhas been thought that the selective forces acting empirical and theoretical scientific energy that ison the primate cortex for vocalization devoted to studying objects (Nakayama, 1999;processing were related to the communicative Pylyshyn, 2006; Scholl, 2007; Scholl et al., 2001;content of the signals (Ghazanfar & Hauser, Spelke, 1994; Yantis, 2000), a formal definition1999; Hauser, 1997; Wordon & Galambos, has been difficult to establish (Feldman, 2003).1972). This, however, represents only one ele- This discrepancy is primarily due to the fact thatment of vocalizations: As spatiotemporally objects cannot be defined by the mere presencebounded units (Farris et al., 2005; Miller et al., or absence of particular features. Rather, objects2001a), vocalizations are also auditory objects come into existence (awareness) through non-(Griffiths & Warren, 2004). Since objects are linear grouping principles that bind featuresone of the most central components of percep- together into perceptually stable spatiotemporaltion, it is likely that, in the primate auditory units. Importantly, although these groupingsystem, evolution selected for those neural mechanisms are necessary for object formation,mechanisms that bind the acoustic features no single one is sufficient for object perception.into perceptual units. Thus, we propose that an Most of what is known about objects comesobject-centered view of vocalization processing from studies in the visual system. This researchmay yield new insights into the nature of the agenda comprises studies in human adults andauditory system as well as reveal potential paral- infants, as well as nonhuman primates (Hauser,lels with object processing in the visual system. 2001; Santos, 2004). As discussed below, parallelIn this chapter, we (1) provide some background work on auditory objects, though less extensive,on objects and their categorical organization, is notably similar to the body of literature on(2) review the behavioral significance of nonvisual objects. Together, these data suggest thathuman primate vocalizations, (3) discuss a gen- commonalities in object perception may existeral framework for auditory-object analysis in across developmental states, species, and perhapsthe context of vocalizations, and (4) overview sensory modalities. 237

238 PRIMATE NEUROETHOLOGY The notion that grouping principles govern edges, such as sands and liquids, are not per-object perception began with the Gestalt psy- ceived as objects (Huntley-Fenner et al., 2002).chologists (Kaffka, 1935). More recently, Spelke Moreover, as an object moves through space,sought to quantify these principles experimen- these stable boundaries persist. Second, the prin-tally in a series of studies with human infants ciple of continuity states that objects moving(Spelke, 1994). This work suggests that visual- through space and time follow a single contin-object perception is guided by three primary uous path. During experiments in which a sec-grouping principles: cohesion, continuity, and tion of an object’s path is occluded, for example,contact (Fig. 13.1). First, the principle of cohe- subjects expect the object to follow the samesion states that objects persist with stable bound- trajectory behind the occluder (Spelke et al.,aries: Sand and other substances with unstable 1994). And third, the principle of contact states Cohesion Continuity Contact Figure 13.1 Drawings depict the three principles of object perception (Spelke, 1994). Cohesion: Objects have definable, consistent boundaries. Continuity: Objects traveling on a path continue on the same trajectory even when occluded. Contact: Two objects cannot occupy the same spatial location at the same time. Purple X’s are shown on events that are inconsistent with these object properties.

VOCALIZATIONS AS AUDITORY OBJECTS: BEHAVIOR AND NEUROPHYSIOLOGY 239that because two objects cannot occupy the same the first occluder. The interpretation of this studyspatial location at the same time, an object that is that the monkeys reasoned that if the plum wasmoves into another object’s location will either not visible in the gap between the occluders, ithave its motion stopped or displace the other must have stopped behind the first occluder.object. In a classic experiment, Spelke and col- Thus, the use of spatiotemporal grouping princi-leagues (1992) presented young infants with a ples for visual objects appears to be conservedtest display in which a ball was dropped onto a across at least two primate species. The questiontable. Even when the path of the ball was now is whether these same mechanisms are evi-occluded, the infants expected the ball to come dent in other sensory modalities.to rest on top of the table and not fall through In audition, the study of auditory objects isthe solid surface. Together, these studies provide typically referred to as ‘‘auditory-scene analysis,’’empirical evidence of the perceptual principles in reference to Bregman’s seminal book on thethat govern object perception in young infants. subject (Bregman, 1990). Bregman proposed two Parallel work suggests that similar grouping forms of perceptual integration—sequential andprinciples govern object perception in adult simultaneous—which have become a key frame-human vision as well (Feldman & Tremoulet, work for thinking about the perceptual and neuro-2006; Pylyshyn, 2006; Scholl, 2001). Specifically, physiological processes underlying auditory-objecthuman adults perceive visual objects as spatio- formation. Sequential integration involves twotemporally bound units. This research also sug- complementary processes. The first process is thegests that attention and object perception are integration of temporally separated sounds fromintricately entwined, such that what is attended one sound source (e.g., syllables, words, ormay constitute an object. Scholl and colleagues vocalizations) into a coherent auditory stream.(Pylyshyn & Storm, 1988; Scholl et al., 2001) The second process is segregating sounds thatemployed the multiple-object tracking paradigm originate from separate sources, especially inter-(Sears & Pylyshyn, 2000) to examine the role of vening and overlapping sounds. Simultaneousattention in object perception. They found that integration refers to the perceptual grouping ofperceptual units whose properties were consistent different, simultaneously occurring, and often har-with Spelke’s grouping principles (Spelke, 1994) monically related components of the frequencycould be tracked through space and time, whereas spectrum into a single sound source. As in theperceptual units with ambiguous boundaries, visual system, the integration of spatiotemporalsuch as the ends of lines, could not. Together, properties is fundamental to auditory-objectthese studies suggest that the same perceptual formation.mechanisms underlying object perception in Behavioral evidence suggests that bothhuman infants persist during adulthood and that sequential and simultaneous integration are uti-attention is critical for object perception. lized during vocalization perception. This inte- Importantly, these grouping principles extend gration is perhaps best exemplified by amodalto object perception in nonhuman primates as completion (also referred to as auditory conti-well. For example, Hauser (2001) showed that nuity) (Bregman, 1990). Auditory continuitynonhuman primates understand the principle of can be seen when cotton-top tamarins are pre-contact as it relates to object perception (Spelke sented with vocalizations in which the middleet al., 1994). A study by Santos (2004) tested segment is either (1) left intact, (2) ‘‘occluded’’whether rhesus monkeys can use spatiotemporal by white noise, or (3) deleted entirely (Millerinformation when determining the location of a et al., 2001a) (Fig. 13.2). When presented withmoving object. Monkeys viewed a display in an occluded call, the tamarins respond to thiswhich a plum rolled down a ramp and behind call in the same manner as they respond to thean occluder. During certain key test conditions, intact call. Their response, however, diminishestwo occluders separated by several inches were when the middle segment is deleted, suggestingplaced along the path of the plum. Santos found that tamarins can integrate acoustic informationthat the monkeys searched for the plum behind across different frequency spectra to form a

240 PRIMATE NEUROETHOLOGY 4 ns * Number of antiphonal calls 3 * 2 1 Whole call Whistle Middle Middle noise empty Figure 13.2 Evidence of amodal completion for vocalizations (Miller et al., 2001a). Above: representative amplitude waveforms for the three test stimuli. Below: The mean number (standard error) of antiphonal calls per session is shown in the bar graph for each stimulus. Statistically significant differences are marked with an asterisk. Used with permission.coherent vocalization, even when a portion of be perceptually predictable. Consequently,the vocalization is occluded. Given that similar vocalizations represent a unique and poten-grouping has been observed in human auditory tially insightful line of research in under-(Bregman, 1990; Kubovy & Van Valkenburg, standing auditory objects. Rather than2001) and visual (Kellman et al., 1986) percep- question whether or not the unit under studytion, it is likely that this grouping principle of is an object based on the grouping principlescohesion is conserved across primate species as governing its perception, one can assume thatwell as sensory modalities. vocalizations are objects and then study more Determining which units in the world are explicitly the principles that lead to its percep-objects is a complex theoretical and empirical tual grouping.problem. The task in audition is somewhat more In conclusion, objects are among the mostdifficult than vision because sounds are spatially thoroughly studied topics in cognitive psy-omnipresent and temporally continuous. As chology. Experimental studies of the groupingsuch, establishing the boundaries of sounds in principles that bind features into objects sug-space and time during real-world situations is gest that the same perceptual mechanismsexperimentally difficult. Vocalizations, however, may underlie object perception in the visualrepresent an acoustic unit that may not succumb system of human infants and adults as well asto space- and time-boundary issues that gener- nonhuman primates. Although the picture isally surround auditory-object processing for at less clear in audition, certain commonalitiesleast two reasons. First, since a vocalization is are evident. The study of vocalizations asproduced from a single source, the auditory objects may help to learn more about thesystem can assume spatial continuity. Second, grouping principles in the auditory system.since the temporal structure of vocalizations is Such data would inform our understandingstereotyped, both in terms of the length and of whether a core set of perceptual principlesthe different acoustic elements within the calls, governs object perception across all sensorythe temporal boundaries of these objects will modalities.

VOCALIZATIONS AS AUDITORY OBJECTS: BEHAVIOR AND NEUROPHYSIOLOGY 241 Visual Auditory Frequency Time Superordinate Ball Vocalization Basic-level Soccer ball Phee call Subordinate Black & white Female soft BB group Fun to kick Small bodied etc. ‘i’m over here guys!’ etc.Figure 13.3 The categorical structure of a visual (left) and auditory (right) object. The superordinate,basic-level, and subordinate categorical structure are shown for each of the objects. The auditory object(i.e., vocalization) to the left is shown in a spectrogram, which plots the signal in a time (x-axis) byfrequency (y-axis) display. CATEGORIES objects into basic-level categories than when asked to categorize objects into superordinate An object is commonly a member of several or subordinate categories (Rosch, 1978). Also, different categories depending on what rules or the level at which an object is categorized mechanisms are used to represent the object. depends on a subject’s previous experience and These different categories can be conceptualized knowledge: A person with classical-music as forming a hierarchy (Rosch, 1978) (Fig. 13.3). training might consider the basic-level catego- At the top of the hierarchy are ‘‘superordinate’’ rical descriptor of Handel’s Messiah to be ‘‘bar- categories, the most general of the levels. ‘‘Basic- oque music’’ or even ‘‘oratorio’’ instead of level’’ categories are next and are less general ‘‘classical music’’ (Gauthier & Logothetis, 1999; than superordinate categories. Basic-level cate- Marschark et al., 2004). gories are the terms that people most commonly Different hypothetical frameworks can be use when describing an object. ‘‘Subordinate’’ used to describe the relationship between an categories are found at the bottom of the hier- object’s membership in basic-level and more archy. For example, Lassie can be categorized as superordinate categories. One framework an animal, a dog, or a collie. ‘‘Animal’’ is super- posits that superordinate categories contain a ordinate categorical-level descriptor. ‘‘Dog’’ is set of features that belong to all of the members the basic-level descriptor. ‘‘Collie’’ is the subor- of the more basic category (Damasio, 1989; dinate descriptor. Using another example, Devlin et al., 1998; Martin et al., 2002; Rosch Handel’s Messiah can be categorized as et al., 1976; Smith et al., 1974). For instance, the ‘‘music’’ (superordinate), ‘‘classical music’’ basic-level category of ‘‘dog’’ might contain (basic level), or ‘‘baroque-period music’’ (sub- descriptors like ‘‘has fur,’’ ‘‘has wet nose,’’ ‘‘has ordinate). Importantly, a single object can have four legs,’’ ‘‘breathes,’’ ‘‘is mobile,’’ ‘‘can repro- multiple subordinate categorical descriptors. duce on its own,’’ etc. In contrast, the superordi- The different categories are not equipotent nate category of animal contains descriptors that since they require different amounts of neural are not included in the basic-level category such processing. Subjects respond faster and more as ‘‘breathes,’’ ‘‘is mobile,’’ ‘‘can reproduce on its accurately when they are asked to categorize own,’’ etc. A second alternative view is that

242 PRIMATE NEUROETHOLOGYthe properties of a basic-level category are not speech units into phonetic categories (Holt, 2006;omitted from the superordinate category but Kuhl & Padden, 1982, 1983; Liberman et al., 1967;are represented as more abstract variable Lotto et al., 1998; Mann, 1980). In visual proces-values in this higher-order category. sing, this type of categorization is referred to as ‘‘object constancy’’ and allows us to recognize aPerceptual and Abstract Categories visual object from a variety of different view- points, light levels, etc. (Walsh & Kulikowski,In addition to the hierarchical structure of cate- 1998). In categorical perception, a subject’s per-gories for objects, there is at least one other ception of an object does not vary smoothly withdimension along which objects can be categor- changes in the physical properties of the objectized: perceptual and abstract. The primary dif- (Ashby & Berretty, 1997; Liberman et al., 1967;ference is that perceptual categories are formed Miller et al., 2003). In other words, two objectsbased on the common features of different that are on the same side of the categoricalobjects, whereas abstract categories are formed boundary are treated similarly, despite relativelybased on shared characteristics or previous large differences in their physical properties.knowledge about different objects. We may, for When two objects straddle the categoryexample, have a perceptual category of ‘‘dogs’’ boundary, small changes in an object’s propertiesbecause they share a similar set of features, but can lead to large changes in perception.we may also have a more abstract category of Why are objects that straddle a category‘‘past pet dogs’’ that is linked by all the dogs we boundary easier to discriminate than objectshave owned over our lives. Abstract categories that lie on the same side of the categoricalare typically learned through experience, boundary? One potential mechanism maytraining, and other top-down mechanisms, relate to the distribution of neural resources.whereas perceptual categories are formed Following behavioral training, more neuronsthrough bottom-up perceptual mechanisms. are activated in the auditory cortex during the Auditory objects with common perceptual discrimination of sounds that straddle a cate-features are combined to form perceptual categorical boundary than during the discrimina-gories (Kuhl & Padden, 1982, 1983; Lasky et al., tion of sounds that are on the same side of1975; Liberman et al., 1967). Perceptual simi- a categorical boundary (Guenther et al.,larity is one of the key elements that determine a 2004). Thus, discrimination may be primarilystimulus’s categorical membership (Boyton & limited by the number of active neurons. ThisOlson, 1987, 1990; Doupe & Kuhl, 1999; Eimas type of reorganization may be analogous toet al., 1971; Kuhl & Miller, 1975; Kuhl & Padden, the redistribution of neural tuning properties1982, 1983; Lasky et al., 1975; Liberman et al., following other types of training (Recanzone1967; Miyawaki et al., 1975; Streeter, 1976; et al., 1993).Wyttenbach et al., 1996). Perceptual categories Perceptual categories are integrated with otherare based on the physical attributes of an object. types of information to form more abstract cate-For example, male and female voices can be gorical representations. These categories arecategorized as such by attending to the pitch of formed when arbitrary stimuli are linked togetherthe voice, with female voices characteristically as a category based on some shared (nonpercep-having a higher pitch than those of males. In tual) feature, a functional characteristic, oranother example, listeners can perceive different acquired knowledge. For instance, a combinationspeech signals as belonging to the same pho- of physical characteristics and knowledge aboutnemic category, or viewers can perceive different their reproductive processes allow us to cate-visual signals as being members of the same gorize ‘‘dogs,’’ ‘‘cats,’’ and ‘‘killer whales’’ in thecolor category. category of ‘‘mammals.’’ However, if we use dif- One prominent feature of perceptual cate- ferent criteria to form a category of ‘‘pets,’’gories is that they are often accompanied by cate- ‘‘dogs’’ and ‘‘cats’’ would be members of thisgorical perception, such as the categorization of category, but ‘‘killer whales’’ would not.

VOCALIZATIONS AS AUDITORY OBJECTS: BEHAVIOR AND NEUROPHYSIOLOGY 243 Nonhuman primates can be trained to cate- idiosyncrasies in each individual’s body weightgorize stimuli into abstract categories based on and size, it still represents an important level oftheir physical characteristics. For example, mon- categorical information to conspecifics.keys can be trained to categorize objects as A further level of categorical content in voca-‘‘dogs’’ or ‘‘cats’’ (Freedman et al., 2001, 2002), lizations pertains to cues about a caller’s identity.‘‘animals’’ or ‘‘nonanimals’’ (Fabre-Thorpe Numerous experiments have demonstrated thatet al., 1998), or ‘‘trees’’ or ‘‘nontrees’’ (Vogels, nonhuman primates can extract information1999). The capacity to represent even more about a caller’s individual identity and that thisabstract categories, such as ordinal number and information plays a central role in behavioralmotion direction, is present as well (Freedman & decisions. Vervet monkey mothers, for example,Assad, 2006; Hauser et al., 2003; Nieder et al., respond only to their own infants’ cries and2002; Orlov et al., 2000; Roitman et al., 2007). ignore the same call when it is produced byOne important caveat in studies like that of other infants (Cheney & Seyfarth, 1980).Freedman and colleagues is identifying the cri- Although the ability recognize the identity ofteria that the monkeys use to classify the stimuli the caller is a salient feature in vocalizationsas ‘‘dogs’’ or ‘‘cats.’’ Do the monkeys make a cat/ (Bergman et al., 2003; Rendall et al., 1996),dog distinction using the same criteria that we other levels of identity are also encoded in voca-would naturally use to assign an animal as a lizations, such as sex (Miller et al., 2005; Rendall‘‘cat’’ or a ‘‘dog’’? Or do the monkeys learn to et al., 2004). In species with dialectic differencesuse a suite of features to distinguish between two between groups (Snowdon & Elowson, 1999;arbitrary categories that happen to coincide with Weiss et al., 2001), conspecifics may be able tohuman categories of ‘‘cat’’ and ‘‘dog’’? extract group identity (Miller et al., 2001b; Seyfarth et al., 1980). A final, potentially abstract, level of categorical information in primate vocalizations is theCATEGORICAL STRUCTURE OF referential content. Several studies of primateVOCALIZATIONS vocal communication show that certain vocali-All nonhuman primates use vocalizations to zations communicate information about objectscommunicate information to conspecifics and/or events that are external to the caller, a(Cheney & Seyfarth, 1992; Hauser, 1997). As an characteristic known as ‘‘functional reference.’’object, each vocalization is encoded with mul- A classic example of referential signaling is thetiple levels of categorical information, each of vervet monkey predator-alarm calls (Seyfarthwhich provides important behaviorally relevant et al., 1980). Vervets produce unique alarminformation to listeners. The vocalization itself calls for three different predators: a snake, arepresents the basic-level category, but much of leopard, and an eagle. The information that isthe more sophisticated information extracted transmitted to the listeners initiates specific pat-from vocalizations comes from the subordinate terns of behaviors. For instance, when vervetscategories. hear an eagle-alarm call, they scan the sky for Beyond the call type, each vocalization com- the airborne predator, and in some cases, run tomunicates a diverse range of information. Some locations that provide overhead coverage. Inof this communicative content is correlated contrast, when they hear a snake-alarm call,directly to physical characteristics of the caller. they stand up and scan the ground. Finally, aBody size, for example, is highly correlated with leopard-alarm call initiates a third distinct beha-the length of the vocal tract. Animals with larger vior: Vervets run up the nearest tree while scan-body weight and body size have longer vocal ning the horizon for the leopard.tracts, which correlate with greater formant fre- Referential signaling can also occur by com-quency dispersion for a call (Fitch, 1997; bining different vocalizations. For example,Ghazanfar et al., 2007). Although information Diana monkeys, while responding appropriatelyon this level may only exist as a consequence of to the leopard- and eagle-alarm calls of

244 PRIMATE NEUROETHOLOGYCampbell’s monkeys, have a different response 2004; Micheyl et al., 2005; Murray et al., 2006;to these alarm calls when they are preceded by a Nelken et al., 2003; Poremba et al., 2004;Campbell’s monkeys ‘‘boom’’ vocalization; Rauschecker, 1998; Scott, 2005; Scott & Wise,the boom signifies a lower-level threat 2004; Scott et al., 2000, 2004; Sussman, 2004; ¨(Zuberbuhler, 2000). More recent studies sug- Ulanovsky et al., 2003; Wise et al., 2001;gest that when putty-nosed monkeys elicit leo- Zatorre et al., 2004). Under this framework, wepard-alarm call and an eagle-alarm call can generate a model of how vocalizations aresequentially, they form a new referent that trans- transformed from acoustic waveforms to per-mits information about group movement ceptual/cognitive states that guide action and ¨(Arnold & Zuberbuhler, 2006). Experimental decisions. This model can then be used as atests of a vocalization’s referential quality have guide to design experiments that will test thebeen limited to those calls that elicit strong relationship between neural activity and the per-behavioral responses during playback experi- ceptual organization of information communi-ments, such as alarm calls and food calls. Most cated by the vocalizations.primate vocalizations, however, do not elicit The first step in auditory-object analysis is forsuch overt behavioral responses during play- the perceptual system to extract and code thebacks, making it logistically difficult to examine spectrotemporal properties, localization cues,a call’s functional reference. Consequently, any and other low-level features in the signal. Theseconclusion that a call does not have a functional features are then ‘‘bound’’ together to form areferent may not be correct. Rather, it may representation of the object. The next compo-reflect a limitation in the sensitivity of our nents of auditory-object analysis involve com-experimental methods. putations that lead to the formation of Primate vocalizations consist of multiple increasingly more abstract category-based repre-levels of categorical information. Each call type sentations. We consider these steps to be serial inrepresents a basic-level category. Several nature only as a useful conceptual heuristic.subordinate categories are also evident, ranging Ultimately, however, the cortex is likely to pro-from acoustic information that directly corre- cess the object properties and categorical struc-sponds to physical characteristics to individual ture of vocalizations in a dynamic parallelidentity and referential content. Importantly, system.those calls that have a functional referentshould not be considered to be a distinct class A PATHWAY FOR VOCALIZATIONof calls. Instead, these calls possess a more PROCESSINGabstract level of categorical information. Asobjects, vocalizations consist of rich categorical Where in the cortical hierarchy are objects andinformation. Traditionally, studies of primate categories processed? In the cortex, the mostcommunication have not approached vocaliza- likely pathway for vocalization processing is thetions from the perspective of its categorical orga- so-called ‘‘ventral’’ processing stream, a pathwaynization. However, we believe that much can be that processes the nonspatial attributes of anlearned about vocalizations, at both the beha- auditory stimulus (Rauschecker & Tian, 2000;vioral and neural levels, from this approach. Ungerleider & Mishkin, 1982). This pathway originates in the auditory cortex (Kaas & Hackett, 2000). In the auditory cortex, thereVOCALIZATION PROCESSING AS are three levels of processing: the core, belt, andAUDITORY-OBJECT ANALYSIS parabelt. These three processing levels form theWe propose that vocalizations can be analyzed bases for the spatial and nonspatial pathways.within a general framework of auditory-object The ventral stream is defined by a series of pro-analysis, in terms of both their perception and jections that includes the anterior belt of thecategorization (Blank et al., 2002, 2003; Darwin, auditory cortex and regions of the prefrontal1997; De Santis et al., 2007; Griffiths & Warren, cortex, specifically the ventrolateral prefrontal

VOCALIZATIONS AS AUDITORY OBJECTS: BEHAVIOR AND NEUROPHYSIOLOGY 245cortex (vPFC) (Rauschecker & Tian, 2000; properties of a stimulus, a fundamental require-Romanski et al., 1999a, 1999b). ment for object perception. Of course, this ventral pathway is thought to Several studies have had substantial successrun parallel to, and independently from, a in demonstrating a role for the A1 and non-‘‘dorsal’’ processing stream; this dorsal pathway primary auditory cortex in processing importantis thought to preferentially process the spatial object-related attributes of vocalizations. One ofproperties of an auditory stimulus (Rauschecker the more striking examples is that of ‘‘pitch’’& Tian, 2000; Ungerleider & Mishkin, 1982). neurons in the awake marmoset A1 (Bendor &Together, these dorsal and ventral pathways Wang, 2005, 2006); these pitch neurons straddleform an important model of auditory function, the border between the AI and R, two core tono-as well as a visual function (Ungerleider & topic fields of the auditory cortex. Bendor andMishkin, 1982). We would caution, though, that Wang demonstrated that these neurons respondthe strict functional segregation of these two pro- equally well to complex stimuli that contain acessing streams is unclear since recent data sug- fundamental-frequency component with highergest that, like the visual system, there is harmonics as they do to stimuli that lack theconsiderable crosstalk between the dorsal and fundamental frequency but contain the higherventral pathways (Sereno & Maunsell, 1998; harmonics. Other examples of object-relatedToth & Assad, 2002). activity are found in studies examining the neural correlates of different psychoacoustic parameters such as spectral contrast, roughness,NEURAL CORRELATES OF flutter, and consonance (Barbour & Wang, 2003;AUDITORY-OBJECT AND Bendor & Wang, 2007; Fishman et al., 2000,AUDITORY-CATEGORY 2001b). Together, these results suggest thatPROCESSING even in the A1, neurons are beginning to integrate sounds across spectral and temporalNeural Correlates of Object Processing dimensions—a necessary mechanism for objectA number of neurophysiological studies using perception.vocalizations as stimuli show that neurons in the Several neurophysiological studies haveauditory cortex likely play an important role in directly assessed the role of the auditory cortexauditory object perception. For example, neu- in object perception. Whereas few of these stu-rons in the primary auditory cortex (A1) are dies have used vocalizations as stimuli, theirpreferentially sensitive to species-specific vocali- results speak to potential mechanisms under-zations (Wang & Kadia, 2001). That is, the firing lying vocalization processing. First, Sutter andrates of A1 neurons are modulated when mar- colleagues demonstrated that, like tamarinsmosets listen to species-specific vocalizations (Miller et al., 2001a), rhesus macaques canbut are not modulated when these same vocali- amodally complete auditory stimuli and thatzations are played in reverse. In contrast, when neurons in the auditory cortex reflect the mon-these same stimuli are presented to cat A1 neu- keys’ behavioral responses to occluded and intactrons, A1 activity does not differentiate between stimuli (Petkov et al., 2003; Sutter et al., 2000).the forward and reverse marmoset vocalizations. Second, when presented with alternating two-This response pattern may be due, in part, to the tone sequences that are perceived by humans asfact that A1 neurons are extremely sensitive to one or two auditory streams, neurons in thethe spectrotemporal features of a stimulus: Such auditory cortex respond to these sequences in a‘‘optimal’’ stimuli elicit sustained firing pat- manner that mirrors a human subject’s psycho-terns, whereas ‘‘nonoptimal’’ stimuli elicit tran- physical percepts (Fishman et al., 2001a, 2004;sient responses (Wang et al., 2005). Together, Micheyl et al., 2005); the perception of one orthese studies suggest that at the level of the A1, two streams depends on the spectrotemporalif not earlier (Nelken et al., 2003), neurons are dynamics of the sequences. The auditory cortexintegrating the dynamic spectrotemporal may be the source of these percepts since the

246 PRIMATE NEUROETHOLOGYtemporal evolution of auditory-cortical activity computational role in object analysis, a rolefollows the temporal evolution of the percept. beyond its traditional one in feature analysis By the time information processing reaches (Nelken et al., 2003). Recent studies suggest thatnonprimary areas of the auditory cortex, this the auditory cortex may be involved in even moresensitivity to species-specific vocalizations has sophisticated types of processing. Indeed, studiesmarkedly increased. Neurophysiology of lateral of both human (Guenther et al., 2004; Poeppeland parabelt belt neurons shows an increased et al., 2004) and nonhuman primates (Seleznevaresponsiveness to vocalizations (Rauschecker & et al., 2006; Steinschneider et al., 1995)Tian, 2004; Rauschecker et al., 1995; Russ et al., indicate that the auditory cortex is also2008; Tian & Rauschecker, 2004; Tian et al., involved in the computations underlying cate-2001), whereas neuroimaging studies suggest gory processing. For example, work from Broschthat anterior regions of the temporal lobe may and colleagues (Selezneva et al., 2006) shows thatbe exquisitely tuned for vocalization processing auditory-cortex neurons respond categorically to(Petkov et al., 2008; Poremba et al., 2003). A tone sequences that either increase or decrease inrecent study quantified explicitly the capacity of frequency. The primary auditory cortex alsoneurons in the nonprimary auditory cortex to contains a distributed representation of neuralrespond differentially to different vocalizations activity that reflects the perceptual(Russ et al., 2008). One analysis demonstrated categorization of human phonemes that differ inthat the capacity of these neurons to code dif- voice-onset time (Steinschneider et al., 1995).ferent vocalizations depended on the tested time These studies suggest that the auditory system isinterval and amount of temporal integration. At readily able to represent categorical boundariesthe shortest time interval (50 ms) and the finest even in the initial processing stage of the auditorybin resolution (2 ms), a linear pattern discrimi- cortex.nator could decode %25% of the vocalizations What types of categorical processing, then,correctly. However, as the duration of the test occur in subsequent areas of the cortical hier-time interval increased from 50 ms to 750 ms, archy? We suggest that there are two majorthe performance of the discriminator improved classes of computational processing. First, neu-dramatically. Indeed, when the time interval was rons become increasingly sensitive to more750 ms, the discriminator decoded %90% of the abstract categories. For example, recent workvocalizations at the finest bin resolution. from our group suggests a role for the vPFC inInterestingly, when Russ and colleagues tested categorizing the referential information that amultiple neurons simultaneously, they found vocalization transmits, as opposed to the vocali-that the number of vocalizations that could zation’s acoustic properties (Cohen et al., 2006;be correctly coded increased exponentially as Gifford et al., 2005). Using an oddball paradigmthey increased the number of simultaneously ¨¨ ¨ (Naatanen & Tiitinen, 1996), we found thattested neurons. Subsequent analyses indicated vPFC activity was not modulated by transitionsthat these neurons were not tuned for the spectro- between presentations of food vocalizations thattemporal similarities that exist between different transmitted the same information (high-qualityvocalizations, eliminating the possibility that food) but had significantly different acousticthese neurons are tuned for spectrotemporal fea- structures. vPFC activity, however, was modu-tures. Overall, all of these studies are consistent lated by transitions between presentations ofwith the notion that neurons in the nonprimary food vocalizations that transmitted differentauditory cortex are involved in object processing, types of information (low-quality vs. high-and are not only in acoustic-feature processing. quality food). These data suggested that, on average, vPFC neurons are modulated preferentially by transitions between presentations ofNeural Correlates of Category Processing food vocalizations that belong to functionallyThe previous section highlighted the fact that the meaningful and different categories. Second,auditory cortex, particularly A1, plays a categorical representations in more central

VOCALIZATIONS AS AUDITORY OBJECTS: BEHAVIOR AND NEUROPHYSIOLOGY 247 areas like the prefrontal cortex may be used to revealed that these neurons carry significantly flexibly guide an animal’s behavior (Miller, less information about vocalization identity 2000; Miller et al., 2002). That is, categorical than those in the nonprimary auditory cortex information in the prefrontal cortex is critical (Russ et al., 2008). In addition, the pairwise for both the selection and retrieval of task- responses of neurons in the vPFC are less redun- relevant information as it relates to the rules of dant (i.e., have the capacity to code more com- an ongoing task (Asaad et al., 2000; Ashby & plex, orthogonal properties of a vocalization) Spiering, 2004; Badre et al., 2005; Bunge, 2004; than the pairwise responses of neurons in the Bunge et al., 2005; Spiering, 2004). Indeed, the superior temporal gyrus (Fig. 13.4). That is, the responses of prefrontal neurons tend to vary responses in the vPFC are capable of coding with the rules mediating a task or the behavioral more complex features than those in the non- significance of stimuli, whereas responses in the primary auditory cortex. Indeed, a post hoc inferior temporal cortex tend to be invariant to analysis of the study by Russ and colleagues these variables while tending to be better corre- (2008) indicated that vPFC neurons code more lated with the stimulus’s physical properties of the referential properties of a vocalization than prefrontal neurons (Ashby & Spiering, than those in the superior temporal gyrus (Fig. 2004; Freedman et al., 2003). In the auditory 13.5). All of these analyses are consistent with system, there is, to date, no analogous evidence. the hypothesis that the vPFC is at a hierarchically However, evidence for perhaps comparable higher level of vocalization processing (object hierarchical processing between the superior processing) than the nonprimary auditory temporal gyrus and prefrontal cortex has cortex. recently between demonstrated (Cohen et al., 2007; Russ et al., 2008). As discussed above, the CONCLUSION firing rates of neurons in nonprimary auditory cortex code several different vocalizations (Russ Vocalizations represent a unique class of stimuli et al., 2008). A similar analysis in the vPFC for studies of neural coding and representation. –0.05 redundancy –0.1 less Redundancy (bits) –0.15 redundancy –0.2 more 0.25 –0.3 0 200 400 600 800 Time Interval (msec)Figure 13.4 Redundancy between pairs of neurons in the superior temporal gyrus and between pairs ofneurons in the ventrolateral prefrontal cortex (vPFC). A redundancy analysis tested whether neurons in thesuperior temporal gyrus and the vPFC code the same attributes of a vocalization. Decreases in redundancyare associated with an increased capacity to code more complex stimulus attributes (Chechik et al., 2006;Schneidman et al., 2003). The redundancy analysis was conducted as a function of increasing time intervalsfollowing onset of a vocalization. The data in red were calculated from the responses of neurons in thesuperior temporal gyrus. The data in blue were calculated from the responses of vPFC neurons. Error barsrepresent standard deviations.

248 PRIMATE NEUROETHOLOGY Maximum distribution of values if errors are 3.0 due to categorical relationships between stimuli [assuming an unbiased estimator]. 2.5 Information (bits) 2.0 1.5 1.0 0.5 Minimum distribution of values if errors are random. 0 10 20 30 40 50 60 70 80 90 100 Percentage decoded correctly Figure 13.5 Metric content of neurons in the superior temporal gyrus and the ventrolateral prefrontal cortex (vPFC). Metric content (Treves, 1997) is calculated from the confusion matrices generated from an analysis that tested the capacity of neurons in the superior temporal gyrus and the vPFC to code vocalizations as distinct auditory objects (Russ et al., 2008). On a neuron-by-neuron basis, the metric content was calculated by correlating the percentage of correctly decoded vocalizations with the amount of information carried in a neuron’s response. The data in red were calculated from the responses of neurons in the superior temporal gyrus. The data in blue were calculated from the responses of vPFC neurons.At one level, vocalizations have evolved as highly neural systems: birdsong and primate face recog-salient communicative signals (Hauser, 1997). nition. With respect to birdsong, it is importantOn the other hand, they are auditory objects. to distinguish between the classic sensory-motorUltimately, the task at hand is to ascertain how song system (i.e., HVc, RA, area X, etc.particular acoustic features, or combinations of [Margoliash, 1997]) and the auditory systemfeatures, are transformed into higher-order (field L, NCM, etc.) of songbirds when makingrepresentations that confer information to a lis- comparisons to primate vocal communication.tener and how this information is represented in The primary difference between the song systemthe cortex. As discussed here, a vocalization is and primate vocal communication is in thenot merely a vocal signal that communicates a degree of specialization. Whereas the songsole piece of information. Rather, each call is a system represents a highly specialized neuralperceptually bound unit encoded with a rich system that almost exclusively processes ele-array of categorical content. Although tradition- ments of birdsong (Margoliash, 1997), theally, studies of vocalizations at both the beha- neural mechanisms underlying nonhuman pri-vioral and neural levels have approached this mate vocal communication are more generalizedtopic thinking primarily about the main func- (Ghazanfar, 2003). Moreover, the substratestion of the call, an object-oriented view of voca- underlying the song system are primarily sub-lizations may yield important insights. These cortical nuclei (Nottebohm et al., 1976), whereastwo perspectives are not mutually exclusive; an the primate system is largely situated in theunderstanding of how vocalizations are repre- cortex (Kaas & Hackett, 1998; Romanski et al.,sented in the cortex will likely involve studies 1999a), thereby limiting any direct parallelsfrom both approaches. between the systems. Although there may be The primate vocal communication system analogous mechanisms between primates andhas often drawn parallels with two analogous songbird song system, it is likely that these

VOCALIZATIONS AS AUDITORY OBJECTS: BEHAVIOR AND NEUROPHYSIOLOGY 249similarities reflect a homoplasy and comparisons a unique system that likely has parallels withwith other sensory-motor systems within the several related processes. But, like all complexprimate cortex will yield more homologous cortical systems, the primate communicationneural mechanisms. Some more direct parallels, system possesses intricacies that are idiosyn-however, may exist between these taxonomic cratic to its own selective pressures. With agroups with respect to the sensory processing renewed interest in this topic, significant stridesof vocalizations and other auditory objects in are likely to be made in our understanding ofthe auditory system (Gentner & Margoliash, this complex system.2003; Woolley et al., 2005). More work isneeded, however, to ascertain commonalitiesthat may exist in the auditory systems of these ACKNOWLEDGMENTStaxonomic groups. We would like to thank Brian Russ, Jung Hoon Comparisons with the face recognition Lee, and Heather Hersh for helpful commentssystem, in general, and ‘‘face cells,’’ more speci- on this manuscript. CTM was supported by afically, in nonhuman primates stems more from grant from the NIDCD-NIH (K99DC009007).potential analogies between visual and acoustic YEC was supported by grants from the Whitehallcommunication signals. Potential homologies Foundation, NIDCD-NIH, NIMH-NIH, and abetween the systems are supported by evidence Burke Award.showing an increase in processing specificity forvocalizations in the rostral areas of nonprimaryauditory cortex (Petkov et al., 2008; Poremba REFERENCESet al., 2003) and the presence of ‘‘face cells’’ in ¨ Arnold, K., & Zuberbuhler, K. (2006). Languageinferotemporal cortex (Fujita et al., 1992; Gross evolution: Semantic combinations in primateand Sergent, 1992; Perrett et al., 1984, 1985). calls. Nature, 441, 303.One caveat, however, is an evolutionary distinc- Asaad, W. F., Rainer, G., & Miller, E. K. (2000).tion between faces and vocalizations. Whereas Task-specific neural activity in the primatethe structure of primate faces (i.e., the stable prefrontal cortex. Journal of Neurophysiology,arrangement of two eyes, nose, and mouth), and 84, 451–459.most mammalian faces for that matter, has been Ashby, F. G., & Berretty, P. M. (1997).conserved over a long evolutionary history, the Categorization as a special case of decision-structure of vocalizations (i.e., spectrotemporal making or choice. In: A. A. J. Marley (Ed.), Choice, decision, and measurement: Essays inarrangement of acoustic features) varies between honor of R. Duncan Luce (pp. 367–388).all species of nonhuman primates. As such, it is Lawrence Erlbaum Associates, Inc.possible that vocalizations have not undergone Ashby, F. G., & Spiering, B. J. (2004). Thethe same type of selection for a highly specialized neurobiology of category learning. Behavioralobject like a face and are processed as more of a and Cognitive Neurosciences Review, 3, 101–113.general higher-order object, albeit a highly salient Badre, D., Poldrack, R. A., Pare-Blagoev, E. J., Insler,one. Although both systems may reflect a general R. Z., & Wagner, A. D. (2005). Dissociablepattern of increased specificity for object proces- controlled retrieval and generalized selectionsing in higher-level areas along the ventral mechanisms in ventrolateral prefrontal cortex.pathway, vocalizations may not ultimately be Neuron, 47, 907–918.processed with the same degree of specialization Barbour, D. L., & Wang, X. (2003). Contrast tuning in auditory cortex. Science, 299, 1073–1075.as faces. Bendor, D., & Wang, X. (2005). The neuronal Our focus was not to de-emphasize the representation of pitch in primate auditoryimportance of a vocalization as an evolved cortex. Nature, 436, 1161–1165.signal, but to show the possibilities that can be Bendor, D., & Wang, X. (2006). Corticalgained by approaching the study of vocalizations representations of pitch in monkeys andfrom an object-based view. Ultimately, the humans. Current Opinions in Neurobiology, 16,neural basis of primate vocal communication is 391–399.

250 PRIMATE NEUROETHOLOGYBendor, D., & Wang, X. (2007). Differential neural cortex: Evidence for hierarchical processing. In. coding of acoustic flutter within primate auditory San Diego, CA, Program No. 227.6 2007 cortex. Nature Neurosciences, 10, 763–771. Neuroscience Meeting Planner.Bergman, T. J., Beehner, J. C., Cheney, D. L., & Damasio, A. R. (1989). Time-locked multiregional Seyfarth, R. M. (2003). Hierarchical classification retroactivation: A systems-level proposal for the by rank and kinship in baboons. Science, 302, neural substrates of recall and recognition. 1234–1236. Cognition, 33, 25–62.Blank, S. C., Bird, H., Turkheimer, F., & Wise, R. J. Darwin, C. J. (1997). Auditory grouping. Trends in (2003). Speech production after stroke: The role Cognitive Science, 1, 327–333. of the right pars opercularis. Annals of De Santis, L., Clarke, S., & Murray, M. M. (2007). Neurology, 54, 310–320. Automatic and intrinsic auditory ‘‘what’’ andBlank, S. C., Scott, S. K., Murphy, K., Warburton, ‘‘where’’ processing in humans revealed by E., & Wise, R. J. (2002). Speech production: electrical neuroimaging. Cerebral Cortex, 17, Wernicke, Broca and beyond. Brain, 125, 9–17. 1829–1838. Devlin, J. T., Gonnerman, L. M., Andersen, E. S., &Boyton, R. M., & Olson, C. X. (1987). Locating Seidenberg, M. S. (1998). Category-specific basic color terms in the OSA space. Color semantic deficits in focal and widespread brain Research Applications, 12, 94–105. damage: A computational account. Journal ofBoyton, R. M., & Olson, C. X. (1990). Salience of Cognitive Neuroscience, 10, 77–94. basic color terms confirmed by three measures. Doupe, A. J., & Kuhl, P. K. (1999). Birdsong Vision Research, 30, 1311–1317. and human speech: Common themes andBregman, A. S. (1990). Auditory scene analysis. mechanisms. Annual Review of Neuroscience, Boston, MA: MIT Press. 22, 567–631.Bunge, S. A. (2004). How we use rules to select Eimas, P. D., Siqueland, E. R., Jusczyk, P., & actions: A review of evidence from cognitive Vigorito, J. (1971). Speech perception in neuroscience. Cognitive and Affective Behavior infants. Science, 171, 303–306. Neuroscience, 4, 564–579. Fabre-Thorpe, M., Richard, G., & Thorpe, S. J.Bunge, S. A., Wallis, J. D., Parker, A., Brass, M., (1998). Rapid categorization of natural images Crone, E. A., Hoshi, E., et al. (2005). Neural by rhesus monkeys. Neuroreport, 9, 303–308. circuitry underlying rule use in humans and Farris, H. E., Rand, A. S., & Ryan, M. J. (2005). The nonhuman primates. Journal of Neuroscience, effects of time, space and spectrum on auditory 25, 10347–10350. grouping in tungara frogs. Journal of ComprehensiveChechik, G., Anderson, M. J., Bar-Yosef, O., Young, Physiology Part A: Neuroethological, Sensory, Neural, E. D., Tishby, N., & Nelken, I. (2006). Reduction and Behavioral Physiology, 191, 1173–1183. of information redundancy in the ascending Feldman, J. (2003). What is a visual object? Trends auditory pathway. Neuron, 51, 359–368. in Cognitive Science, 7, 252–256.Cheney, D. L., & Seyfarth, R. M. (1980). Vocal Feldman, J., & Tremoulet, P. (2006). Individuation recognition in free ranging vervet monkeys. of visual objects over time. Cognition , 99. Animal Behaviour, 28, 362–367. Fishman, Y. I., Arezzo, J. C., & Steinschneider, M.Cheney, D. L., & Seyfarth, R. M. (1992). Meaning, (2004). Auditory stream segregation in reference, and intentionality in the natural monkey auditory cortex: Effects of frequency vocalizations of monkeys. In: T. Nishida, W. C. separation, presentation rate, and tone McGrew, P. Marler, M. Pickford, F. de Waal duration. Journal of the Acoustical Society of (Eds.), Topics in primatology, vol 1: Human America, 116, 1656–1670. origins (pp. 315–330). Tokyo: Tokyo University Fishman, Y. I., Reser, D. H., Arezzo, J. C., & Press. Steinschneider, M. (2000). Complex toneCohen, Y. E., Hauser, M. D., & Russ, B. E. (2006). processing in primary auditory cortex of the Spontaneous processing of abstract categorical awake monkey. I. Neural ensemble correlates information in the ventrolateral prefrontal of roughness. Journal of the Acoustical Society cortex. Biology Letters, 2, 261–265. of America, 108, 235–246.Cohen, Y. E., Russ, B. E., Ackelson, A. L., Baker, A. Fishman, Y. I., Reser, D. H., Arezzo, J. C., & E., Chowdhury, F. N., & Lee, Y. S. (2007). Steinschneider, M. (2001a). Neural correlates Feature and object processing in the auditory of auditory stream segregation in primary

VOCALIZATIONS AS AUDITORY OBJECTS: BEHAVIOR AND NEUROPHYSIOLOGY 251 auditory cortex of the awake monkey. Hearing functionally meaningful categories: Macaque Research, 151, 167–187. ventrolateral prefrontal cortex plays a criticalFishman, Y. I., Volkov, I. O., Noh, M. D., Garell, P. role in spontaneous categorization of species- C., Bakken, H., Arezzo, J. C., et al. (2001b). specific vocalizations. Journal of Cognitive Consonance and dissonance of musical chords: Neuroscience, 17, 1471–1482. Neural correlates in auditory cortex of monkeys Griffiths, T. D., & Warren, J. D. (2004). What is an and humans. Journal of Neurophysiology, 86, auditory object? Nature Review Neuroscience, 5, 2761–2788. 887–892.Fitch, W. T. (1997). Vocal tract length and formant Gross, C. G., & Sergent, J. (1992). Face recognition. frequency dispersion correlate with body size in Current Opinions in Neurobiology, 2, 156–161. rhesus macaques. Journal of the Acoustical Guenther, F. H., Nieto-Castanon, A., Ghosh, S. S., & Society of America, 102, 1213–1222. Tourville, J. A. (2004). Representation of soundFreedman, D. J., & Assad, J. A. (2006). Experience- categories in auditory cortical maps. Journal of dependent representation of visual categories in Speech, Language, and Hearing Research, 47, parietal cortex. Nature, 443, 85–88. 46–57.Freedman, D. J., Riesenhuber, M., Poggio, T., Hauser, M. D. (1997). The evolution of & Miller, E. K. (2001). Categorical communication. Cambridge, MA: MIT Press. representation of visual stimuli in the Hauser, M. D. (2001). Searching for food in the primate prefrontal cortex. Science, 291, wild: A nonhuman primate’s expectations 312–316. about invisible displacement. DevelopmentalFreedman, D. J., Riesenhuber, M., Poggio, T., & Science, 4, 84–93. Miller, E. K. (2002). Visual categorization and Hauser, M. D., Tsao, F., Garcia, P., & Spelke, E. S. the primate prefrontal cortex: Neurophysiology (2003). Evolutionary foundations of number: and behavior. Journal of Neurophysiology, 88, Spontaneous representation of numerical 929–941. magnitudes by cotton-top tamarins. ProceedingsFreedman, D. J., Riesenhuber, M., Poggio, T., & in Biological Sciences, 270, 1441–1446. Miller, E. K. (2003). A comparison of primate Holt, L. L. (2006). Speech categorization in context: prefrontal and inferior temporal cortices during joint effects of nonspeech and speech visual categorization. Journal of Neuroscience, precursors. Journal of the Acoustical Society of 23, 5235–5246. America, 119, 4016–4026.Fujita, I., Tanaka, K., Ito, M., & Cheng, K. (1992). Huntley-Fenner, G., Carey, S., & Solimando, A. Columns for visual features of objects in (2002). Objects are individuals but stuff doesn’t monkey inferotemporal cortex. Nature, 360, count: perceived rigidity and cohesiveness 343–346. influence infants’ representations of smallGauthier, I., & Logothetis, N. K. (1999). Is face groups of discrete entities. Cognition, 85, 203–221. recognition not so unique after all? Cognitive Kaas, J. H., & Hackett, T. A. (1998). Subdivisions of Neuropsychology. auditory cortex and levels of processing inGentner, T. Q., & Margoliash, D. (2003). Neuronal primates. Audiology Neurootology, 3, 73–85. populations and single cells representing Kaas, J. H., & Hackett, T. A. (2000). Subdivisions of learned auditory objects. Nature, 424, 669–674. auditory cortex and processing streams inGhazanfar, A. A. (2003). Primate audition: Ethology primates. Proceedings of the Nationall Academy and neurobiology. New York: CRC Press. of Sciences, 97, 11793–11799.Ghazanfar, A. A., & Hauser, M. D. (1999). The Kaffka, K. (1935). Principles of Gestalt psychology. neuroethology of primate vocal communication: New York: Harcourt, Brace, & World. Substrates for the evolution of speech. Trends in Kellman, P. J., Spelke, E. S., & Short, K. R. (1986). Cognitive Science, 3, 377–384. Infant perception of object unity fromGhazanfar, A. A., Turesson, H. K., Maier, J. X., van translatory motion in depth and vertical Dinther, R., Patterson, R. D., & Logothetis, N. translation. Child Development, 57, 72–86. K. (2007). Vocal-tract resonances as indexical Kubovy, M., & Van Valkenburg, D. (2001). Auditory cues in rhesus monkeys. Current Biology, 17, and visual objects. Cognition, 80, 97–126. 425–430. Kuhl, P. K., & Miller, J. D. (1975). Speech perceptionGifford III, G. W., MacLean, K. A., Hauser, M. D., by the chinchilla: Voiced-voiceless distinction in & Cohen, Y. E. (2005). The neurophysiology of alveolar plosive consonants. Science, 190, 69–72.

252 PRIMATE NEUROETHOLOGYKuhl, P. K., & Padden, D. M. (1982). Enhanced antiphonal calling. Animal Behaviour, 69, discriminability at the phonetic boundaries for 1387–1398. the voicing feature in macaques. Perceptions and Miller, E. K. (2000). The prefrontal cortex and Psychophysics, 32, 542–550. cognitive control. Nature Review Neuroscience,Kuhl, P. K., & Padden, D. M. (1983). Enhanced 1, 59–65. discriminability at the phonetic boundaries Miller, E. K., Freedman, D. J., & Wallis, J. D. (2002). for the place feature in macaques. Journal The prefrontal cortex: Categories, concepts, and of the Acoustical Society of America, 73, cognition. Philosophical Transactions of the Royal 1003–1010. Society of London Part B: Biological Sciences, 29,Lasky, R. E., Syrdal-Lasky, A., & Klein, R. E. (1975). 1123–1136. VOT discrimination by four to six and a half Miller, E. K., Nieder, A., Freedman, D. J., & Wallis, month old infants from Spanish environments. J. D. (2003). Neural correlates of categories and Journal of Experimental Child Psychology, 20, concepts. Current Opinions in Neurobiology, 13, 215–225. 198–203.Liberman, A. M., Cooper, F. S., Shankweiler, D. P., Miyawaki, K., Strange, W., Verbrugge, R., & Studdert-Kennedy, M. (1967). Perception Liberman, A. M., Jenkins, J. J., & Fujimura, O. of the speech code. Psychological Review, 5, (1975). An effect of linguistic experience: the 552–563. discrimination of [r] and [l] by native speakersLotto, A. J., Kluender, K. R., & Holt, L. L. (1998). of Japanese and English. Perceptions in Depolarizing the perceptual magnet effect. Psychophysiology, 18, 331–340. Journal of the Acoustical Society of America, Murray, M. M., Camen, C., Gonzalez Andino, S. L., 103, 3648–3655. Bovet, P., & Clarke, S. (2006). Rapid brainMann, V. A. (1980). Influence of preceding liquid discrimination of sounds of objects. Journal of on stop-consonant perception. Perceptions in Neuroscience, 26, 1293–1302. Psychophysiology, 28, 407–412. ¨¨ ¨ Naatanen, R., & Tiitinen, H. (1996). AuditoryMargoliash, D. (1997). Functional organization of information processing as indexed by the forebrain pathways for song production and mismatch negativity. In: M. Sabourin, F. Craik, perception. Journal of Neurobiology, 33, & M. Robert (Eds.), Advances in psychological 671–693. science: Biological and cognitive aspects (pp.Marschark, M., Convertino, C., McEvoy, C., & 145–170). Montreal: Taylor and Francis Group. Masteller, A. (2004). Organization and use of Nakayama, K. (1999). Mid-level vision. In: R. A. the mental lexicon by deaf and hearing Wilson & F. C. Keil (Eds.), The MIT encylopedia individuals. American Annals of the Deaf, 149, of the cognitive sciences. Cambridge, MA: MIT 51–61. Press.Martin, A., Ungerleider, L. G., & Haxby, J. V. (2002). Nelken, I., Fishbach, A., Las, L., Ulanovsky, N., & Category specificity and the brain: The sensory/ Farkas, D. (2003). Primary auditory cortex of motor model of semantic representations of cats: Feature detection or something else? objects. In: M. S. Gazzaniga (Ed.), The new Biological Cybernetics, 89, 397–406. cognitive neurosciences. Cambridge, MA: MIT Nieder, A., Freedman, D. J., & Miller, E. K. (2002). Press. Representation of the quantity of visual items inMicheyl, C., Tian, B., Carlyon, R. P., & Rauschecker, the primate prefrontal cortex. Science, 297, J. P. (2005). Perceptual organization of tone 1708–1711. sequences in the auditory cortex of awake Nottebohm, F., Stokes, T. M., & Leonard, C. M. macaques. Neuron, 48, 139–148. (1976). Central control of song in the canary,Miller, C. T., Dibble, E., & Hauser, M. D. (2001a). Serinus canaries. Journal of Comprehensive Amodal completion of acoustic signals by a Neurology, 165, 457–468. nonhuman primate. Nature Neurosciences, 4, Orlov, T., Yakovlev, V., Hochstein, S., & Zohary, E. 783–784. (2000). Macaque monkeys categorize images byMiller, C. T., Gil-da-Costa, R., & Hauser, M. D. their ordinal number. Nature, 404, 77–80. (2001b). Selective phonotaxis by cotton-top Perrett, D. I., Smith, P. A., Potter, D. D., Mistlin, A. tamarins. Behavior, 138, 811–826. J., Head, A. S., Milner, A. D., et al. (1984).Miller, C. T., Iguina, C., & Hauser, M. D. (2005). Neurones responsive to faces in the temporal Processing vocal signals for recognition during cortex: Studies of functional organization,

VOCALIZATIONS AS AUDITORY OBJECTS: BEHAVIOR AND NEUROPHYSIOLOGY 253 sensitivity to identity and relation to frequency representation of primary auditory perception. Human Neurobiology, 3, 197–209. cortex following discrimination training inPerrett, D. I., Smith, P. A., Potter, D. D., Mistlin, A. adult owl monkeys. Journal of Neuroscience, J., Head, A. S., Milner, A. D., et al. (1985). 13, 87–103. Visual cells in the temporal cortex sensitive to Rendall, D., Owren, M. J., Weerts, E., & Hienz, R. face view and gaze direction. Proceedings of the D. (2004). Sex differences in the acoustic Royal Society of London Part B: Biological structure of vowel-like grunt vocalizations in Sciences, 223, 293–317. baboons and their perceptual discriminationPetkov, C. I., Kayser, C., Steudel, T., Whittingstall, by baboon listeners. Journal of the Acoustical K., Augath, M., & Logothetis, N. K. (2008). A Society of America, 115, 411–421. voice region in the monkey brain. Nature Rendall, D., Rodman, P. S., & Emond, R. E. (1996). Neurosciences. Vocal recognition of individuals and kin in free-Petkov, C. I., O’Connor, K. N., & Sutter, M. L. ranging rhesus monkeys. Animal Behaviour, 51, (2003). Illusory sound perception in macaque 1007–1015. monkeys. Journal of Neuroscience, 23, 9155–9161. Roitman, J. D., Brannon, E. M., & Platt, M. L.Poeppel, D., Guillemin, A., Thompson, J., Fritz, J., (2007). Monotonic coding of numerosity in Bavelier, D., & Braun, A. R. (2004). Auditory macaque lateral intraparietal area. PLoS lexical decision, categorical perception, and FM Biology, 5, e208. direction discrimination differentially engage left Romanski, L. M., Bates, J. F., & Goldman-Rakic, P. and right auditory cortex. Neuropsychologia, 42, S. (1999a). Auditory belt and parabelt 183–200. projections to the prefrontal cortex in thePoremba, A., Saunders, R. C., Crane, A. M., Cook, rhesus monkey. Journal of Comprehensive M., Sokoloff, L., & Mishkin, M. (2003). Neurology, 403, 141–157. Functional mapping of the primate auditory Romanski, L. M., Tian, B., Fritz, J., Mishkin, M., system. Science, 299, 568–572. Goldman-Rakic, P. S., & Rauschecker, J. P.Poremba, A., Malloy, M., Saunders, R. C., Carson, (1999b). Dual streams of auditory afferents R. E., Herscovitch, P., & Mishkin, M. (2004). target multiple domains in the primate Species-specific calls evoke asymmetric activity prefrontal cortex. Nature Neurosciences, 2, in the monkey’s temporal poles. Nature, 427, 1131–1136. 448–451. Rosch, E. (1978). Principles of categorization. In: E.Pylyshyn, Z. W. (2006). Seeing and visualizing: It’s Rosch & B. B. Lloyd (Eds.), Cognition and not what you think. Cambridge, MA: MIT Press. categorization (pp. 27–48). Hillsdale, NH:Pylyshyn, Z. W., & Storm, R. W. (1988). Tracking Erlbaum. multiple independent targets: Evidence for a Rosch, E., Mervis, C. B., Gray, W. D., Johnson, D. M., parallel tracking mechanism. Spatial Vision, 3, & Boyes-Braem, P. (1976). Basic objects in natural 179–197. categories. Cognitive Psychology, 8, 382–439.Rauschecker, J. P. (1998). Cortical processing of Russ, B. E., Ackelson, A. L., Baker, A. E., & Cohen, complex sounds. Current Opinions in Y. E. (2008). Coding of auditory-stimulus Neurobiology, 8, 516–521. identity in the auditory non-spatial processingRauschecker, J. P., & Tian, B. (2000). Mechanisms stream. Journal of Neurophysiology, 99, 87–95. and streams for processing of ‘‘what’’ and Santos, L. R. (2004) ‘Core knowledges’: a ‘‘where’’ in auditory cortex. Proceedings of the dissociation between spatiotemporal knowledge Nationall Academy of Sciences, 97, 11800–11806. and contact-mechanics in a non-humanRauschecker, J. P., & Tian, B. (2004). Processing of primate? Developmental Science, 7, 167–174. band-passed noise in the lateral auditory belt Schneidman, E., Bialek, W., Berry 2nd, M. J. (2003). cortex of the rhesus monkey. Journal of Synergy, redundancy, and independence in Neurophysiology, 91, 2578–2589. population codes. Journal of Neuroscience, 23,Rauschecker, J. P., Tian, B., & Hauser, M. D. 11539–11553. (1995). Processing of complex sounds in the Scholl, B. J. (2001). Objects and attention: The state macaque nonprimary auditory cortex. Science, of the art. Cognition, 80, 1–46. 268, 111–114. Scholl, B. J. (2007). Object persistence inRecanzone, G. H., Schreiner, C. E., & philosophy and psychology. Mind & Language, Merzenich, M. M. (1993). Plasticity in the 22, 563–591.

254 PRIMATE NEUROETHOLOGYScholl, B. J., Pylyshyn, Z. W., & Feldman, J. (2001). correlates of the voice onset time boundary in What is a visual object? Evidence from target primary auditory cortex (A1) of the awake merging in multiple object tracking. Cognition, monkey: Temporal response patterns. Brain 80, 159–177. and Language, 48, 326–340.Scott, S. K. (2005). Auditory processing–speech, Streeter, L. A. (1976). Language perception of space and auditory objects. Current Opinions 2-month-old infants shows effects of both in Neurobiology, 15, 197–201. innate mechanisms and experience. Nature,Scott, S. K., Blank, C. C., Rosen, S., & Wise, R. J. 259, 39–41. (2000). Identification of a pathway for Sussman, E. S. (2004). Integration and segregation intelligible speech in the left temporal lobe. in auditory scene analysis. Journal of the Brain, 123 Pt 12, 2400–2406. Acoustical Society of America, 117, 1285–1298.Scott, S. K., Rosen, S., Wickham, L., & Wise, R. J. Sutter, M. L., Petkov, C., Baynes, K., & O’Connor, (2004). A positron emission tomography study K. N. (2000). Auditory scene analysis in of the neural basis of informational and dyslexics. Neuroreport, 11, 1967–1971. energetic masking effects in speech perception. Tian, B., & Rauschecker, J. P. (2004). Processing of Journal of the Acoustical Society of America, 115, frequency-modulated sounds in the lateral 813–821. auditory belt cortex of the rhesus monkey.Scott, S. K., & Wise, R. J. (2004). The functional Journal of Neurophysiology, 92, 2993–3013. neuroanatomy of prelexical processing in Tian, B., Reser, D., Durham, A., Kustov, A., & speech perception. Cognition, 92, 13–45. Rauschecker, J. P. (2001). FunctionalSears, C. R., & Pylyshyn, Z. W. (2000). Multiple specialization in rhesus monkey auditory object tracking and attentional processing. cortex. Science, 292, 290–293. Canadian Journal of Experimental Psychology, Toth, L. J., & Assad, J. A. (2002). Dynamic coding 54, 1–14. of behaviourally relevant stimuli in parietalSelezneva, E., Scheich, H., & Brosch, M. (2006). cortex. Nature, 415, 165–168. Dual time scales for categorical decision Treves, A. (1997). On the perceptual structure of making in auditory cortex. Current Biology, 16, face space. Biosystems, 40, 189–196. 2428–2433. Ulanovsky, N., Las, L., & Nelken, I. (2003).Sereno, A. B., & Maunsell, J. H. (1998). Shape Processing of low-probability sounds by cortical selectivity in primate lateral intraparietal neurons. Nature Neurosciences, 6, 391–398. cortex. Nature, 395, 500–503. Ungerleider, L. G., & Mishkin, M. (1982). TwoSeyfarth, R. M., Cheney, D. L., & Marler, P. (1980). cortical visual systems. In: D. J. Ingle, M. A. Monkey responses to three different alarm calls: Goodale, & R. J. W. Mansfield (Eds.), Analysis Evidence of predator classification and semantic of visual behavior. Cambridge, MA: MIT Press. communication. Science, 210, 801–803. Vogels, R. (1999). Categorization of complex visualSmith, E. E., Shoben, E. J., & Rips, L. J. (1974). images by rhesus monkeys. Part 1: Behavioural Structure and process in semantic memory: A study. European Journal of Neuroscience, 11, feature model for semantic decisions. 1223–1238. Psychological Review, 81, 214–221. Walsh, V., & Kulikowski, J. (Eds.). (1998).Snowdon, C. T., & Elowson, A. M. (1999). Perceptual constancy: Why things look as they do. Marmosets modify call structure when paired. Cambridge, UK: Cambridge University Press. Ethology, 105. Wang, X., & Kadia, S. C. (2001). DifferentialSpelke, E. (1994). Initial knowledge: six representation of species-specific primate suggestions. Cognition, 50, 431–445. vocalizations in the auditory cortices ofSpelke, E. S., Breinlinger, K., Macomber, J., & marmoset and cat. Journal of Neurophysiology, Jacobson, K. (1992). Origins of knowledge. 86, 2616–2620. Psychological Review, 99, 605–632. Wang, X., Lu, T., Snider, R. K., & Liang, L. (2005).Spelke, E. S., Katz, G., Purcell, S. E., Ehrlich, S. M., Sustained firing in auditory cortex evoked by & Breinlinger, K. (1994). Early knowledge of preferred stimuli. Nature, 435, 341–346. object motion: Continuity and inertia. Weiss, D. J., Garibaldi, B. T., & Hauser, M. D. Cognition, 51, 131–176. (2001). The production and perception ofSteinschneider, M., Schroeder, C. E., Arezzo, J. C., long calls by cotton-top tamarins (Saguinus Vaughan, H. G., Jr. (1995). Physiologic oedipus): Acoustic analyses and playback

VOCALIZATIONS AS AUDITORY OBJECTS: BEHAVIOR AND NEUROPHYSIOLOGY 255 experiments. Journal of Comprehensive Neuroscience Research Progress Bulletin, 10, Psychology, 115, 258–271. 1–119.Wise, R. J., Scott, S. K., Blank, S. C., Mummery, Wyttenbach, R. A., May, M. L., & Hoy, R. R. C. J., Murphy, K., & Warburton, E. A. (2001). (1996). Categorical perception of sound Separate neural subsystems within ‘Wernicke’s frequency by crickets. Science, 273, 1542–1544. area’. Brain, 124, 83–95. Yantis, S. (2000). Visual perception: EssentialWoolley, S. M., Fremouw, T. E., Hsu, A., & readings. Psychology Press. Theunissen, F. E. (2005). Tuning for Zatorre, R. J., Bouffard, M., & Belin, P. (2004). spectro-temporal modulations as a Sensitivity to auditory object features in mechanism for auditory discrimination of human temporal neocortex. Journal of natural sounds. Nature Neurosciences, 8, Neuroscience, 24, 3637–3642. 1371–1379. ¨ Zuberbuhler, K. (2000). Interspecies semanticWordon, F. G., & Galambos, R. (1972). Auditory communication in two forest primates. processing of biologically significant sounds. Proceedings in Biological Sciences, 267, 713–718.

CHAPTER 14Encoding and Beyond in the Motor CortexNicholas G. Hatsopoulos, Maryam Saleh, and Julian A. MattielloINTRODUCTION In this chapter, we propose that the motor cortex does not encode movement as any oneThe physiology of the primary motor cortex simple Newtonian parameter of motion. Rather,(MI) has been examined for well over a hundred we suggest that, if motor cortical neurons actu-years, beginning with the seminal electrical sti- ally encode anything at all, they represent move-mulation experiments by Fritsch and Hitzig ment fragments as Sherrington and Leyton first(1870). In the early twentieth century, Leyton proposed. We will begin by defining what weand Sherrington put forth one of the earliest mean by ‘‘encoding’’ in the nervous system.hypotheses regarding the functional role of MI We will then provide evidence against the con-based largely on focal and short-duration elec- tention that motor cortical neurons encode anytrical stimulation. By pointing to physiological simple movement parameter and argue for ourand anatomical data, they proposed that MI acts hypothesis that motor cortex may form a sub-as a synthetic organ of complex, goal-directed strate where elementary movement fragmentsmovements. Focal stimulation across the motor are assembled into motor behaviors. In the lastcortex resulted in a very large set of ‘‘fractional’’ part of the chapter, we will argue that encoding,movements that appeared coordinated but albeit an important scientific topic of inquiry, isfunctionally incomplete. They postulated that only part of a full explanation of motor corticalthe extensive anatomical connectivity within functioning. We will point out that the motorMI could serve to combine or associate cortex not only encodes information but alsomultiple fractional movements in different transforms information by processing its inputscombinations to generate a rich variety of to generate its output much like any informationfunctional behaviors. processing system. Our focus will be on the Despite this early influential viewpoint of upper limb, although many of our argumentsmotor cortical functioning, most electrophysio- can be applied more generally to the corticallogical research in awake, behaving animals over control of other segments in the motorthe past 40 years has attempted to isolate parti- apparatus.cular parameter(s) of motion such as force, velo-city, and direction that are encoded in individualmotor cortical neurons. Unfortunately, by WHAT DO WE MEAN BYfocusing on abstract movement parameters, ENCODING?almost every conceivable kinematic and kineticparameter has been shown to covary with the Before addressing the question as to whatfiring rates of motor cortical neurons. No con- the motor cortex encodes, it is essential to startsensus has emerged and has left us with a with a definition of neural encoding. In theconfused state of affairs in the field. neuroscience community, the term ‘‘neural 256

ENCODING AND BEYOND IN THE MOTOR CORTEX 257encoding’’ has been ascribed a variety of rates of single motor cortical neurons covaried,meanings. We believe that neural encoding often linearly, with joint torque, hand force, andshould be characterized as a relationship their time derivatives (Cabel et al., 2001;between some aspect of neural activity and fea- Cheney & Fetz, 1980b; Evarts, 1968; Hepp-tures of the outside world. For a sensory physiol- Reymond et al., 1978, 1999; Kalaska et al.,ogist, these features correspond to visual, 1989; Smith et al., 1975; Taira et al., 1996). Inauditory, somatosensory, and olfactory aspects addition to kinetic parameters, the firing rates ofof the world among others. For a motor physiol- MI neurons also have been shown to covary withogist, the outside world corresponds to the mus- a number of kinematic variables including staticculoskeletal system that is being controlled by position (Georgopoulos et al., 1984), dynamicthe nervous system. There are at least three position (Paninski et al., 2004a), speed (Moranspecific definitions that are either implicitly or & Schwartz, 1999), acceleration (Stark et al.,explicitly stated in studies of neural encoding. 2007b), direction (Georgopoulos et al., 1982), movement distance, or some combinations of those parameters (Ashe & Georgopoulos, 1994;Correlation/Covariation Fu et al., 1993, 1995; Kurata, 1993). There is alsoA commonly held viewpoint is that a neural strong evidence that motor cortical neuronsstructure encodes a sensory, cognitive, or carry higher-level information related to move-motor feature if the responses of the neurons ment planning (Tanji & Evarts, 1976), targetwithin the neural structure are correlated with location independent of the movement directionthe values that the feature can take. This correla- (Alexander & Crutcher, 1990; Shen & Alexander,tion can be linear or nonlinear and can be mea- 1997), or direction of action regardless of parti-sured with a variety of mathematical techniques cular muscle activity (Kakei et al., 1999).including simple cross-correlation techniques, Some investigators have argued that theregression methods, and information theoretic motor cortex multiplexes many of these para-methods (i.e., mutual information). According meters in their responses (Johnson & Ebner,to this definition, the motor cortex encodes a 2000). This would require areas receiving signalsnumber of features as documented in Table 14.1. from the motor cortex to somehow demultiplexBeginning with the first behavioral electrophy- the incoming signals (not a trivial problem).siological studies in primates by Evarts, a Others have argued that the motor cortex onlynumber of studies have indicated that the firing appears to encode all these parameters becauseTable 14.1 Motor Cortical Encoding of Movement ParametersDirection Ashe & Georgopoulos, 1994; Fu et al., 1995; Georgopoulos et al., 1982; Kakei et al., 1999; Schwartz et al., 1988; Thach, 1978Position Ashe & Georgopoulos, 1994; Georgopoulos et al., 1984; Paninski et al., 2004a; Stark et al., 2007b; Thach, 1978Speed/velocity Ashe & Georgopoulos, 1994; Churchland et al., 2006; Moran & Schwartz, 1999; Paninski et al., 2004a; Stark et al., 2007bAcceleration Ashe & Georgopoulos, 1994; Stark et al., 2007bForce/torque Cabel et al., 2001; Cheney & Fetz, 1980a; Evarts, 1968; Georgopoulos et al., 1992; Hepp-Reymond et al., 1978, 1999; Kalaska et al., 1989; Smith et al., 1975; Taira et al., 1996Electromyography/muscle Kakei et al., 1999; Morrow & Miller, 2003; Thach, 1978; Todorov, 2000 activationAmplitude/distance to target Fu et al., 1993, 1995Target position/direction Alexander & Crutcher, 1990; Fu et al., 1995; Shen & Alexander, 1997Trajectory Hatsopoulos et al., 2007; Hocherman & Wise, 1991; Paninski et al., 2004b

258 PRIMATE NEUROETHOLOGYmany of the parameters are correlated with each majority of motor cortical neurons, therefore, itother due to biological and physical constraints may be difficult to experimentally demonstrate aof motion. This is an important problem that is causal role in generating a feature of movement.somewhat unique to motor physiology and will In fact, it may be the case that for some motorbe further discussed later (see ‘‘The Problem of cortical neurons, they play no causal role towardCorrelated Parameters’’). Setting aside this pro- movement in that they only communicate theblem for the moment, we find that this defini- intentions of the motor cortex to other areas viation is too lax to be satisfying. For example, the efference copy (Nelson, 1996).firing rates of MI neurons vary between awakeand sleep states (Evarts, 1964), but no one would Invariance and Unique Specificityseriously argue that the motor cortex encodeswakefulness except in the weakest sense. A third definition of encoding, one to which weCorrelation may be a necessary but not sufficient subscribe, states that a neural structure shouldcondition for neural encoding. invariantly and uniquely specify the feature it encodes, regardless of context. Specification can be viewed as a function that maps an inputCausality space to an output space. The mapping can beA second definition of neural encoding contends many to one but not one to many. In the codonthat there must be a causal relationship between example, the triplet of bases invariantly specifiesa neural structure and what it encodes. In the or maps to a particular amino acid. This map-sensory domain, one might argue that neurons ping is degenerate in that several triplets canin the primary visual cortex (V1) encode specify a particular amino acid. However, it isoriented contrast edges because their activity unique in that a single triplet maps only to oneoccurs in response to a causal chain of events, amino acid. Likewise, the blinker invariantly andbeginning with responses in a population of uniquely specifies the directional intentions ofretinal cells, followed in time by similar the driver. This specification remains invariantresponses in the lateral geniculate nucleus cells to different road or weather conditions. Thisthat provide inputs to V1. However, causality specification, of course, serves a very importantdoes not seem to capture the essence of function as it communicates the intentions ofencoding. For example, a triplet sequence of the driver to other drivers on the road and pre-nucleotides (a codon) is considered to code for vents accidents from occurring. If the motora particular amino acid. Yet, the codon by itself cortex encodes a parameter or feature of move-does not ‘‘cause’’ the recruitment of a particular ment, then it must specify that feature in theamino acid within a larger sequence of amino same way regardless of different behavioral con-acids forming a protein. There are many other texts. These contexts can include different partsbiophysical processes that are required for this to of the arm’s workspace, different postural con-happen. As another more concrete example, the figurations, different loading conditions, dif-left blinker in a car does not cause the car to ferent task paradigms (i.e., isometric vs.move to the left. Rather, it encodes the driver’s isotonic), different ordinal positions within aintention to move to the left. complex movement sequence, and even In the context of the motor cortex, if a feature different points in time.of movement is encoded in the motor cortex, itmay be too limiting to assume that the motor THE PROBLEM OF CORRELATEDcortex ‘‘causes’’ that feature to occur. On strictly PARAMETERSanatomical grounds, it is known that only asmall percentage (perhaps 5% to 10%) of Finding consistent responses to movementlayer 5 output neurons in the motor cortex parameters in different contexts may seem tomake direct monosynaptic connections with be straightforward. However, unlike sensorymotor neurons in the spinal cord. For the vast physiology, motor physiology possesses a

ENCODING AND BEYOND IN THE MOTOR CORTEX 259 fundamental experimental problem that makes The laws of physics and biological motion it difficult to investigate parameters indepen- can also inherently couple certain movement dently: The problem is that movement para- parameters. For example, the force and accel- meters are naturally correlated. In the study of eration are linked according to Newton’s sensory systems, the experimenter has full con- second law, which implies that a neuron’s trol of the stimuli presented to the organism. In response will necessarily correlate with accel- order to isolate the sensory features that are eration if it correlates with force, assuming that encoded by a particular sensory system, the the mass is held fixed. There are also regula- experimenter can design stimuli in which mul- rities in biological motion that correlate move- tiple features are simultaneously presented in a ment parameters. For example, Fitt’s law statistically independent fashion. However, to describes the well-known speed-accuracy tra- study motor systems, the motor act is not deoff such that faster movements are generated under the experimenter’s full control but rather with less accuracy (Fitts, 1954). Likewise, the depends on the ‘‘will’’ of the organism. Under two-thirds power law relates the instantaneous many situations, multiple movement para- angular speed of a movement with the radius meters are coupled so that it becomes unclear of curvature. This is also characterized as a which variable is actually related to the activity one-third power law between the tangential of single neurons. These couplings can be an velocity and radius of curvature such that the artifact of the behavioral paradigm. For example, movements are slower at points when the cur- the center-out task pioneered by Georgopoulos vature is larger (Fig. 14.1) (Lacquaniti et al., (Georgopoulos et al., 1982) has been very useful 1983; Viviani & Cenzato, 1985; Viviani & in characterizing the directional tuning of motor Terzuolo, 1982). cortical neurons. However, this behavioral para- Finally, the isochrony principle describes the digm creates links between movement direction, observation that larger-amplitude movements speed, and position. Movements to the left, for tend to be made at faster speeds in order to instance, occur on the left side of the workspace approach a constant movement duration over so that direction and position are correlated different distances (Viviani & McCollum, 1983) (see Fig. 1A in Paninski et al., 2004a). Likewise, (Fig. 14.2). While many of these correlations can the speed of the hand tends to be lower near the be reduced or eliminated with proper controls, it center and the periphery of the workspace so is nearly impossible to uncouple all possible that speed and position are linked (see Fig. 1B movement parameters. Therefore, in any indivi- in Paninski et al., 2004a). dual study, it is essential to be careful in A B 350 103 tangential velocity (mm/s) y-position (mm) 300 250 102 200 150 101 –200 –100 0 100 10–5 100 105 x-position (mm) radius of curvature (mm)Figure 14.1 (A) The hand path during one trial (seven targets) of the random target pursuit task describedlater. The dashed circles define the local radii of curvature and the arrows indicate the local tangentialvelocity of the hand. (B) The power relationship between the local tangential velocity and radius ofcurvature is shown on a log-log scale.

260 PRIMATE NEUROETHOLOGY 120 100Movement distance (mm) 80 60 40 20 Figure 14.2 The relationship between the movement 0 distance to a target and the mean speed to the target. 0 50 100 150 200 250 300 Each point represents a movement to one target in Mean speed (mm/s) the random target pursuit task.interpreting claims that motor cortical neurons (Georgopoulos et al., 1982). A large proportionencode a specific movement parameter. of cells in primary motor cortex are tuned to the movement direction of the hand in two- and three-dimensional space. This is usually assessedENCODING OF MOVEMENT by measuring the firing rate of the cell during aPARAMETERS fixed period in a given movement trial (typically,If we accept invariant specification as our defini- the reaction time period between a go signal andtion of encoding, we are hard-pressed to identify movement initiation) and computing theany single movement parameter that is encoded average firing rate over multiple trials of ain the motor cortex. For example, force is a given movement direction. This is repeated forkinetic variable that has been considered to be each direction tested from which a tuning curveencoded in motor cortical neurons (Ashe, 1997; is plotted relating the average firing rate to theEvarts, 1968). However, context-dependent direction of movement. Various analytic func-effects have been shown to alter the relationship tions such as a cosine or a circular Gaussian arebetween firing rate and force in individual motor fit to the data from which the preferred directioncortical neurons during an isometric precision of the cell is determined by the peak value in thegrip task involving the thumb and index finger fitted function. Directional tuning is highly(Hepp-Reymond et al., 1999). These authors robust and common in the motor cortex suchfound that while the average firing rate of that no one can reasonably dispute this phenom-motor cortical neurons increased with static enon. What is in dispute, however, is whetherforce, the slope relating the firing rate with the direction measured in an extrinsic, Cartesianforce magnitude varied significantly depending coordinate system, or any coordinate systemon the task condition. That is, the slope across for that matter, is invariantly encoded in thelow- and medium-force levels differed motor cortex. Variations in initial hand posi-depending on whether a third high-force level tion, arm posture, and behavioral paradigmwas required. Interestingly, this context depen- have effects on the directional tuning curves ofdency was not observed in the digit muscle motor cortical neurons (Caminiti et al., 1990,activity. 1991; Scott & Kalaska, 1995). For example, Directional tuning is a kinematic variable Caminiti and colleagues measured the preferredthat is considered a hallmark of motor directions (PDs) of motor cortical cells in a two-cortical encoding in multijoint movements dimensional center-out task in which the animal

ENCODING AND BEYOND IN THE MOTOR CORTEX 261initiated its movement from one of three center remained relatively stable in the isometricpositions parallel to the shoulder girdle. They paradigm, the PDs exhibited large and com-found systematic shifts in the PDs such that plex shifting patterns in the movement taskthey rotated along the z-axis (Caminiti et al., reminiscent of the patterns observed by1990). More recently, Scott and Kalaska (1995) Mason and colleagues.showed that the PD shifted on average by 30 The observed temporal shifts in PD observeddegrees depending on whether the arm posture in those studies led us to examine the possibilitywas unconstrained in an adducted position in that MI neurons do not encode any simplethe sagittal plane or forced to be abducted. movement parameter but rather represent com-Interestingly, it was also observed that some plex, time-dependent trajectory fragmentscells possessed unimodal directional tuning in similar to the ‘‘fractional’’ movement hypothesisone posture but not in the other. first proposed by Leyton and Sherrington. The most striking variations in directional As previous studies had shown, we observedtuning are the temporal shifts in preferred direc- systematic shifts in PD (Fig. 14.3A, black circles)tion that have been observed by a number of that traced out ‘‘preferred trajectories’’ in two-groups (Johnson et al., 1999; Mason et al., dimensional space over the course of a trial in an1998; Sergio & Kalaska, 1998; Sergio et al., instructed delay, center-out task (Fig. 14.3B,2005). These temporal variations were measured black arrows). We sought to determine whetherby shrinking the time window that spikes were these preferred trajectory representations gener-counted and computing the preferred direction alized across a less constrained behavioral para-at multiple time windows throughout the trial. digm, which allowed the monkey to generate aUsing an instructed-delay, center-out task, rich variety of trajectories and paths in the hor-Mason and colleagues (1998) found that neu- izontal plane. This was accomplished by havingrons in the dorsal premotor cortex (PMd) the monkey reach for a sequence of randomlyshifted their PDs temporally over the whole positioned targets by moving a two-link exoske-trial by %48 degrees on average. Cells often letal robot that controlled a cursor (the randomexhibited clockwise, counterclockwise, and target pursuit or RTP task). Because the mon-other complex shifting patterns in the PDs even key’s arm was continuously moving and shiftingduring the instructed-delay period prior to its instantaneous direction of movement, wemovement. computed the PDs by comparing the instanta- These temporal shifts in PD have been neous movement direction at different time lagsobserved in the primary motor cortex and leads with respect to the firing rate of theduring a movement task as compared to an neuron (measured in 50-ms bins). Similar shiftsisometric task (Sergio & Kalaska, 1998; Sergio in PD were observed in the RTP task as wereet al., 2005). Sergio and Kalaska were parti- observed in the center-out task (Fig. 14.3A, redcularly interested in comparing the tuning and blue dots).characteristics of MI neurons within the ros- We then developed a mathematical modeltral bank of the central sulcus under iso- that explicitly characterized the preferred trajec-metric and movement versions of the tories of individual neurons and demonstratedcenter-out task. In the isometric condition, that temporally extended trajectories capturedthe monkey pushed on a fixed manipu- the tuning of motor cortical neurons more accu-landum to move a visual cursor to one of rately. We developed a generalized linear modeleight peripherally positioned targets. In the that estimated the probability of a spike emittedmovement condition, the same manipu- by a neuron (within a small spike samplinglandum was free to move, thereby moving window of 10 ms) given that the monkey’sthe colocalized cursor to the targets. A tem- hand generated a particular velocity trajectory.poral analysis of preferred direction showed a As others had shown using a differentdramatic difference between the two para- mathematical approach (Paninski et al., 2004b;digms. While the PD of most MI neurons Shoham et al., 2005), we found that an

262 PRIMATE NEUROETHOLOGY A 90 90 120 60 120 60 150 30 150 30 180 0 180 0 210 330 210 330 240 300 240 300 270 270 B C Figure 14.3 Systematic shifts in preferred direction (PD) under two behavioral paradigms. (A) Polar plots from two MI neurons depicting the shifts in PD in absolute time in the center-out task (black circles) as well as the PD shifts in relative time (lag time, red dots; lead time, blue dots) in the random target pursuit (RTP) task. Missing black circles in the left panel indicate that the PDs were not significantly fit with a cosine function. (B) By vectorally adding the shifting PDs, preferred trajectories are traced out in the center-out task (black vectors) and in the RTP task (blue and red vectors). Thin black vectors in the left panel indicate PDs that were not significantly fit with a cosine function. (C) The pathlets generated from the general linear model described later.exponential relationship between the spiking prob- neuron can be generated (Fig. 14.3C). We viewability and the inner product between the preferred ! these pathlets as a sort of movement alphabetvelocity trajectory, k , and the normalized from which more complex motor actions can bevelocity trajectory of the hand, ^t0 , fit the data: v synthesized just as language primitives are com- ! bined to form words and sentences. We havePðspikeðt0 Þj^ t0 Þ ¼ exp½ k Á^ t0 þ v v ð1Þ also demonstrated that these movement primitives remain relatively invariant in different partswhere each velocity trajectory extends over a of the workspace and under different dynamicrange of times before and after the spike sam- contexts (see Figs. 3 and 7 in Hatsopoulos et al.,pling time, to. g is an offset parameter of the 2007). Finally, we have also showed that amodel. The normalized velocity trajectory, ^ t0 , v decoding algorithm that incorporated a time-is a sequence of velocities (i.e., directions and dependent preferred trajectory for each neuronrelative speeds) over a predefined time duration. ! could be used to predict the instantaneousWe call k the preferred velocity trajectory since movement direction with less error than an algo-the inner product in the exponent is maximized rithm that assumed a static preferred directionwhen the ^t0 vector is aligned with it. By tempo- v (Hatsopoulos et al., 2007).rally integrating the x- and y-components of! Despite the fact that movement fragmentk , the preferred paths or ‘‘pathlets’’ for each encoding appears to capture the information

ENCODING AND BEYOND IN THE MOTOR CORTEX 263content of MI neurons more effectively than activity on an ensemble level may reveal aspectstime-independent parametric encoding, there of encoding to which we have been blind withare still a number of unanswered questions that single-unit recording.will require further research. It is far from clearwhether such a time-dependent representation BEYOND ENCODINGwill remain invariantly specified in single MIneurons across many different conditions and Understanding what the motor cortex encodes iscontexts. Our experiments have already indi- clearly an important endeavor in elucidating itscated that not all neurons invariantly encode a function. However, as an information proces-particular pathlet when computed under dif- sing system, encoding is only part of the story.ferent regions of the workspace (Hatsopoulos The transformation or processing of informationet al., 2007). Moreover, while a number of neu- is the complement of encoding (deCharms &rons preserve their pathlet representation when Zador, 2000). There has been very little researchviscous loads are applied to the shoulder and focused on what sort of computation the motorelbow joints, this is not true of all recorded cortex is actually performing. This requiresneurons (Hatsopoulos et al., 2007). The current understanding the nature of the inputs to motorsituation leaves us with a number of possibilities. cortex and how these inputs are transformed toFirst, encoding in MI may be inherently time generate functional outputs.dependent as we have shown, but the theorymay need to be refined to account for all possible INPUTS TO MOTOR CORTEXcontexts. Second, there may be a yet undiscov-ered time-independent movement parameter A large body of anatomical research has identi-that is invariantly specified in motor cortex. fied cortical and subcortical structures that pro-We think this is unlikely because a nearly vide direct inputs to the motor cortex includingexhaustive list of parameters has been examined the lateral premotor, the supplementary motor,over many years such that it is hard to conceive the somatosensory, and the posterior parietalof what this parameter would be. Third, the cortices (Donoghue & Sanes, 1994; Luppino &motor cortex may not encode one thing; dif- Rizzolatti, 2000; Muakkassa & Strick, 1979).ferent neurons within MI may encode different In addition, as with other neocortical structures,movement parameters. In fact, by mathemati- there are inputs from the thalamus (Holsapplecally accounting for linear correlations among et al., 1991; Olszewski, 1952; Strick, 1976) thatmovement parameters, a recent experimental transmit information from the spinal cord, thestudy has indicated that the activities of single cerebellum, and the basal ganglia (Lu et al., 2007;MI neurons are related to single movement Middleton & Strick, 1997a,b; Miyachi et al.,parameters and the responses of different neu- 2006; Sakai et al., 2002). Less is known aboutrons covary with different parameters (Stark the spatiotemporal structure of these inputset al., 2007b). It remains to be seen whether during the performance of motor actions.single neurons actually encode single movement We have been engaging in two lines ofparameters as we have defined encoding pre- research to examine structured inputs to theviously. Fourth, it may be that movement motor cortex during motor behavior. The firstencoding is not a property of any single cell but involves recording spiking activity from corticalis a global property of a neuronal ensemble. That areas such as the premotor cortex that provideis, it may be that large-scale spatiotemporal pat- major inputs to the motor cortex. By simulta-terns across the motor cortex invariantly specify neously recording neuronal ensembles fromaspects of movement even though individual both the dorsal premotor and primary motorneurons participate in an inconsistent fashion cortices, we are beginning to use multivariateto these global patterns. Although we have no autoregressive modeling to relate the outputstrong experimental evidence to support this spiking from single motor cortical neurons toview, we feel strongly that examining cortical the spiking of multiple premotor cortical

264 PRIMATE NEUROETHOLOGY IP Magnitude Lag time DP Figure 14.4 A hypothetical temporal profile of coefficients generated from an autoregressive model predicting the spike activity of a motor cortical neuron from the spiking history of a premotor cortical neuron.neurons (inputs) at multiple time lags (Wu & waves across the arm area of the motorHatsopoulos, 2006). The goal of this ongoing cortex (Rubino et al., 2006). They are particu-research is to examine the structure of the coef- larly evident during an imposed motor pre-ficients in order to determine what sort of opera- paratory period prior to movement but aretion or transformation is being performed also evident during movement execution,between the inputs and the outputs. As a albeit attenuated in amplitude. These waveshypothetical example, if the temporal profile of of input activity propagate at relatively slowthese coefficients exhibits the shape shown in speeds (10 to 20 cm/s) and typically travelFigure 14.4, this would imply that that motor along a stereotyped rostral to caudal propaga-cortex is performing a temporal derivative on its tion axis. That is, at each moment in time theinputs because it is taking the inputs from the wave is typically propagating in the caudal toimmediate past (see ‘‘IP’’ in Fig. 14.4) and sub- rostral direction or in rostral to caudal direc-tracting the inputs from the distant past (see tion. This dominant propagating axis is unaf-‘‘DP’’ in Fig. 14.4). fected by the direction of the target to be Our second line of research has involved reached by the animal and remains invariantrecording local field potentials (LFPs) in the across different behavioral paradigms (Rubinomotor cortex. These LFPs are believed to et al., 2006; Takahashi & Hatsopoulos, 2007).represent the summed voltage of hundreds tothousands of nearly synchronous, postsynapticpotentials of motor cortical neurons near the OUTPUTS FROM MOTORelectrode site and, therefore, constitute a mea- CORTEXsure of the local spatiotemporal structure of Extracellular unit recordings from the motormotor cortical inputs. These LFPs have been cortex provide a direct physiological measureshown to engage in characteristic oscillations of the outputs of the motor cortex. In particular,in the beta frequency range (20 to 30 Hz) layer 5 neurons provide the major corticofugalwithin the motor cortex during periods of pathway that forms corticospinal projections tomotor planning prior to movement execution the spinal cord and ultimately the motor neuron(Donoghue et al., 1998; O’Leary & pools that activate the muscles of the periphery.Hatsopoulos, 2006; Sanes & Donoghue, However, the meaning of these outputs can only1993), postural maintenance (Baker et al., be gleaned from the motor behaviors they affect.1999; Gilbertson et al., 2005), and heightened It is our contention that behavioral electrophy-attention and active engagement (Donoghue siology should expand its repertoire of motoret al., 1998; Murthy & Fetz, 1992, 1996a,b). behaviors from the highly constrained behaviorsWe have recently observed that these beta typically measured in the lab to ethologicallyoscillations participate in propagating planar more relevant motor actions. This may reveal

ENCODING AND BEYOND IN THE MOTOR CORTEX 265aspects of motor cortical functioning and suggested that reaching and grasping compo-encoding that have eluded previous research. nents are interdependent and tightly coordinated in both time and space. For example, a mechanical perturbation of the arm (a prox-PREHENSION imal motor component) elicits an adjustmentOne class of such unconstrained ethological not only in the arm’s position but also in thebehaviors that is currently gaining more atten- shape of the hand so as to recover and main-tion is coordinated reach and grasp, or prehen- tain a characteristic coordinated spatial tra-sion. Indeed, a mobile forelimb and an jectory between these components (Haggard,opposable thumb enabled our earliest ancestors 1991; Haggard & Wing, 1991, 1995).from the Eocene period to locomote and reach Moreover, other studies have contradictedand grasp for insects and fruits among slender Jeannerod’s earlier studies by demonstratingtree branches (Marzke, 1994). As an ethological that variations in object distance affect graspbehavior, reaching and grasping is behaviorally as well as transport components (Chieffi &relevant and provides an excellent model to Gentilucci, 1993; Jakobson & Goodale, 1991)examine how the motor cortex encodes mean- and that variations in object size can affectingful movement. transport kinematics as well as aperture size Early psychophysical studies concluded that (Marteniuk et al., 1990). Also, a number ofreaching and grasping are independent processes perturbation studies in which the object loca-that are loosely coordinated by some higher- tion or size was unexpectedly altered resultedorder central timing mechanism such as the in modifications of both the transport andcerebellum or the basal ganglia (Arbib, 1981; grasp components of movement (GentilucciHoff & Arbib, 1993; Jeannerod, 1981, 1984). et al., 1992; Paulignan et al., 1990, 1991a,b;The idea that these movements are processed Roy et al., 2006).separately stems from the observation of sepa- Anatomical and physiological studies alsorate visual (and other sensory modalities) prop- support the idea that reaching and graspingerties of the object to be grasped. Directing a are highly interdependent and may be pro-reaching movement to an object depends on cessed jointly in the cortex. Retrograde tracervisually registering the object’s relative direction studies have shown that corticospinal projec-and distance from the observer, which are tions from the motor cortex make connec-extrinsic properties of the object. On the other tions onto motoneuron pools of multiplehand, preshaping the hand and grasping the muscles involved in both distal and proximalobject depends on the object’s shape and size, arm muscle control (Shinoda et al., 1981),intrinsic features of the object. The notion of suggesting that single cortical neurons mightseparate control was seemingly confirmed by evoke synergistic activation of distal and prox-data that showed that the maximum velocity of imal muscles during prehension. These anato-the hand varied only with the object’s distance, mical observations are further supported by awhereas the maximum aperture size formed study that identified cortico-motoneuronalbetween the index finger and thumb prior to connections by means of spike-triggeredgrasp varied only with object’s size. Structural averages of electromyographic (EMG) activityfeatures of the musculoskeletal system were also of the arm and hand during prehensile move-recognized to parallel the differences between ments in nonhuman primates (McKiernanreach and grasp. Reaching is an action of axial et al., 1998). This study showed that 45% ofand proximal musculoskeletal structures, while sampled cortico-motoneuronal cells were con-grasp occurs within distal appendicular nected to motoneurons that projected to bothstructures. distal and proximal muscles of the arm. Despite the early conceptualizations of Recent stimulation studies also suggest thathow reach and grasp are distinct and dispa- there may be an overlap between neuronsrate, subsequent psychophysical studies have controlling distal and proximal arm muscles.

266 PRIMATE NEUROETHOLOGYStark and colleagues (2007a), combining Based on these psychophysical, anatomical,intracortical microstimulation (ICMS) and and physiological studies, we hypothesizesingle-unit recording from the premotor that single neurons within motor cortexcortex, found a paradoxical result. In some encode reaching and grasping movements.electrode sites, ICMS elicited distal arm Specifically, we postulate that individual neu-muscle activity of the hand. Yet in these rons encode complex, time-dependent move-same sites, they recorded single units that ment fragments (described previously) thatencoded arm direction, a movement con- include both proximal and distal componentstrolled by proximal arm musculature. of prehension. To test this hypothesis, we haveLikewise, there were other sites in which begun experiments in which monkeys areICMS activated proximal arm muscles while trained to reach and grasp for different objectsthe recorded single units from the same sites presented in the monkey’s three-dimensionalencoded the type of grasp. The authors of that reaching space with a 6-degree-of-freedomstudy argued that such neurons might help robot (Fig. 14.5A). Kinematic data are cap-coordinate distal and proximal muscles of tured in real time with high fidelity by multiplethe arm in a prehensile movement. infrared motion capture cameras (Vicon Using an altered ICMS protocol consisting of system) that track multiple spherical reflectivehigher electrical currents up to 100 mA and pulse markers placed on the arm, forearm, and fingersdurations of several hundred milliseconds, of the monkey (Fig. 14.5B).Graziano and colleagues (2002, 2004) were able As previous studies have demonstratedto elicit complex, apparently goal-directed a spatial relationship between the kinematics ofmovements involving proximal and distal the hand transport and grasp aperturejoints, which appeared to mimic naturalistic (Jeannerod, 1981, 1984; Roy et al., 2000, 2002,reaching and grasping, along with other beha- 2006), our initial goal was to verify this relation-viors. Based on these findings, they suggested ship in our system (Fig. 14.6). Figure 14.6A andthat the motor and premotor cortex initiate the C shows that although the timing of the max-coordinated activation of several muscles that imum aperture varies, its relationship withare involved in prehension and other ethologi- respect to transport distance is relatively fixed.cally relevant behaviors. This finding supports the idea that there may beFigure 14.5 (A) The recording rig used to record the motion of the arm and hand during free reachingand grasping. One of the Vicon cameras is shown in the upper right-hand corner. The 6-degree-of-freedomrobot that provides the objects to be grasped is shown in the foreground. (B) A close-up of the reflectivemarkers placed on the arm, wrist, hand, thumb, index finger, and middle finger.

ENCODING AND BEYOND IN THE MOTOR CORTEX 267Figure 14.6 The coordinated kinematics of the grip aperture and transport distance during multiplereach-to-grasp movements generated by one of our monkeys. (A) The grasp aperture size (measured asthe distance between the thumb and index finger markers) as a function of time. The cyan dots indicatewhen the maximum grasp aperture occurs. (B) The transport paths (measured from the thumb marker)in three dimensions. The green dots indicate the start of the reach, while the red dots indicate objectcontact. (C) The grasp aperture versus transport distance (measured as the integrated distance of thethumb) plots the spatial coordination pattern between the reach and grasp components.some spatial dependencies between the motor, dorsal premotor, and ventral premotormusculature that controls hand transport cortices encode not only kinematic features ofdistance—proximal arm muscles—and grasp reach and grasp but perhaps also the kinematicsaperture—distal arm muscles. Our next goal is of their coordinated action (Fig. 14.6C). We areto further characterize spatial and temporal using the mathematical framework (describeddependencies between reaching and grasping previously) by estimating the spiking probabilitymovements by investigating more complex, from movement fragment covariates extractedkinematic parameters of three-dimensional from whole reach-to-grasp behaviors towardprehension (Fig. 14.6B). We are also interested diverse objects in various locations in the mon-in testing whether single neurons within primary key’s reach space.

268 PRIMATE NEUROETHOLOGYCONCLUSION Ashe, J. (1997). Force and the motor cortex. Behavioural Brain Research, 86, 1–15.Despite being one of the earliest cortical areas to Ashe, J., & Georgopoulos, A. P. (1994). Movementbe examined functionally, the features encoded parameters and neural activity in motor cortexby the motor cortex remain elusive. We have and area 5. Cerebral Cortex, 6, 590–600.revisited an early viewpoint of motor cortical Baker, S. N., Kilner, J. M., Pinches, E. M., & Lemon, R.function proposed by Sherrington and Leyton (1999). The role of synchrony and oscillationsand have provided experimental evidence to in the motor output. Experimental Brainsupport and refine that early viewpoint. Instead Research, 128, 109–117.of encoding a static Newtonian parameter of Brecht, M., Schneider, M., Sakmann, B., & Margrie, T. W. (2004). Whisker movementsmotion, our analysis of motor cortical activity evoked by stimulation of single pyramidalsuggests that individual motor cortical neurons cells in rat motor cortex. Nature, 427, 704–710.encode temporally evolving movement frag- Cabel, D. W., Cisek, P., & Scott, S. H. (2001).ments. Although our data have not definitively Neural activity in primary motor cortexsupported a kinematic versus kinetic representa- related to mechanical loads applied to thetion, we maintain that a time-dependent repre- shoulder and elbow during a postural task.sentation is a fundamental feature of motor Journal of Neurophysiology, 86, 2102–2108.cortical encoding. Recent electrical stimulation Caminiti, R., Johnson, P. B., Galli, C., Ferraina, S., &studies seem to further support this perspective Burnod, Y. (1991). Making arm movementsby showing that stimulation (Graziano et al., within different parts of space: The premotor2002) even at a single neuron level (Brecht and motor cortical representation of a coordinate system for reaching to visual targets.et al., 2004) can elicit complex movement tra- Journal of Neuroscience, 11, 1182–1197.jectories. To further develop the idea that motor Caminiti, R., Johnson, P. B., & Urbano, A.cortex encodes complex, time-dependent move- (1990). Making arm movements withinments, we argue that examining more mean- different parts of space: Dynamic aspects iningful, neuroethological behaviors will be an the primate motor cortex. Journal ofimportant direction in which research should Neuroscience, 10, 2039–2058.head. Cheney, P. D., & Fetz, E. E. (1980a). Functional classes of primate corticomotoneuronal cells and their relation to active force. Journal ofACKNOWLEDGMENTS Neurophysiology, 44, 773–791. Cheney, P. D., & Fetz, E. E. (1980b). FunctionalWe would like to thank Krishna Shenoy, Mark classes of primate corticomotoneuronal cellsChurchland, Jake Reimer, and Kazutaka and their relation to active force. Journal ofTakahashi for helpful discussions related to this Neurophysiology, 44, 773–791.book chapter. This work was supported by a Chieffi, S., & Gentilucci, M. (1993).NIH grant R01 NS45853 from the NINDS Coordination between the transport and theawarded to NGH. grasp components during prehension movements. Experimental Brain Research, 94, 471–477.REFERENCES Churchland, M. M., Santhanam, G., & Shenoy, K. V. (2006). Preparatory activity in premotorAlexander, G. E., & Crutcher, M. D. (1990). Neural and motor cortex reflects the speed of the representations of the target (goal) of visually upcoming reach. Journal of Neurophysiology, guided arm movements in three motor areas of 96, 3130–3146. the monkey. Journal of Neurophysiology, 64, deCharms, R. C., & Zador, A. (2000). Neural 164–178. representation and the cortical code. AnnualArbib, M. (1981). Perceptual structures and Review of Neuroscience, 23, 613–647. distributed motor control. In: V. B. Brooks Donoghue, J. P., & Sanes, J. N. (1994). Motor areas (Ed.), Handbook of physiology. Baltimore: of the cerebral cortex. Journal of Clinical Williams and Wilkins. Neurophysiology, 11, 382–396.

ENCODING AND BEYOND IN THE MOTOR CORTEX 269Donoghue, J. P., Sanes, J. N., Hatsopoulos, N. G., & state is favored at the expense of new movement Gaal, G. (1998). Neural discharge and local field during 13-35 Hz oscillatory synchrony in the potential oscillations in primate motor cortex human corticospinal system. Journal of during voluntary movements. Journal of Neuroscience, 25, 7771–7779. Neurophysiology, 77, 159–173. Graziano, M. S., Cooke, D. F., Taylor, C. S., &Evarts, E. V. (1964). Temporal patterns of Moore, T. (2004). Distribution of hand location discharge of pyramidal tract neurons during in monkeys during spontaneous behavior. sleep and waking in the monkey. Journal of Experimental Brain Research, 155, 30–36. Neurophysiology, 27, 152–171. Graziano, M. S., Taylor, C. S., & Moore, T. (2002).Evarts, E. V. (1968). Relation of pyramidal tract Complex movements evoked by activity to force exerted during voluntary microstimulation of precentral cortex. Neuron, movement. Journal of Neurophysiology, 31, 34, 841–851. 14–27. Haggard, P. (1991). Task coordination in humanFitts, P. M. (1954). The information capacity of the prehension. Journal of Motor Behavior, 23, 25–37. human motor system in controlling the Haggard, P., & Wing, A. (1995). Coordinated amplitude of movement. Journal of responses following mechanical perturbation Experimental Psychology, 47, 381–391. of the arm during prehension. ExperimentalFritsch, G., & Hitzig, E. (1870). Uber die elektrische Brain Research, 102, 483–494. Erregbarkeit des Grosshirns. Archives fur Haggard, P., & Wing, A. M. (1991). Remote Anatomie, Physiologie, und Wissenschaftliche responses to perturbation in human Medizin, 300–332. prehension. Neuroscience Letters, 122, 103–108.Fu, Q.-G., Flament, D., Coltz, J. D., & Ebner, T. J. Hatsopoulos, N. G., Xu, Q., & Amit, Y. (2007). (1995). Temporal encoding of movement Encoding of movement fragments in the kinematics in the discharge of primate primary motor cortex. Journal of Neuroscience, 27, motor and premotor neurons. Journal of 5105–5114. Neurophysiology, 73, 836–854. Hepp-Reymond, M., M. K.-T., Gabernet, L., H.X.,Fu, Q.-G., Suarez, J. I., & Ebner, T. J. (1993). Q., & Weber, B. (1999). Context-dependent Neuronal specification of direction and force coding in motor and premotor cortical distance during reaching movements in the areas. Experimental Brain Research, 128, superior precentral premotor area and primary 123–133. motor cortex of monkeys. Journal of Hepp-Reymond, M.-C., Wyss, U. R., & Anner, R. Neurophysiology, 70, 2097–2116. (1978). Neuronal coding of static force in theGentilucci, M., Chieffi, S., Scarpa, M., & Castiello, U. primate motor cortex. Journal of Physiology, (1992). Temporal coupling between transport Paris, 74, 287–291. and grasp components during prehension Hocherman, S., & Wise, S. P. (1991). Effects of movements: Effects of visual perturbation. hand movement path on motor cortical Behavioral Brain Research, 47, 71–82. activity in awake, behaving rhesus monkeys.Georgopoulos, A. P., Ashe, J., Smyrnis, N., & Taira, Experimental Brain Research, 83, 285–302. M. (1992). The motor cortex and the coding of Hoff, B., & Arbib, M. A. (1993). Models of force. Science, 256, 1692–1695. Trajectory Formation and TemporalGeorgopoulos, A. P., Caminiti, R., & Kalaska, J. F. Interaction of Reach and Grasp. Journal of (1984). Static spatial effects in motor cortex and Motor Behavior, 25, 175–192. area 5: Quantitative relations in a two- Holsapple, J. W., Preston, J. B., & Strick, P. L. dimensional space. Experimental Brain (1991). The origin of thalamic inputs to Research, 54, 446–454. the ‘‘hand’’ representation in the primaryGeorgopoulos, A. P., Kalaska, J. F., Caminiti, R., & motor cortex. Journal of Neuroscience, 11, Massey, J. T. (1982). On the relations between 2644–2654. the direction of two-dimensional arm Jakobson, L. S., & Goodale, M. A. (1991). Factors movements and cell discharge in primate affecting higher-order movement planning: motor cortex. Journal of Neuroscience, 2, A kinematic analysis of human prehension. 1527–1537. Experimental Brain Research, 86, 199–208.Gilbertson, T., Lalo, E., Doyle, L., Di Lazzaro, V., Jeannerod, M. (1981). Intersegmental coordination Cioni, B., & Brown, P. (2005). Existing motor during reaching at natural visual objects. In:

270 PRIMATE NEUROETHOLOGY J. Long & A. Baddeley (Eds.), Attention and of the discharge of dorsal premotor cortical performance IX (pp. 153–168). Hillsdale, cells. Neuroreport, 9, 989–995. NJ: Lawrence Erlbaum Associates Publishers. McKiernan, B. J., Marcario, J. K., Karrer, J. H., &Jeannerod, M. (1984). The timing of natural Cheney, P. D. (1998). Corticomotoneuronal prehension movements. Journal of Motor postspike effects in shoulder, elbow, wrist, Behavior, 16, 235–254. digit, and intrinsic hand muscles during aJohnson, M. T., Coltz, J. D., Hagen, M. C., & Ebner, reach and prehension task. Journal of T. J. (1999). Visuomotor processing as reflected Neurophysiology, 80, 1961–1980. in the directional discharge of premotor and Middleton, F. A., & Strick, P. L. (1997a). Cerebellar primary motor cortex neurons. Journal of output channels. International Review of Neurophysiology, 81, 875–894. Neurobiology, 41, 61–82.Johnson, M. T., & Ebner, T. J. (2000). Processing of Middleton, F. A., & Strick, P. L. (1997b). New multiple kinematic signals in the cerebellum concepts about the organization of basal ganglia and motor cortices. Brain Research Reviews, 33, output. Advance in Neurology, 74, 57–68. 155–168. Miyachi, S., Lu, X., Imanishi, M., Sawada, K.,Kakei, S., Hoffman, D. S., & Strick, P. L. (1999). Nambu, A., & Takada, M. (2006). Muscle and movement representations in the Somatotopically arranged inputs from primary motor cortex. Science, 285, putamen and subthalamic nucleus to primary 2136–2139. motor cortex. Neuroscience Research, 56,Kalaska, J. F., Cohen, D. A. D., Hyde, M. L., & 300–308. Prud’homme, M. (1989). A comparison of Moran, D. W., & Schwartz, A. B. (1999). Motor movement direction-related versus load cortical representation of speed and direction direction-related activity in primate motor during reaching. Journal of Neurophysiology, 82, cortex, using a two-dimensional reaching 2676–2692. task. Journal of Neuroscience, 9, 2080–2102. Morrow, M. M., & Miller, L. E. (2003). PredictionKurata, K. (1993). Premotor cortex of monkeys: of muscle activity by populations of sequentially Set- and movement-related activity reflecting recorded primary motor cortex neurons. amplitude and direction of wrist movements. Journal of Neurophysiology, 89, 2279–2288. Journal of Neurophysiology, 77, 1195–1212. Muakkassa, K. F., & Strick, P. L. (1979). FrontalLacquaniti, F., Terzuolo, C., & Viviani, P. (1983). lobe inputs to primate motor cortex: Evidence The law relating the kinematic and figural for four somatotopically organized ‘premotor’ aspects of drawing movements. Acta areas. Brain Research, 177, 176–182. Psychologica, 54, 115–130. Murthy, V. N., & Fetz, E. E. (1992). CoherentLu, X., Miyachi, S., Ito, Y., Nambu, A., & Takada, 25- to 35-Hz oscillations in the sensorimotor M. (2007). Topographic distribution of output cortex of awake behaving monkeys. neurons in cerebellar nuclei and cortex to Proceedings of the National Academy of somatotopic map of primary motor cortex. Sciences, 89, 5670–5674. European Journal of Neuroscience, 25, Murthy, V. N., & Fetz, E. E. (1996a). Oscillatory 2374–2382. activity in sensorimotor cortex of awakeLuppino, G., & Rizzolatti, G. (2000). The monkeys: Synchronization of local field organization of the frontal motor cortex. potentials and relation to behavior. Journal of News in Physiological Science, 15, 219–224. Neurophysiology, 76, 3949–3967.Marteniuk, R. G., Leavitt, J. L., MacKenzie, C. L., Murthy, V. N., & Fetz, E. E. (1996b). Synchronization & Athenes, S. (1990). Functional relationships of neurons during local field potential oscillations between grasp and transport components in a in sensorimotor cortex of awake monkeys. Journal prehension task. Human Movement Science, 9, of Neurophysiology, 76, 3968–3982. 149–176. Nelson, R. J. (1996). Interactions between motorMarzke, M. (1994). Evolution. In: K. Bennett (Ed.), commands and somatic perception in Insights into the reach to grasp movement. sensorimotor cortex. Current Opinions in Amsterdam: Elsevier. Neurobiology, 6, 801–810.Mason, C. R., Johnson, M. T., Fu, Q. G., O’Leary, J. G., & Hatsopoulos, N. G. (2006). Early Gomez, J. E., & Ebner, T. J. (1998). visuomotor representations revealed from Temporal profile of the directional tuning evoked local field potentials in motor and

ENCODING AND BEYOND IN THE MOTOR CORTEX 271 premotor cortical areas. Journal of cortex during voluntary movement. Proceedings Neurophysiology, 96, 1492–1506. of the. National Academy of Sciences, 90,Olszewski, J. (1952). The thalamus of the macaca 4470–4474. mulatta. Basel: Karger. Schwartz, A. B., Kettner, R. E., & Georgopoulos,Paninski, L., Fellows, M. R., Hatsopoulos, N. G., & A. P. (1988). Primate motor cortex and free arm Donoghue, J. P. (2004a). Spatiotemporal tuning movements to visual targets in three- of motor cortical neurons for hand position and dimensional space. 1. Relations between single velocity. Journal of Neurophysiology, 91, cell discharge and direction of movement. 515–532. Journal of Neuroscience, 8, 2913–2927.Paninski, L., Fellows, M. R., Shoham, S., Scott, S. H., & Kalaska, J. F. (1995). Changes in Hatsopoulos, N. G., & Donoghue, J. P. motor cortex activity during reaching (2004b). Superlinear population encoding of movements with similar hand paths but dynamic hand trajectory in primary motor different arm postures. Journal of cortex. Journal of Neuroscience, 24, 8551–8561. Neurophysiology, 73, 2563–2567.Paulignan, Y., Jeannerod, M., MacKenzie, C., & Sergio, L. E., Hamel-Paquet, C., & Kalaska, J. F. Marteniuk, R. (1991a). Selective perturbation (2005). Motor cortex neural correlates of of visual input during prehension movements. output kinematics and kinetics during 2. The effects of changing object size. isometric-force and arm-reaching tasks. Experimental Brain Research, 87, 407–420. Journal of Neurophysiology, 94, 2353–2378.Paulignan, Y., MacKenzie, C., Marteniuk, R., & Sergio, L. E., & Kalaska, J. F. (1998). Changes in the Jeannerod, M. (1990). The coupling of arm temporal pattern of primary motor cortex and finger movements during prehension. activity in a directional isometric force versus Experimental Brain Research, 79, 431–435. limb movement task. Journal ofPaulignan, Y., MacKenzie, C., Marteniuk, R., & Neurophysiology, 80, 1577–1583. Jeannerod, M. (1991b). Selective perturbation Shen, L., & Alexander, G. E. (1997). Preferential of visual input during prehension movements. representation of instructed target location 1. The effects of changing object position. versus limb trajectory in dorsal premotor Experimental Brain Research, 83, 502–512. cortex. Journal of Neurophysiology, 77,Roy, A. C., Paulignan, Y., Farne, A., Jouffrais, C., & 1195–1212. Boussaoud, D. (2000). Hand kinematics during Shinoda, Y., Yokota, J., & Futami, T. (1981). reaching and grasping in the macaque monkey. Divergent projection of individual Behavioral Brain Research, 117, 75–82. corticospinal axons to motoneurons ofRoy, A. C., Paulignan, Y., Meunier, M., & multiple muscles in the monkey. Neuroscience Boussaoud, D. (2002). Prehension movements Letters, 23, 7–12. in the macaque monkey: Effects of object size Shoham, S., Paninski, L. M., Fellows, M. R., and location. Journal of Neurophysiology, 88, Hatsopoulos, N. G., Donoghue, J. P., & 1491–1499. Normann, R. A. (2005). Statistical encodingRoy, A. C., Paulignan, Y., Meunier, M., & model for a primary motor cortical brain- Boussaoud, D. (2006). Prehension movements machine interface. IEEE Transactions on in the macaque monkey: Effects of perturbation Biomedical Engineering, 52, 1312–1322. of object size and location. Experimental Brain Smith, A. M., Hepp-Reymond, M. C., & Wyss, Research, 169, 182–193. U. R. (1975). Relation of activity in precentralRubino, D., Robbins, K. A., & Hatsopoulos, N. G. cortical neurons to force and rate of force (2006). Propagating waves mediate information change during isometric contractions of finger transfer in the motor cortex. Nature muscles. Experimental Brain Research, 23, Neuroscience, 9, 1549–1557. 315–332.Sakai, S. T., Inase, M., & Tanji, J. (2002). The Stark, E., Asher, I., & Abeles, M. (2007a). relationship between MI and SMA afferents Encoding of reach and grasp by single and cerebellar and pallidal efferents in the neurons in premotor cortex is independent macaque monkey. Somatosensory and Motor of recording site. Journal of Neurophysiology, Research, 19, 139–148. 97, 3351–3364.Sanes, J. N., & Donoghue, J. P. (1993). Oscillations Stark, E., Drori, R., Asher, I., Ben-Shaul, Y., & in local field potentials of the primate motor Abeles, M. (2007b). Distinct movement

272 PRIMATE NEUROETHOLOGY parameters are represented by different neurons activity, joint position, and direction of in the motor cortex. European Journal of intended next movement in motor cortex and Neuroscience, 26, 1055–1066. cerebellum. Journal of Neurophysiology, 41,Strick, P. L. (1976). Anatomical analysis of 654–676. ventrolateral thalamic input to primate motor Todorov, E. (2000). Direct cortical control of cortex. Journal of Neurophysiology, 39, 1020–1031. muscle activation in voluntary armTaira, M., Boline, J., Smyrnis, N., Georgopoulos, movements: A model. Nature Neuroscience, 3, A. P., & Ashe, J. (1996). On the relations 391–398. between single cell activity in the motor cortex Viviani, P., & Cenzato, M. (1985). Segmentation and the direction and magnitude of three- and coupling in complex movements. Journal of dimensional static isometric force. Experimental Psychology: Human Perception and Experimental Brain Research, 109, 367–376. Performance, 11, 828–845.Takahashi, K., & Hatsopoulos, N. G. (2007). Viviani, P., & McCollum, G. (1983). The Copropagating waves of local field potentials relation between linear extent and velocity and single-unit spiking in motor cortex. In: in drawing movements. Neuroscience, 10, 37th Annual Meeting of the Society for 211–218. Neuroscience, San Diego, CA. Viviani, P., & Terzuolo, C. (1982). TrajectoryTanji, J., & Evarts, E. V. (1976). Anticipatory determines movement dynamics. Neuroscience, activity of motor cortex neurons in relation to 7, 431–437. direction of an intended movement. Journal of Wu, W., & Hatsopoulos, N. (2006). Evaluating Neurophysiology, 39, 1062–1068. temporal relations between motor andThach, W. T. (1978). Correlation of neural premotor cortex. In: 36th Annual Meeting of discharge with pattern and force of muscular the Society for Neuroscience, Atlanta, GA.

CHAPTER 15Looking at Sounds: Neural Mechanismsin the Primate BrainJennifer M. Groh and Dinesh K. PaiWhen you hear a salient sound, it is natural to relationship to each other depending on thelook at it to find out what is happening. orbital position of the eyes (Fig. 15.1). This dis-Orienting the eyes to look at sounds is essential crepancy in reference frame should be resolvedto our ability to identify and understand the prior to or as part of the convergence of visualevents occurring in our environment. This beha- and auditory signals onto a common oculo-vior involves both sensorimotor and multisen- motor pathway.sory integration: A sound elicits a movement of A second computational hurdle is that visualthe visual sense organ, the eye, to bring the and auditory signals are not necessarily encodedsource of the sound under visual scrutiny. How in the same format. From the retina on, neuronsare auditory signals converted into oculomotor in the early visual pathway have receptive fieldscommands? This chapter describes our recent that tile the visual scene and produce a ‘‘placework concerning the necessary computational code’’ for stimulus location (Fig. 15.2). In con-steps between sound and eye movement, and trast, the binaural computations performed inhow they may be implemented in neural popu- the auditory pathway might or might not pro-lations in the primate brain. duce receptive fields. If they do not, then there In principle, the brain must determine the may be a discrepancy in the coding format oflocation of the sound, encode that location in a visual and auditory signals.reference frame and format that allows for con- Ultimately, either visual or auditory or bothvergence with visual signals onto a common signals must undergo a transformation into amotor pathway, and create a suitable time- reference frame and a coding format that arevarying signal in the extraocular muscles to similar to each other and appropriate for acces-move the eyes. In practice, it is not clear exactly sing the oculomotor pathway. We will begin byhow these computations unfold. Several specific describing the evidence concerning the referencehurdles must be overcome. First, auditory and frame of auditory signals as they progress fromvisual signals arise in different reference frames. auditory to multimodal and oculomotor areasBinaural and spectral cues provide information before turning to coding format and some com-about where a sound is located, but only with putational analyses that shed light on the neuralrespect to the head and ears, not the eyes. In algorithms that may be at play in this process.contrast, visual information is intrinsically eyecentered: The pattern of illumination of theretina depends on the locations of objects in REFERENCE FRAMEthe visual scene with respect to the direction of The earliest area along the auditory pathwaygaze. These two reference frames vary in their where the reference frame of auditory signals 273

274 PRIMATE NEUROETHOLOGY Head-centered neuron Eye fixation: Left Center Right Eye-centered neuron Hybrid Activity Head-centered Eye-centered target location target locationFigure 15.1 Head-centered receptive fields are fixed in space defined with respect to the head. Eye-centered receptive fields are fixed with respect to the eyes. These reference frames shift with respect to eachother when the eyes move with respect to the head. Head- and eye-centered reference frames can thereforebe distinguished by evaluating the discharge patterns of individual neurons as a function of head- and eye-centered target location across different fixation positions. ‘‘Hybrid’’ response patterns are defined as thosethat are not well aligned in either head- or eye-centered coordinates. A Simulated place code B Simulated rate code Discharge rate Stimulus location (deg.) Stimulus location (deg.)Figure 15.2 Place codes for space contain neurons with nonmonotonic (peaked) response functions, suchas circumscribed receptive fields, whereas rate codes contain neurons with monotonic location sensitivity.has been investigated is the inferior colliculus the superior colliculus (SC) (for review, see(IC). The IC is part of the ascending auditory Sparks & Hartwich-Young, 1989b), and thuspathway, receiving input from the superior oli- could play a specific role in the control of eyevary complex and projecting to the auditory movements to sound sources.thalamus (medial geniculate body) (Moore, Originally, it was thought that the IC encodes1991; Nieuwenhuys, 1984; Oliver, 2000). The sound location in a head-centered referenceIC also projects to an oculomotor structure, frame (Jay & Sparks, 1987a). We tested this

LOOKING AT SOUNDS: NEURAL MECHANISMS IN THE PRIMATE BRAIN 275hypothesis by investigating the responses of IC functions of IC neurons should shift when theneurons to sounds as a function of eye position eyes move, and by the same amount that the eyes(Fig. 15.3). If IC neurons represent sound loca- move (e.g., Fig. 15.1). We found that eye posi-tion in a head-centered reference frame, then eye tion affects the responses of about 40% of ICposition should have no impact on neural neurons (Groh et al., 2001; see also Porter &responses. In contrast, if the IC uses an eye- Groh, 2006, and Zwiers et al., 2004). However,centered reference frame, the spatial response we did not find an eye-centered representation: A B 22 22 20 Fixation: 20 Left 18 Center 18 Discharge rate (Hz) Right 16 16 14 14 12 12 10 10 8 8 6 lmic062 6 4 4 –24 –12 0 12 24 –24 –12 0 12 24 Head-centered target location (deg) Eye-centered target location (deg) C D 1 20 contra Discharge rate (Hz) correlation coefficient Eye position (deg) 0.5 Head-centered 0 –0.5 Eye > Head ipsi lmic062 6 Eye ~ Head Eye < Head –1 ipsi contra –1 –0.5 0 0.5 1 Sound location (deg) Eye-centered correlation coefficientFigure 15.3 (A, B) Responses of example inferior colliculus (IC) neuron as a function of head- and eye-centered target location for three different fixation positions. Response functions do not align perfectly ineither reference frame. (C) Activity of the same neuron in color as a function of all the eye positions andsound locations that were tested. The triangles indicate the fixation positions corresponding to the data inpanels a and b. (D) Population plot showing that the reference frame in the population of IC neuronsspanned a continuum from more eye centered to more head centered, with most neurons lying betweenthese two canonical extremes. A correlation coefficient between each neuron’s response functions in head-vs. eye-centered coordinates was calculated and plotted on this graph; crosses indicate 95% confidenceintervals. Neurons were classed as eye > head (red) or head > eye (green) only if the confidence intervalsshow that the eye-centered correlation was greater than the head-centered correlation or vice versa. SeeResearch Design for details. From Porter, K. K., Metzger, R. R., & Groh, J. M. (2006). Representation of eyeposition in primate inferior colliculus. Journal of Neurophysiology, 95, 1826–1842. Used with permission.

276 PRIMATE NEUROETHOLOGYThe effect of eye position, while statistically sig- that visual neurons were nearly as likely to havenificant, interacted with the auditory response head-centered as eye-centered receptive fieldsbut did not cause systematic shifts related to the (Fig. 15.4c). Across the population, the distribu-change in eye position (Fig. 15.3d). Overall, the tion of response patterns spanned a continuumrepresentation reflected a hybrid of head- and from predominantly eye centered to predomi-eye-centered information. nantly head centered, with hybrid reference The presence of this hybrid reference frame led frames being the most common response pat-us to investigate the reference frame at several tern (Mullette-Gillman et al., 2005).later stages of processing: auditory cortex and In keeping with our results in the IC andthe intraparietal sulcus. The motivation behind auditory cortex, we found that the auditorythese studies was to see if the hybrid representa- signals in the parietal cortex reflected a mixturetion in the IC was ultimately converted into a of head- and eye-centered sensitivity (Fig.more eye-centered representation at a later 15.4b). A quantitative examination of the refer-stage. Auditory cortex and the intraparietal ence frame of across the IC, auditory cortex,sulcus are not only situated later in the processing and parietal cortex showed that there was littlestream but also provide direct input to oculo- difference between the auditory signals in thesemotor structures, in particular the SC (for structures. There was a small but statisticallyreview, see Sparks & Hartwich-Young, 1989a). significant difference between the visual and In core auditory cortex, the representation auditory reference frame within the parietalwas similar to that of the IC (Fig. 15.4a) cortex, suggesting that even though visual and(Werner-Reiss et al., 2003). Approximately one auditory signals converge onto a commonthird of individual neurons showed a statistically neural population (and, in some cases, ontosignificant influence of eye position on their individual bimodal neurons), there remains aresponses. Across the population, including all slight discrepancy between how visual andneurons regardless of whether they showed a auditory information are encoded.statistically significant effect of eye position, the After parietal cortex, visual and auditory sig-spatial sensitivity patterns of the majority of nals pass through the SC prior to reaching theneurons reflected a hybrid of head- and eye- eye muscles. The SC is thought to contain a placecentered information. code for the eye-centered saccade vector, and the The lateral and medial banks of the intrapar- same saccade-related burst neurons are thoughtietal sulcus (lateral intraparietal [LIP] and medial to control visual, auditory, and somatosensoryintraparietal [MIP] areas) contain both visual and saccades (Groh & Sparks, 1992, 1996a,b,c; Jay &auditory neurons. It had been assumed that the Sparks, 1984, 1987a,b; Klier et al., 2001;representation of visual information is generally Meredith & Stein, 1983, 1996; Populin et al.,eye centered, with an eye position gain-modula- 2004; Robinson, 1972; Schiller & Stryker, 1972;tion affecting the response magnitude but not the Sparks, 1978; Stein & Meredith, 1993; Stein et al.,location of the receptive fields (e.g., Andersen & 1993). However, there are some very puzzlingMountcastle, 1983; Andersen & Zipser, 1988; aspects to the current story regarding the SC,Andersen et al., 1985; Zipser & Andersen, 1988), which call into question some of thesebut this view requires a demonstration that the assumptions.receptive field location does not change with eye In particular, Jay and Sparks investigated theposition, and systematic mapping of receptive reference frame of both visual and auditory sen-field locations for each different fixation position1 sory responses in this structure in primates andhad not previously been conducted. reported a discrepancy in reference frame: Visual Accordingly, we mapped the visual receptive signals were predominantly eye centeredfields in the LIP and MIP areas at multiple fixa- whereas auditory signals were intermediatetion positions (Mullette-Gillman et al., 2005). between head- and eye-centered coordinatesOur results did not support the interpretation (Fig. 15.5) (Jay & Sparks, 1984, 1987a,b).of largely eye-centered representation: We found Similar results have been reported in the cat SC

LOOKING AT SOUNDS: NEURAL MECHANISMS IN THE PRIMATE BRAIN 277 Core auditory cortex (A1) B LIP/MIP: Auditory responses A 1 1 0.5 0.5 0 0 –0.5 –0.5 Eye > Head Eye ~ Head N=52 Eye < Head N=113 –1 –1 –1 –0.5 0 0.5 1 –1 –0.5 0 0.5 1 C LIP/MIP: Visual responses 1 Head-centered corr. coeff. 0.5 0 –0.5 N=125 –1 –1 –0.5 0 0.5 1 Eye-centered corr. coeff.Figure 15.4 Reference frame results for auditory and parietal cortex. In A1 (A), lateral intraparietal (LIP),and medial intraparietal (MIP) (B, C) areas, the observed reference frames span a continuum from head- toeye-centered coordinates for both auditory (B) and (in LIP/MIP) visual signals (C). From Mullette-Gillman,O. A., Cohen, Y. E., & Groh, J. M. (2005). Eye-centered, head-centered, and complex coding of visual andauditory targets in the intraparietal sulcus. Journal of Neurophysiology, 94, 2331–2352; and Werner-ReissmU., & Groh, J. M. (2008). A rate code for sound azimuth in monkey auditory cortex: implications for humanneuroimaging studies. Journal of Neuroscience, 28(14), 3747–3758. Used with permission.as well (Hartline et al., 1995; Peck et al., 1995; Stein & Meredith, 1993), these studies have notPopulin et al., 2004; Zella et al., 2001). Jay and addressed the effects of eye position and haveSparks did not investigate the alignment generally evaluated the receptive fields in a qua-between the visual and auditory receptive fields litative fashion. Quantitative data on the loca-of bimodal neurons, but the implication of their tions, shape, and alignment of the receptivereference frame finding is that these receptive fields as a function of eye position at the singlefields cannot maintain perfect alignment across neuron and population levels are needed.different initial eye positions. Although there If the visual and auditory receptive fields of SChave been numerous investigations of the neurons are not aligned, and if these neuronsresponse properties of SC neurons to visual, control saccadic eye movements, then oneauditory, and combined modality stimuli, sug- would expect a signature of this misalignment ingesting that visual and auditory receptive fields the accuracy of saccades to sounds across dif-overlap (e.g., Wallace et al., 1996; for review see ferent initial eye positions. Specifically, saccades

278 PRIMATE NEUROETHOLOGY A B 30 Fixation position: 30 Left Center Number of spikes Right 20 20 10 10 0 0 –30 –10 10 30 50 70 –30 –10 10 30 50 70 Head-centered target position (degrees) Eye-centered target position (degrees) C 15 10 AUDITORY Number of cells 5 0 5 VISUAL 10 15 0 8 16 24 32 40 Horizontal shiftFigure 15.5 Results of Jay and Sparks’s investigation of visual and auditory reference frame in thesuperior colliculus. (A, B) Responses of an auditory neuron as a function of head-centered and eye-centeredtarget position: The receptive field shifts with eye position but not by the exact amount of the change in eyeposition. (C) On average, auditory receptive fields shift about half as much (arrow) as would be needed tomaintain a constant position in eye-centered coordinates (dashed line). Visual receptive fields tend to shiftthe full amount, suggesting a lack of registry between visual and auditory tuning. From Jay, M. F., & Sparks,D. L. (1987a). Sensorimotor integration in the primate superior colliculus. II. Coordinates of auditorysignals. Journal of Neurophysiology, 57, 35–55. Used with permission.to a given target location might be more or less eye position on saccade endpoint, although theaffected by initial eye position depending on the auditory saccades were more variable. This sug-modality of the target. Assuming visual signals are gests that, ultimately, the saccade command doesthe ‘‘correct’’ ones, then saccades to visual targets not depend very strongly on whether the targetshould compensate completely for initial eye was visual or auditory, implying that visual andposition but saccades to auditory targets should auditory signals do end up in a commonshow a characteristic pattern of errors suggestive representation.of a failure to complete a coordinate transforma- It is currently uncertain how this could betion from head- to eye-centered coordinates. accomplished. One potential explanation is that We looked for such an effect and did not find visual and auditory signals might be initiallyone (Fig. 15.6) (Metzger et al., 2004; see also Peck misaligned, at the time of the sensory stimulus,et al., 1995, and Populin et al., 2004). Instead, we but come into alignment prior to the initiationfound that visual and auditory saccades were of the movement. If this is the case, then thegenerally very similar to each other (Fig. 15.6). visual and auditory saccade-related bursts, asBoth showed only a very modest effect of initial opposed to the sensory responses studied by

LOOKING AT SOUNDS: NEURAL MECHANISMS IN THE PRIMATE BRAIN 279 A Auditory B Visual Horizontal Saccade Endpoint (deg) 20 Monkey C 20 Monkey C 10 10 0 0 Fixation: –10 Left –10 Center Right –20 –20 –20 –10 0 10 20 –20 –10 0 10 20 C D Horizontal Saccade Endpoint (deg) 20 Monkey X 20 Monkey X 10 10 0 0 –10 –10 –20 –20 –20 –10 0 10 20 –20 –10 0 10 20 Horizontal target location (deg) Horizontal target location (deg)Figure 15.6 Eye position did not affect the accuracy of saccades to visual and auditory stimuli: Thedifferent colored traces, corresponding to saccades endpoints from three different initial fixation positions,are largely superimposed. From Metzger, R. R., Mullette-Gillman, O. A., Underhill, A. M., Cohen, Y. E., &Groh, J. M. (2004). Auditory saccades from different eye positions in the monkey: Implications forcoordinate transformations. Journal of Neurophysiology, 92, 2622–2627. Used with permission.Jay and Sparks, should be in the same reference previously, visual neurons exhibit receptiveframe and in spatial alignment. This possibility fields from the very earliest stages of the visualcould also help account for the better correspon- pathway. These receptive fields arise due to thedence between auditory tuning and saccade optics of the eye: Light from a given location invector that was observed in a recent study of the world passes through the aperture of thethe cat SC (Populin et al., 2004): The response pupil and illuminates only a restricted portionwindow used in that study included motor- of the retina. Each photoreceptor can only ‘‘see’’related activity in addition to sensory activity. out in a particular direction. Receptive fieldsMore research is needed to resolve this issue. become more complex as signals progress along the visual pathway, but at base, the code for space remains a code in which the location of a visual stimulus can be inferred from the identityREPRESENTATIONAL FORMAT of the neurons that are responding to it. ThisThe second potential computational challenge type of code is referred to as a place code,for integrating the visual and auditory codes because neurons are often topographically orga-for space is representational format. As noted nized according to their receptive field locations.

280 PRIMATE NEUROETHOLOGY In contrast, in the auditory system, spatial IC neurons to determine whether they have cir-location is inferred by comparing cues such as cumscribed receptive fields tiling the auditorysound arrival time and level across the two ears. scene. Instead, we found that they showed con-What kind of code is produced as part of this sistent preferences for locations along the axis ofcomputation cannot be determined from first the contralateral ear. This pattern is character-principles. There would seem to be two possi- istic of a rate code for sound location (Fig. 15.8a)bilities: (1) a place code similar to that for visual (Groh et al., 2003).information, in which auditory neurons have We found similar results in auditory cortexcircumscribed receptive fields that tile the audi- (Fig. 15.8b) (Werner-Reiss & Groh, 2008).tory scene; the location of a stimulus could then Interestingly, the code was less smooth in audi-be inferred from knowing which neurons were tory cortex than in IC: Individual neurons oftenresponding to that stimulus (e.g., Jeffress, had ‘‘bumpy’’ response functions that were1948), as is the case for visual information; broadly tuned for the contralateral ear, but alsoand (2) a rate code in which neurons respond had other sound locations that they alsobroadly to a wide range of locations, but with a responded well to. One possible reason for thisfiring rate that varies with sound locations. The is that there could be a transformation from ratelocation of a stimulus could be inferred by code to a place code as the auditory signals‘‘reading out’’ the firing rate of the active neu- approach or join with visual signals. If this isrons rather than the identity of the active the case, then neurons in brain regions such asneurons. parietal cortex or the superior colliculus might The key difference between these two types of show circumscribed receptive fields. Quanti-codes is the shape of the tuning function of tative information on the representationalindividual neurons. Do neurons respond only format of auditory signals in these structures isto a restricted range of locations, with different currently lacking in primates.neurons showing different preferences? Or do It will be interesting to determine whetherindividual neurons respond broadly, with the auditory signals are ultimately translated intomaximum responses occurring at the extremes a place code. The chief advantage of thisof the possible range of space (e.g., the axis of the would be to facilitate integration with place-contralateral ear) (Fig. 15.2). coded visual information. However, other We have conducted several studies to assess than that, the advantages might be few. Placethe coding format in the primate auditory codes are better than rate codes for encodingpathway. We developed a statistical assay based many locations simultaneously, but the audi-on the success of Gaussian and sigmoidal func- tory system may not be able to encode largetions at fitting the responses as a function of numbers of sound locations. Perceptually, twosound location. The idea is that Gaussian func- very similar simultaneous sound sources tendtions would be substantially better than sig- to be perceived at an intermediate locationmoids at fitting the response patterns if the (summing localization) (for review seeneurons had nonmonotonic spatial response Blauert, 1997). Furthermore, if signals arefunctions characteristic of receptive fields and a rate coded at one stage and converted into aplace code, but that either sigmoids or broad place code at a later stage, it is not clear thathalf-Gaussians would be successful at fitting the benefits of place coding would thenmonotonic tuning patterns characteristic of a accrue, as the rate-coding stage would serverate code (Fig. 15.7). as an information processing bottleneck that To our knowledge, nothing is known about would prevent subsequent place-coding stagesthe coding of spatial location in the primate from representing multiple simultaneous sti-auditory pathway prior to the level of the IC. mulus locations.The IC itself is known to contain spatially sensi- A possible advantage for retaining auditorytive neurons (Groh et al., 2001, 2003; Zwiers et spatial information in a rate code is that ital., 2004). We evaluated the spatial sensitivity of may facilitate interactions with eye position

LOOKING AT SOUNDS: NEURAL MECHANISMS IN THE PRIMATE BRAIN 281Figure 15.7 Simulations of place and rate codes and how Gaussian and sigmoidal curve fits can be used todistinguish between these representational formats. (A) Simulation of three Gaussian-tuned neurons,showing both Gaussian and sigmoidal curve fits. (B) Simulation of three sigmoidal neurons. TheGaussian and sigmoidal curve fits are so similar as to obscure each other. (C) Population plot of thecorrelation coefficients of Gaussian and sigmoidal curves for a population of individual neurons whoseunderlying tuning functions were Gaussian. Gaussian curves were always successful at fitting such responsepatterns; sigmoidal functions became increasingly successful as the eccentricity (i.e., the absolute value ofthe azimuthal location) of the Gaussian peak increased. (D) Same as c, but for a population of individualneurons whose underlying tuning functions were sigmoidal. Both Gaussian and sigmoidal curves weresuccessful at fitting such response patterns. From Werner-Reiss, U., & Groh, J. M. (2008). A rate code forsound azimuth in monkey auditory cortex: implications for human neuroimaging studies. Journal ofNeuroscience, 28(14), 3747–3758. Used with permission.information, which appears to be encoded in a MOTOR COMMANDSsimilar format. We found that in the IC, eyeposition sensitivity is generally monotonic What is the reference frame and representational(e.g., Fig. 15.3), consistent with a rate code for format of the motor command? The pattern ofeye position. This finding is consistent with stu- force needed to move the eyes to look in adies in the parietal cortex (Andersen et al., 1990), particular direction reflects a combination offrontal eye fields (Bizzi, 1968), cerebellar floc- reference frames and a combination of represen-culus (Noda & Suzuki, 1979), somatosensory tational formats. For a movement in a givencortex (Wang et al., 2007), and premotor cir- direction, the amount of force that needs to becuitry of the oculomotor pathway (Keller, 1974; applied varies monotonically with the size of theLuschei & Fuchs, 1972; McCrea & Baker, 1980; movement, consistent with a rate code. TheSylvestre & Cullen, 1999a). direction of the movement is controlled by the

282 PRIMATE NEUROETHOLOGYFigure 15.8 Representational format in inferior colliculus (IC) and core auditory cortex. (A) In the IC,most spatially sensitive neurons responded in a graded, monotonic fashion peaking for sounds along the axisof the contralateral ear, as shown for this example neuron (left panel). Across the population, this pattern isevident in the fact that sigmoidal functions were as good as Gaussians at capturing the response patternsFrom Groh, J. M., Kelly, K. A., & Underhill, A. M. (2003). A monotonic code for sound azimuth in primateinferior colliculus. Journal of Cognitive Neuroscience, 15, 1217–1231. (B) The pattern of results was similar inauditory cortex, although some individual neurons had ‘‘bumpy’’ response functions (data points lie slightlyabove the line of slope one in the right panel). From Werner-Reiss, U., & Groh, J. M. (2008). A rate code forsound azimuth in monkey auditory cortex: implications for human neuroimaging studies. Journal ofNeuroscience, 28(14), 3747–3758. Used with permission.

LOOKING AT SOUNDS: NEURAL MECHANISMS IN THE PRIMATE BRAIN 283ratio of activation in different muscle groups, a MODELINGformat that is more akin to a place code: Whichmuscles are active controls the movement Although it remains unclear exactly what kindsdirection. of transformations unfold as visual and audi- The reference frame of oculomotor com- tory signals converge onto the oculomotormands is referred to as eye centered by some pathway, it is certainly evident that some trans-sources and head centered by others. The con- formations between coding formats and refer-fusion stems from both what is meant by motor ence frames are needed, and it can be fruitful tocommand—is this term properly reserved only explore the neural mechanisms that mightfor the extraocular motor neurons or may it be underlie such transformations while additionalapplied to slightly earlier stages such as the experimental studies are pending. Accordingly,SC?—as well as a lack of quantitative investiga- we have worked on several models for trans-tion into this question. We favor reserving the forming signals between different referenceterm ‘‘motor command’’ for the signals carried frames and between different coding formats.by the extraocular motor neurons. The reference We will begin with the models for transforma-frame of extraocular motor neurons has not tions of coding format, because how informa-been investigated per se, but their discharge pat- tion is encoded impacts which algorithms forterns are so well characterized using other means coordinate transformations may be mostthat it is possible to draw some inferences. appropriate.Specifically, the discharge patterns can bedescribed as a linear differential equation (e.g., Models for Transformations of CodingSylvestre & Cullen, 1999b): Format We have designed several models that involve FR ¼ k1 þ k2 P þ k3 V; transformations of signals from either a place code to a rate code or vice versa. Figure 15.9where FR = instantaneous firing rate, P = eye illustrates several ways that Gaussian tuningposition, and V = eye velocity. This equationillustrates that the firing pattern depends on functions can be created from a population ofboth initial and final eye position—that is, fixa- neurons with sigmoidal response functionstion position as well as the head-centered loca- with varying inflection points (Porter & Groh,tion of the target. Thus, the motor command 2006). Suppose neurons exhibit sigmoidalcannot be properly formed if premotor circuitry tuning functions, with some preferring leftwardhas access only to target location in a single pure locations (such as spatial neurons in the rightreference frame—some combination of head- IC) and others preferring leftward locationscentered, eye-centered, and eye position infor- (e.g., the left IC). Assume further that there ismation is needed. This suggests that the use of a population of neurons whose inflectionhybrid reference frames at earlier stages of the points vary across the range of space.audio-oculomotor pathway may reflect the con- Excitatory connections from two neurons withstraints of the motor periphery. Most existing opposite tuning preferences and neighboringmodels for how the motor command is formed inflection points would cause a recipientcall for separate representations of head- or eye- neuron to be responsive to sound locationscentered information to be combined with eye between the inflection points of the input neu-position information as the time-varying motor rons (Fig. 15.9a).neuron discharge pattern is created A circumscribed receptive field could also be(Moschovakis, 1996; Van Gisbergen & Van created by combining excitatory and inhibitoryOpstal, 1989), but it might also be possible to inputs from two neurons with sigmoidal tuninggenerate this command from an input signal that functions in the same direction, provided oncealready has these component signals mixed again that their inflection points are appropriatelytogether. staggered. Suppose two neurons both prefer

A D Input: place code Discharge level of IC neurons Activity Loud Soft Left IC Right IC Sensory parameter value Sound location ai + + Output: wi rate code Sound location B E Place code Discharge level of IC neurons Right IC Right IC Denominator Numerator channel channel Sound location – + Rate code Sound location F Place code Input: rate code Denominator channel Numerator channel C Inhibitory interneurons Inhibitory and output units with inteneuron graded thresholds with threshold Output: place code Rate codeFigure 15.9 Models for converting signals between place and rate codes. (A) Sigmoidal units with oppositepreferences and different inflection points could converge, producing a receptive field. If sound loudnessaffected the response patterns of the input neurons as indicated, then the receptive field would be larger forlouder sounds but would be centered in the same place. (B) Sigmoidal units preferring the same side but withdifferent inflection points could converge using a combination of excitatory and inhibitory synapses. As inpanel a, the receptive field would expand for louder sounds if the two input neurons were affected by soundloudness in different directions. (C) An alternative mechanism involving a cascade of units with differentthresholds, with inhibitory interneurons. (D–F) Three models for converting place codes to rate codes, withdifferent ways of normalizing for the level of activity. The first model involves no normalization, the secondinvolves full normalization, and the third involves normalization only for high levels of activity. See text foradditional details. Groh, J. M. (2001). Converting neural signals from place codes to rate codes. BiologicalCybernetics, 85, 159–165; and Porter, K. K., & Groh, J. M. (2006). The other transformation required for visual-auditory integration: Representational format. Progress in Brain Research, 155, 313–323. Used with permission. 284

LOOKING AT SOUNDS: NEURAL MECHANISMS IN THE PRIMATE BRAIN 285leftward locations, but one has an inflection point (Fig. 15.9d) simply calculates the weightedat 0 degrees and the other has an inflection point sum of activity, with no normalization whatso-at 10 degrees to the right. A recipient neuron that ever. The problem with such a model is thatis inhibited by the first neuron and excited by the typically there are many other features thatsecond neuron will have a receptive field between might alter neural activity (e.g., the loudness0 and 10 degrees to the right—the region of space of a sound or the contrast of a visual stimulus),where only its excitatory input is active and without normalization the changes in(Fig. 15.9b). neural responsiveness associated with these fea- Both of these algorithms involve a certain tures would affect the read-out. There is someelement of place coding in the input stage: The perceptual evidence for this kind of effect: Forinput neurons must have tuning functions example, low-contrast visual stimuli appear towhose inflection points show heterogeneity move more slowly than high-contrast visualspanning the range of possible spatial locations. stimuli (e.g., Snowden et al., 1998; ThompsonThus, this is not a pure rate code at the input. At et al., 1996), but more generally such factorspresent, we have not attempted to determine if appear to be corrected for when determiningthe response functions are truly sigmoidal as spatial location.opposed to some other monotonic function, Accordingly, we developed several additionalnor have we assessed whether the inflection models for converting place codes to rate codes that include normalization for the overall level ofpoints span a range of locations. Thus, it is activity. The vector averaging model (Fig. 15.9e)unclear whether these algorithms are biologi- has two read-out pathways, one to calculate thecally plausible or not. sum of activity weighted by its location in the A third algorithm might apply if the input place code (the numerator channel) and thesignals are more linear than sigmoidal and if they other to calculate the unweighted sum (thetherefore lack inflection points, much less varia- denominator channel). Then, the weighted sumtion in inflection points. Figure 15.9c illustrates is divided by the unweighted sum, producing aa local circuit with a cascade of thresholds. The signal corresponding to the average location ofthresholds introduce the necessary nonlinearity activity in the place code.into the processing of a linear signal to create the One problem with this model is that it is notreceptive fields. Each output neuron (open cir- clear how neural circuits might implement thecles) has both a threshold and an inhibitory division of one number by another. Inhibitoryinterneuron that is paired with it. The inhibitory synapses can exert a divisive-like effect, but moreinterneuron has a slightly higher threshold for generally the nature of the inhibitory influenceactivation. Thus, the output neuron is active will vary with the membrane potential: Whatonly when its input exceeds its own threshold seems like division when the membrane poten-but is less than the threshold for its matched tial is near rest (e.g., shunting inhibition) mightinhibitory interneuron. This pattern, when become more like subtraction when the mem-repeated with varying thresholds across the brane is more depolarized. The vector averagingpopulation, can create a range of circumscribed model requires that the inhibitory influence ofreceptive fields across the population (Groh & the denominator channel should mimic divisionSparks, 1992). for a large range of possible numerator and We have also developed several models for denominator values.converting signals from a place code to a rate The third model circumvents this problem bycode (Groh, 2001; Porter & Groh, 2006) (Fig. implementing normalization in a different15.9d–f). The conversion is accomplished using fashion. This model, the summation-with-a graded pattern of synaptic weights. The saturation model, calculates a weighted sum ofmodels differ in whether and how they accom- the activity in the input layer, and then clips offplish normalization for the overall level of any extra activity above a certain threshold. Thisactivity. The vector summation model is accomplished using a combination of neural

286 PRIMATE NEUROETHOLOGYintegrators and thresholds. The numerator and from which rate-coded eye position informationdenominator channels both integrate their is subtracted. This produces a rate code for eye-input, weighted by location in the case of the centered sound location, just as in the originalnumerator channel. When the denominator version. It may or may not be necessary to thenchannel reaches a certain threshold, it clips off convert these signals into a place code, but if it isthe input to the numerator channel. The activity necessary, one of the rate-to-place algorithmslevel of the numerator channel will vary only described previously could still be included.with the location of the input provided there is A second type of model, the dendrite model,sufficient activity to trigger the clipping action of was originally proposed with the goal ofthe denominator channel, and otherwise will avoiding rate-coding stages in mind (Groh &reflect the weighted sum of the input. Sparks, 1992) (Fig. 15.11). The rationale was This model successfully mimics the pattern of that the rate-coding stages would limit theevoked saccades elicited by microstimulation of number of sound locations that could bethe SC. Stimulation above a certain frequency encoded to one. Since it now appears that rate-evokes saccades that do not depend on the fre- coding stages do exist in the brain’s auditoryquency of stimulation (known as the site-specific pathways, the motivation behind this modelamplitude), but below that value the amplitude of has been reduced. However, it remains uncertainthe saccade falls off as the frequency or duration how the brain handles multiple sound locations,of stimulation is reduced. The evoked saccade so elements of the dendrite model may yet provedepends on the total number of stimulation to be of some utility.pulses delivered until a saturation point isreached (Stanford et al., 1996). CONCLUSIONSModels for Coordinate Transformations To guide an eye movement to the source of aWe have developed two models for coordinate sound requires a net transformation of auditorytransformations. At the time these models were information from the initially purely head-cen-designed, little was known about either the tered interaural timing and level cues to a refer-representational format or the frame of reference ence frame appropriate for controlling the eyeof signals in the auditory pathway, so both muscles. Our studies as well as others havemodels assumed that the input consisted of a found evidence for hybrid, but not purely eye-head-centered map of auditory space. Since centered, frames of reference at several stages ofthis kind of representation has yet to be found, the audio-oculomotor reference frame. This typeit is worth updating these models to consider of hybrid reference frame may be appropriate forother possible forms of input (as well as output). controlling saccades because the motor command The vector subtraction model (Fig. 15.10a) requires information about both the initial eyebegins by converting head-centered, place- position and the desired amplitude of the saccade.coded auditory signals into a rate code so that Eye movements to sounds may also requirerate-coded eye position signals could be sub- one or more transformations of auditory sig-tracted. The resulting eye-centered rate code nals from one kind of coding format intofor sound location was then converted into a another. At present, we have only found evi-place code for eye-centered sound location. dence for rate coding of auditory spatial infor-This model was essentially constructed from mation. It remains to be seen whether rate-the place-to-rate and rate-to-place component coded auditory spatial information is trans-parts. formed into a place code, and if so, where and Since it now appears that sound location may how this transformation occurs. It has longbe encoded in a rate code, a simpler version of been assumed that this transformation doesthis model can be constructed (Fig. 15.10b). The take place, but it should be noted that it is notinput can consist of a rate-coded sound location, necessary to create a place code for auditory

A “Vector Subtraction” model: original version Auditory map (head-centered) Response rate Response rate Intermediate unit Eye position (rate code) Eye position Sound w.r.t. eyes Inhibitory interneurons Auditory map (eye-centered) B “Vector Subtraction” model: rate code only version Rate code Head-centered sound location Rate code Eye position Output: rate code Eye-centered sound locationFigure 15.10 Vector subtraction model for transforming auditory signals from head-centered to eye-centered coordinates. (A) This model assumes place-coded auditory inputs. It now appears possible that theauditory inputs encode sound location in a rate-coded format. (B) In this case, the model could be modifiedto use the rate-coded auditory input signals. From Groh, J. M., & Sparks, D. L. (1992). Two models fortransforming auditory signals from head-centered to eye-centered coordinates. Biological Cybernetics, 67,291–302. Used with permission. 287

288 PRIMATE NEUROETHOLOGY Dendrite model it d .... .... un ere ry nt to e di d-c au ea .... H it y n un rn or ro te it n eu in hib tio si In po e Ey Eye-centered auditory unit Head-centered auditory receptive field for an individual dendrite Eye position “receptive field” for an individual dendrite Eye-centered receptive field + Head-centered spaceFigure 15.11 Dendrite model for transforming auditory signals from head-centered to eye-centeredcoordinates. Each dendrite receives input from eye position units and one head-centered auditory unit.Thresholds and synaptic weights are balanced so that the cell body receives net excitation via a givendendrite if an auditory stimulus is present in the head-centered receptive field of the auditory unit thatactivates that dendrite and the eyes are within a certain range of positions. The range of eye positions thatallow excitation to reach the soma varies across dendrites and is matched to the head-centered receptive fieldof the auditory input that activates a given dendrite. The combination produces tuning for the eye-centeredlocation of the sound. In short, the dendrites of a given unit sample all the possible head-centered locationsthat could yield a certain eye-centered location, and the eye position inputs filter out that input unless theeye position is in the appropriate range for that dendrite. From Groh, J. M., & Sparks, D. L. (1992). Twomodels for transforming auditory signals from head-centered to eye-centered coordinates. BiologicalCybernetics, 67, 291–302. Used with permission.information simply to guide an eye movement. has likely produced auditory information-Since the motor output consists of a rate code, coding strategies that serve more than oneany place-coded auditory information would behavioral and perceptual master. Thus, otherhave to be converted back into a rate code to constraints may account for the aspects ofgenerate a motor command. audio-oculomotor transformations that may Of course, auditory signals do not exist appear at face value to be inefficient. Furthersolely to trigger eye movements. The perceptual research on whether the behavioral task affectsand behavioral endpoints of auditory proces- the type of code employed will therefore be ofsing are many and varied, and natural selection great interest.

LOOKING AT SOUNDS: NEURAL MECHANISMS IN THE PRIMATE BRAIN 289NOTE Groh, J. M., & Sparks, D. L. (1996a). Saccades to somatosensory targets. III. Eye-position-1. Several studies did map the receptive fields, but dependent somatosensory activity in primate sampled primarily along a dimension superior colliculus. Journal of Neurophysiology, orthogonal to the direction in which fixation 75, 439–453. position varied (Andersen et al., 1985; Batista Groh, J. M., & Sparks, D. L. (1996b). Saccades to et al., 1999); the effects of eye position on somatosensory targets. II. Motor convergence response patterns observed in these studies in primate superior colliculus. Journal of were consistent with either the head- Neurophysiology, 75, 428–438. centered or eye-centered-with-eye-position- Groh, J. M., & Sparks, D. L. (1996c). Saccades to gain hypotheses. somatosensory targets. I. Behavioral characteristics. Journal of Neurophysiology, 75, 412–427. Groh, J. M., Trause, A. S., Underhill, A. M., Clark,REFERENCES K. R., & Inati, S. (2001). Eye position influencesAndersen, R., & Zipser, D. (1988). The role of the auditory responses in primate inferior posterior parietal cortex in coordinate colliculus. Neuron, 29, 509–518. transformations for visual-motor integration. Hartline, P. H., Vimal, R. L., King, A. J., Kurylo, D. Canadian Journal of Physiology and D., & Northmore, D. P. (1995). Effects of eye Pharmacology, 66, 488–501. position on auditory localization and neuralAndersen, R. A., Bracewell, R. M., Barash, S., Gnadt, representation of space in superior colliculus of J. W., & Fogassi, L. (1990). Eye position effects cats. Experimental Brain Research, 104, 402–408. on visual, memory, and saccade-related activity Jay, M. F., & Sparks, D. L. (1984). Auditory in areas LIP and 7a of macaque. Journal of receptive fields in primate superior colliculus Neuroscience, 10, 1176–1196. shift with changes in eye position. Nature, 309,Andersen, R. A., Essick, G. K., & Siegel, R. M. (1985). 345–347. Encoding of spatial location by posterior parietal Jay, M. F., & Sparks, D. L. (1987a). Sensorimotor neurons. Science, 230, 456–458. integration in the primate superior colliculus.Andersen, R. A., & Mountcastle, V. B. (1983). The II. Coordinates of auditory signals. Journal of influence of the angle of gaze upon the Neurophysiology, 57, 35–55. excitability of the light-sensitive neurons of the Jay, M. F., & Sparks, D. L. (1987b). Sensorimotor posterior parietal cortex. Journal of integration in the primate superior colliculus. I. Neuroscience, 3, 532–548. Motor convergence. Journal of Neurophysiology,Batista, A. P., Buneo, C. A., Snyder, L. H., & Andersen, 57, 22–34. R. A. (1999). Reach plans in eye-centered Jeffress, L. A. (1948) A place theory of sound coordinates [see comments]. Science, 285, 257–260. localization. Journal of Comparative PhysiologyBizzi, E. (1968). Discharge of frontal eye field and Psychology, 41, 35–39. neurons during saccadic and following eye Keller, E. L. (1974). Participation of medial pontine movements in unanesthetized monkeys. reticular formation in eye movement Experimental Brain Research, 6, 69–80. generation in monkey. Journal ofBlauert, J. (1997). Spatial hearing. Cambridge, MA: Neurophysiology, 37, 316–332. MIT Press. Klier, E. M., Wang, H., & Crawford, J. D. (2001).Groh, J. M. (2001). Converting neural signals from The superior colliculus encodes gaze commands place codes to rate codes. Biological Cybernetics, in retinal coordinates. Nature Neuroscience, 4, 85, 159–165. 627–632.Groh, J. M., Kelly, K. A., & Underhill, A. M. (2003). Luschei, E. S., & Fuchs, A. F. (1972). Activity of A monotonic code for sound azimuth in brain stem neurons during eye movements of primate inferior colliculus. Journal of Cognitive alert monkeys. Journal of Neurophysiology, 35, Neuroscience, 15, 1217–1231. 445–461.Groh, J. M., & Sparks, D. L. (1992). Two models for McCrea, R., & Baker, R. (1980). Evidence for the transforming auditory signals from head- hypothesis that the prepositus neucleus centered to eye- centered coordinates. distributes ‘‘efference’’ copy signals to the Biological Cybernetics, 67, 291–302. brainstem. Anatomy Research, 196, 122–123.

290 PRIMATE NEUROETHOLOGYMeredith, M. A., & Stein, B. E. (1983). Interactions Robinson, D. A. (1972). Eye movements evoked by among converging sensory inputs in the collicular stimulation in the alert monkey. superior colliculus. Science, 221, 389–391. Vision Research, 12, 1795–1807.Meredith, M. A., & Stein, B. E. (1996). Spatial Schiller, P. H., & Stryker, M. (1972). Single-unit determinants of multisensory integration in recording and stimulation in superior colliculus cat superior colliculus neurons. Journal of of the alert rhesus monkey. Journal of Neurophysiology, 75(5), 1843–1857. Neurophysiology, 35, 915–924.Metzger, R. R., Mullette-Gillman, O. A., Underhill, Snowden, R. J., Stimpson, N., & Ruddle, R. A. A. M., Cohen, Y. E., & Groh, J. M. (2004). (1998). Speed perception fogs up as visibility Auditory saccades from different eye positions drops [letter]. Nature, 392, 450. in the monkey: Implications for coordinate Sparks, D. L. (1978). Functional properties of transformations. Journal of Neurophysiology, neurons in the monkey superior colliculus: 92, 2622–2627. Coupling of neuronal activity and saccadeMoore, D. R. (1991). Anatomy and physiology of onset. Brain Research, 156, 1–16. binaural hearing. Audiology, 30, 125–134. Sparks, D. L., & Hartwich-Young, R. (1989a). TheMoschovakis, A. K. (1996). Neural network deep layers of the superior colliculus. In: R. H. simulations of the primate oculomotor system. Wurtz & M. E. Goldberg (Eds.), The II. Frames of reference. Brain Research Bulletin, neurobiology of saccadic eye movements (pp. 40, 337–345. 213–255). New York: Elsevier.Mullette-Gillman, O. A., Cohen, Y. E., & Groh, J. Sparks, D. L., & Hartwich-Young, R. (1989b). The M. (2005). Eye-centered, head-centered, and deep layers of the superior colliculus. Reviews of complex coding of visual and auditory targets Oculomotor Research, 3, 213–255. in the intraparietal sulcus. Journal of Stanford, T. R., Freedman, E. G., & Sparks, D. L. Neurophysiology, 94, 2331–2352. (1996). Site and parameters of microstimulation:Nieuwenhuys, R. (1984). Anatomy of the Evidence for independent effects on the auditory pathways, with emphasis on the properties of saccades evoked from the primate brain stem. Advances in Otorhinolaryngology, superior colliculus. Journal of Neurophysiology, 34, 25–38. 76, 3360–3381.Noda, H., & Suzuki, D. A. (1979). Processing of eye Stein, B. E., & Meredith, M. A. (1993). The merging movement signals in the flocculus of the of the senses. Cambridge, MA: MIT Press. monkey. Journal of Physiology, 294, 349–364. Stein, B. E., Meredith, M. A., & Wallace, M. T.Oliver, D. L. (2000). Ascending efferent projections (1993). The visually responsive neuron and of the superior olivary complex. Microscopic beyond: Multisensory integration in cat and Research Technology, 51, 355–363. monkey. Progress in Brain Research, 95,Peck, C. K., Baro, J. A., & Warder, S. M. (1995). 79–90. Effects of eye position on saccadic eye Sylvestre, P. A., & Cullen, K. E. (1999a). movements and on the neuronal responses to Quantitative analysis of abducens neuron auditory and visual stimuli in cat superior discharge dynamics during saccadic and slow colliculus. Experimental Brain Research, eye movements. Journal of Neurophysiology, 82, 103(2), 227–242. 2612–2632.Populin, L. C., Tollin, D. J., & Yin, T. C. (2004). Sylvestre, P. A., & Cullen, K. E. (1999b). Effect of eye position on saccades and neuronal Quantitative analysis of abducens neuron responses to acoustic stimuli in the superior discharge dynamics during saccadic and slow colliculus of the behaving cat. Journal of eye movements. Journal of Neurophysiology, 82, Neurophysiology, 92, 2151–2167. 2612–2632.Porter, K. K., & Groh, J. M. (2006). The other Thompson, P., Stone, L. S., & Swash, S. (1996). Speed transformation required for visual-auditory estimates from grating patches are not contrast- integration: Representational format. Progress normalized. Vision Research, 36, 667–674. in Brain Research, 155, 313–323. Van Gisbergen, J. A., & Van Opstal, A. J. (1989).Porter, K. K., Metzger, R. R., & Groh, J. M. (2006). Models of saccadic control. In: R. Wurtz & Representation of eye position in primate M. E. Goldberg (Eds.), The neurobiology inferior colliculus. Journal of Neurophysiology, of saccadic eye movements. Amsterdam: 95, 1826–1842. Elsevier.

LOOKING AT SOUNDS: NEURAL MECHANISMS IN THE PRIMATE BRAIN 291Wallace, M. T., Wilkinson, L. K., & Stein, B. E. position affects activity in primary auditory (1996). Representation and integration of cortex of primates. Current Biology, 13, multiple sensory inputs in primate superior 554–562. colliculus. Journal of Neurophysiology, 76, Zella, J. C., Brugge, J. F., & Schnupp, J. W. (2001). 1246–1266. Passive eye displacement alters auditory spatialWang, X., Zhang, M., Cohen, I. S., & Goldberg, M. receptive fields of cat superior colliculus E. (2007). The proprioceptive representation neurons. Nature Neuroscience, 4, 1167–1169. of eye position in monkey primary Zipser, D., & Andersen, R. A. (1988). A back- somatosensory cortex. Nature Neuroscience, propagation programmed network that 10, 640–646. simulates response properties of a subset ofWerner-Reiss, U., & Groh, J. M. (2008). A rate code posterior parietal neurons. Nature, 331, for sound azimuth in monkey auditory cortex: 679–684. Implications for human neuroimaging studies. Zwiers, M. P., Versnel, H., & Van Opstal, A. J. Journal of Neuroscience. 28(14), 3747–3758 (2004). Involvement of monkey inferiorWerner-Reiss, U., Kelly, K. A., Trause, A. S., colliculus in spatial hearing. Journal of Underhill, A. M., & Groh, J. M. (2003). Eye Neuroscience, 24, 4145–4156.

CHAPTER 16Circuits of Emotion in the Primate BrainKatalin M. Gothard and Kari L. HoffmanINTRODUCTION threshold for threat-related stimuli is lowered, reaction time is shortened, and widespread sym-Emotions are coordinated brain-body states that pathetic effects lock the internal organs in a func-allow animals to cope with the challenges of their tional mode biased toward energy expenditure.physical and social environment. Emotional A completely different brain-body state isstates are characterized by a specific configura- triggered by an affiliative approach, signalingtion of inputs (triggering events), outputs (auto-nomic and somatic responses), and the neural the intention to groom. Such an approach, espe-processes that mediate their transformation. cially if there is a history of affiliative interactionMany emotional states, especially acute states between the players, leads to relaxed postures,such as fear or anger, are coupled with enhanced accompanied by vagal tone, increased levels ofperceptual processing, decision making, action oxytocin, and often gestures of reciprocation.selection, and increased energetic expenditure. The behavioral outcomes of brain-body activa- The brain-body state triggered by a threa- tion are not unique to a particular emotion; thetening facial expression, a common event in the initiating factors of emotions and the internaldaily life of a macaque, exemplifies the phe- states they generate are much more diverse thannomena ascribed to emotion. When a midranking the relatively limited behavioral repertoire avail-male monkey is confronted with a threat display, able for their expression. The autonomic andthe covert autonomic and overt behavioral behavioral expressions of emotions are thereforeresponses that are triggered by observing this dis- low-dimensional and highly reproducible acrossplay are the results of a complex process of eval- situations and individuals. In contrast, the stimuliuating the emotional and social significance of that evoke emotional expressions are often high-that display given the present internal state of dimensional, and their emotional content isthe receiver monkey. This evaluation takes into dependent also on context and the emotionalaccount the identity of the displaying monkey; he states preceding the stimulus. In light of the con-is recognized as a dominant male, with a well- trast between the extremely rich array of stimuligroomed, muscular body, red-pigmented face, that can generate emotions and the restricted set ofand large, symmetrical canines. If the display is expressive affordances, the central process of eval-directed to a close ally of the viewer, then based uating the emotional significance of all stimuli andon this evaluation, the viewer decides whether to events encountered by the organism is essentially aflee the scene of an imminent confrontation or to process of dimensionality reduction.rush to defend his ally. For either action, the This dimensionality reduction takes place inautonomic system sets the organism in a ‘‘higher nested circuits, in which phylogenetically oldergear.’’ The animal is more vigilant, the sensory regions form a core circuit (Fig. 16.1). The core 292

Figure 16.1 Structures contributing to circuits of emotion in the macaque. Core structures are shown inred, intermediate structures in orange, and affiliate structures in yellow. (A) Coronal section through severalkey structures. Arrows indicate connections between structures, depicted on the left hemisphere. Structurenames are listed on the right hemisphere. Connections are shown only for structures visible in this imageplane. The basolateral amygdala has bidirectional connections with the superior temporal sulcus (STS),anterior cingulate, and the insula, and projects to the central nucleus of the amygdala. The central nucleus,in turn, sends projections to the hypothalamus. (B) Sagittal section depicting several key structures and themedial forebrain bundle (MFB). The MFB (black line) contains fibers originating in severalneuromodulatory centers in the brainstem and midbrain, and projecting to the hypothalamus and theprefrontal cortex. Neuromodulator-specific fiber tracts continue on to innenervate other regions ofneocortex (see text). (C) Horizontal (axial) section at the level depicted by the blue line in A.Conventions as in A. The brainstem (indicated by the periaqueductal gray arrows) projects to severalstructures including the hippocampus, amygdala, and superior temporal sulcus, among other structures.(D) Schematic diagram of brainstem connections subserving circuits of emotion. Conventions as in A. Thisdiagram loosely depicts the ventral surface of the brain, including several core brainstem nuclei. Sensorynuclei include those from cranial nerves V, VII, IX, and X. Motor nuclei include those from cranial nervesIII, IV, V, VI, VII, IX, X, XI, and XII. The periaqueductal gray (not labeled) is the vertical gray bar connectedto the superior colliculus and the reticular formation. ACC, anterior cingulate cortex; BLA, basolateralamygdala; BS, brainstem; CeA, central nucleus of the amygdala; HIP, hippocampus; HY, hypothalamus;INS, insula; M, motor cranial nerve nuclei; NTS, nucleus of the solitary tract; OFC, orbitofrontal cortex;PAG, periaqueductal gray; PB, parabrachial nucleus; RF, reticular formation; S, sensory cranial nerve nuclei;SC, superior colliculus. 293

294 PRIMATE NEUROETHOLOGYcircuit is localized primarily in the brainstem THE ORGANIZATION OFand midbrain and produces low-dimensional EMOTIONAL CIRCUITSoutput (e.g., heart rate and blood pressure canincrease or decrease; other effects, such as Core Structurespiloerection, sweating, pupil dilation, contrac- Core structures are those whose activity gener-tion of various muscles, etc., also have few ates a relatively restricted array of emotionaldegrees of freedom). Additional circuits are responses (autonomic or somatic) via shortsuperimposed on this core. They encompass pathways that link inputs to outputs. Thesesubcortical and cortical areas that carry out include sensory, motor, and autonomic centerscomplex computations on high-dimensional of the brainstem; the periaqueductal gray (PAG);inputs, but ultimately funnel their outputs the deep layers of the superior colliculus; thethrough the core structures. In this framework, hypothalamus; and the centromedial (subcor-the core structures elaborate low-dimensional tical) nuclei of the amygdala (Fig. 16.1).affordances based on a limited set of stimuli(e.g., defensive behaviors toward predators), Sensory, Motor, and Autonomic Centers of the Brainstemwhile the outer circuits elaborate high-dimen-sional internal states to deliver the optimal All the afferent signals from the sensory organsbehavioral choices (e.g., learning that one can of the head and body (with the exception ofattack the member of a rival group only when olfaction) and all the viscerosensory signalsthe attack cannot be witnessed by those who from the internal organs converge in thecould retaliate). brainstem. Several complex behaviors such as Conservation of the phylogenetically old, foraging, feeding, flight and fight, response tocore structures does not imply that they per- pain, and reproduction can be initiated andform the same, ‘‘old’’ functions across extant maintained from the level of the brainstemspecies; rather, the emotional brain of primates alone (Blessing, 1997). Heart rate, respiration,can be thought of as a palimpsest in evolu- digestion, elimination, and even immune func-tionary terms. Even the function of homolo- tion are coordinated directly from the brain-gous structures across species may have been stem. The brainstem also contains a reticularco-opted for new or species-specific purposes. system that plays an important role in vigilanceThus, even as we refer to core structures that and behavioral activation in response to bothare conserved across vertebrates, mammals, or emotional and neutral stimuli. Based on theprimates, their functions may vary according collateral signals it receives from all theto species-specific, ecological, and ethological ascending sensory pathways, the reticulardemands. system generates a tonic output projected The first section of this chapter will toward the cortex (Blessing, 1997).identify the main structures that comprise Several neurotransmitters that are producedthe emotional circuits of the primate brain. in the brainstem determine emotional traits andThe second section describes what is known mediate emotional states, either by direct synapticabout the neural basis of emotional processes, or modulatory activity. The levels of norepi-from the association of stimuli to positive nephrine (NE) produced in the locus coeruleusor negative outcomes to the interplay of the brainstem are increased during vigilance,between the perception and expression of anxiety, acute fear, and other forms of emotionalsocial signals. Reflecting the biases in the stress. When levels of serotonin (5-hydroxytryp-literature, our descriptions will emphasize tamine [5-HT]) secreted by the raphe nuclei arethe macaque genus and the function of the low, animals are more aggressive and/oramygdala, the most highly connected compo- depressed. Dopamine (DA), a neurotransmitternent of the emotional brain (Young et al., associated with reward and pleasure, is produced1994). by the ventral tegmental area and substantia

CIRCUITS OF EMOTION IN THE PRIMATE BRAIN 295nigra. The gigantocellular nuclei of the reticular and descending modulatory center of the painformation produce acetylcholine (ACh), which pathways, but also a key center that engages themaintains the excitability and coherence of autonomic nervous system (Bandler et al., 2000)activity across multiple cortical areas. A bundle in conjunction with the brainstem centers men-of axons carry these and other neurotransmitters tioned previously. A few important ethologicalfrom the brainstem to the cortex; these axons functions distinguish the PAG from the brain-form the medial forebrain bundle that terminates stem centers, namely, its role in vocalizations,in the prefrontal cortex but gives out massive defensive behaviors, and aggression.collaterals to the hypothalamus (Fig. 16.1B). The Species-specific vocalizations are coordi-hypothalamus, therefore, registers the level of nated via the PAG, as demonstrated by electricalvigilance through NE, the presence of reinforce- ¨ stimulation (Jurgens, 1994). This is achieved byment (reward or lack thereof) through DA, and indirect control of the laryngeal and respiratoryother signals related directly to the activity of the muscles via the reticular formation of the brain-brainstem monoaminergic neurons. stem, (Davis et al., 1996). Motor commands for Of particular importance for emotions are vocalization are initiated in the PAG under des-two nuclei of the brainstem: the nucleus of the cending input from the orbital and medial pre-solitary tract (NTS) and the parabrachial nuclei frontal cortex. These two prefrontal systems are(Fig. 16.1D). The NTS is the main visceral sen- also linked to PAG activity during maternal per-sory nucleus that contains an elaborate viscero- ception of their offspring (Bartels & Zeki, 2004).sensory map of the body. This map, and the Prefrontal projections contact discrete regions of‘‘feeling of the body’’ contained therein, is the PAG that, in turn, connect to differentbroadcast to cortical areas (e.g., the insula and regions of the hypothalamus, forming parallelanterior cingulate cortex) where internal sensa- loops that link higher cognitive aspects of emo-tions are associated with external stimuli (Craig, tion and social behavior with expressive and2002). Ascending fibers from many cranial homeostatic/regulatory components of the corenerves report to the NTS the state of all internal circuit (Bandler et al., 2000).organs of the head and body; these signals are Aggression related to rage and defensiveeither transformed in the NTS into descending behaviors is coordinated jointly by the PAGcommands targeting sympathetic and parasym- and the lateral hypothalamus. The PAG is notpathetic effectors that coordinate autonomic involved, however, in attack associated with pre-reflexes or are transmitted via the thalamus to dation. Complex local neural and pharmacolo-higher cortical centers. The parabrachial nuclei gical mechanisms regulate the level of aggressionregulate respiration, but they also receive visceral within the PAG. The neural networks that coor-sensory information. The majority of emotional dinate different forms of aggression are recipro-states are associated with changes in respiration cally inhibitory and the output of each networkthat are controlled, in part, by the parabrachial is potentiated or dampened by different neuro-nuclei. Stimulation of this area causes a rapid transmitters. Glutamate generally enhances,inhale followed by apnea—a gasp (von Euler and while opioid peptides typically dampenTrippenbach, 1976)—a typical respiratory beha- aggression (Gregg & Siegel, 2001).vior for surprise, pain, fear, etc. A fascinating but unsolved problem in pri- The brainstem, therefore, is both a relay for mate neuroethology is the neural site andhigher-level centers of emotion and a first-stage mechanism of establishing the connectionsprocessing center that elaborates species-specific between social status and aggression. Givensomatic and autonomic behaviors in situations the fluctuating status of males in a macaqueof importance. society, the link between rank and aggression is expected to be highly flexible. Serotonin levelsPeriaqueductal Gray and the efficiency of serotonin transportersThe PAG has important sensory and motor have been invoked as the major determinantsfunctions in emotion. It is a major integration of trait aggression, but state aggression appears

296 PRIMATE NEUROETHOLOGYto be regulated by prefrontal-PAG/hypotha- amygdala, indicating that the role in social beha-lamic-brainstem mechanisms relying on a dif- vior of these two structures is not entirelyferent set of neurotransmitters and steroid redundant.hormones. Finally, the SC has been implicated in detecting emotionally salient visual stimuliSuperior Colliculus without the contribution of cortical visual path-A central role of the superior colliculus (SC) is to ways. A strong argument in favor of this proposalguide and modulate visual orientation. The SC comes from ‘‘blindsight’’ patients and animalscontains sensory, motor, and multimodal maps with uni- or bilateral visual cortical damage.that control the initiation and execution of sac- Subjects report being unaware of stimuli, yet arecades (for an early review, see Sparks & able to navigate visual obstacles, produce defen-Hartwich-Young, 1989). In addition to its sen- sive movements in response to looming stimuli,sorimotor function, the deep layers of the SC are and detect or even discriminate visual stimuli,a critical node in the core circuit of emotion and including threatening images of facial expressionssocial behavior. (de Gelder et al., 1999; Liddell et al., 2005; Morris Electrical stimulation of the rodent SC elicits et al., 2001; Stoerig & Cowey, 1997). One oforienting responses and species-specific defen- several proposed pathways for these preservedsive behaviors, and in some cases, approach abilities includes the SC, the pulvinar, and thebehaviors (Dean et al., 1989). Stimulation also amygdala, which show functional connectivity incauses changes in respiration, blood pressure, neuroimaging experiments and the predictedand heart rate (Keay et al., 1988); reduces pain processing deficits when lesioned (Morris et al.,threshold (Redgrave et al., 1996); and causes 1999, 2001; Vuilleumier, 2003; Ward et al., 2005).desynchronization of the cortical electroence- A colliculo-pulvinar pathway might be importantphalogram (EEG), which is the cortical signature for vision early in development (Wallace et al.,of heightened attention and vigilance. These 1997), enabling rapid orienting and vigilanceautonomic responses might be produced via toward sudden, potentially dangerous stimuli,the bidirectional connections of the SC with but may be less efficient for calculated responsesthe PAG, and with the motor and sensory to complex visual arrays. Although blindsightnuclei of the brainstem. The SC may trigger the patients are notable for their lack ofreallocation of visual attention in response to awareness of their intact visual abilities, theseemotionally salient stimuli via connections parallel visual pathways may normally functionwith the amygdala and the mesencephalic reti- under conditions of visual awareness (Pessoacular formation, involved in the elaboration of et al., 2005).eye movements. Indeed, lesions or local infusion Hypothalamusof neurotransmitter antagonists in the SC blockfear-potentiated startle (Waddell et al., 2003; The hypothalamus is a collection of nuclei con-Zhao & Davis, 2004). cerned primarily with homeostasis through Unlike rodents, where the neural circuitry autonomic and endocrine mechanisms, but itand pharmacology of colliculus-dependent also coordinates basic, drive-related behaviorsdefensive and antinociceptive effects have been (e.g., feeding, reproduction, aggression). Manyworked out in detail, little is known about how of these functions, especially those that involvethese pathways function in primates. Recent species-specific ‘‘instinctive’’ behaviors coordi-work indicates that bicuculline-mediated disin- nated by central pattern generators, are redun-hibition of the SC in monkeys increases defen- dant and overlapping with similar functionssive emotional reactivity (Cole et al., 2006) and controlled by the brainstem. A unique contribu-decreases species-specific social behaviors tion of the hypothalamus in emotion is related(Ludise Malkova, personal communication). to hormone production. The arcuate nucleusThis outcome is only partially overlapping with controls neuroendocrine function, thethe outcome of the same manipulations in the supraoptic and paraventricular nuclei release

CIRCUITS OF EMOTION IN THE PRIMATE BRAIN 297oxytocin and vasopressin (hormones of affilia- perceptual, motor, and memory processestion and social bonding), and corticotrophin- associated with emotional stimuli.releasing hormone initiates a cascade of events The subcortical nuclei are connected pri-to supply cortisol in situations of emotional marily with subcortical structures and arestress. involved in attention and autonomic functions. The hypothalamus receives input from the The central and medial nuclei, but also the ante-brainstem and spinal cord via the medial fore- rior amygdaloid area and the bed nucleus of thebrain bundle (Fig. 16.1B). These inputs carry stria terminalis (BNST), are part of the subcor-signals about the state of all internal organs. tical group (Amaral et al., 1992). These nucleiIntrinsic sensory neurons in the hypothalamus receive and send projections from and to theare specialized to sense the composition of the hypothalamus and the brainstem and, via theseinternal milieu and correct reflexively any connections, control autonomic function. Fordeviation from normal. example, the amygdala modulates heart rate Like the brainstem, the hypothalamus con- and cardiorespiratory reflexes via connectionstains networks of neurons that often form pat- to the dorsal motor nucleus of the vagus andtern generators and control behavioral, the nucleus of the solitary tract. Facial expres-endocrine, and regulatory functions. These net- sions are controlled by direct projections to theworks are under the influence of higher centers. lower motor neurons in the facial nucleusFor example, inputs from the amygdala signal (Fanardjian & Manvelyan, 1987). Likewise, sali-the emotional value of a stimulus or event and vation, lacrimation, respiration, pupil dilation,can trigger, when appropriate, the classical stress pain perception, etc., are modulated by directresponse via the hypothalamic-pituitary-adrenal connections with the respective centers of the(HPA) axis. Descending inputs from the amyg- brainstem. Finally, the amygdala can modifydala, anterior cingulate cortex (ACC), and pre- the output of all major neurotransmitter systemsfrontal cortex modulate the role of the originating in the brainstem. Direct projectionshypothalamus in territoriality, aggression, the from the central nucleus to the ventral tegmentaldesirability of food, mates, etc. Reward-related area influence dopamine release (Gallagher,signals from the ventral tegmental area and 2000); projections to the locus coeruleus set thenucleus accumbens, but also from the orbito- level of vigilance and cortical ‘‘preparedness’’ viafrontal cortex (OFC) and anterior cingulate, widespread projections that release norepi-mediate reward-based learning and action selec- nephrine over the entire cortex (Amaral &tion. Inputs from the reticular formation, the Sinnamon, 1977); and direct projections frombasal forebrain (nucleus basalis), and the bed the central nucleus to the nucleus basalis ofnucleus of the stria terminalis (BNST) contri- Meynert influence acetylcholine levels in thebute to the control of attention and vigilance. brain (Davis & Whalen, 2001). Finally, the cen-The majority of these connections are reciprocal, tral nucleus emits projections to the hypotha-closing loops that combine the internal state of lamic areas, where prohormones and releasingthe organism with the external stimuli and factors are synthesized.initiate (or block) basic, drive-related behaviors(Appenzeller & Oribe, 1997). Intermediate Structures Compared to the core, these structures elaborateThe Subcortical Nuclei of the Amygdala the representations of stimuli and stimulus com-The amygdala contains a heterogeneous collec- binations that are evaluated in these circuits andtion of nuclei with dissociable functions. label them with emotional valence. The outputsTogether these nuclei carry out the evaluation of the intermediate structures are higher-dimen-of stimuli, the initiation of autonomic and sional than the outputs of the core structures;somatic responses most appropriate for each they control somatic and autonomic effectorsstimulus, and the modulation of the ‘‘gain’’ of via the core structures. The high-dimensional

298 PRIMATE NEUROETHOLOGYrepresentations generated by these structures are Mediodorsal Nucleus of the Thalamusshared with multiple neocortical areas involved The mediodorsal thalamus (MD) is an obliga-in attention, perception, memory, and decision tory station in the higher-level interconnectionsmaking. Note that intermediate structures do of emotion circuits (Jones, 2007). This nucleusnot overlap with the classical concept of the connects the prefrontal intermediate struc-‘‘limbic system’’ (Maclean, 1949). The inter- tures—the ACC and OFC—to other emotion-mediate structures include the basolateral related areas. Its positioning as a ‘‘nexus’’ for the(cortical) nuclear group of the amygdala, the other intermediate structures may account formediodorsal nucleus of the thalamus, the its known role in object-reward associationinsula, ACC, and the OFC. memory (Gaffan & Parker, 2000).Amygdala, Basolateral Complex The InsulaThe cortical nuclei of the amygdala, also called The insula is a good example of the diversity andthe basolateral complex, contain neurons of the redundancy of functions carried out by thecortical type and receive and send connections intermediate structures in the circuits ofprimarily from and to the neocortex. One role emotion. The insula is both the primary corticalof these nuclei is to evaluate the emotional sig- center of taste and smell and a major source ofnificance of stimuli. These nuclei receive highly autonomic regulation, redundant with the auto-processed sensory information from all sensory nomic functions of the brainstem. In addition,modalities (Amaral et al., 1992; Stefanacci & the insula integrates interoceptive signals, pain,Amaral, 2002). Olfactory signals arrive from and somatosensory stimuli such as various kindsthe piriform cortex; gustatory information of touch, and remaps the surface of the body infrom the insula; somatosensory and auditory, terms of the quality of stimuli (Augustine, 1996).and visual information from association cortices Social stimuli are also processed in the insulain the parietal and temporal lobe. Coarsely pro- (e.g., facial expressions of disgust and the ‘‘trust-cessed sensory information also arrives here worthiness’’ of faces)—stimuli that also activatedirectly from the thalamus (Romanski & the amygdala (Engell et al., 2007; Phillips et al.,LeDoux, 1992). Viscerosensory signals in the 1997; Winston et al., 2002). Based on these richamygdala come from the insula or from the inputs, the insula generates a representation ofnucleus of the solitary tract. Based on the internal state of the body in which somaticthese inputs, which converge with inputs from and visceral components are fused and ulti-the medial and orbital prefrontal cortex, (OFC) mately give rise to a ‘‘feeling of the body’’ asso-the cortical nuclei of the amygdala determine the ciated with empathy, general disposition, andpositive, negative, or neutral significance of all mood stability (Singer et al., 2004).stimuli. The outcome of this evaluation is sent The Anterior Cingulate Cortexback to the cortex via widespread feedback pro-jections (Fig. 16.1A). Output projections reci- Whereas the insula has mainly sensory and auto-procate the inputs but also project to multiple nomic functions, the ACC hosts both executivestages of sensory processing, including primary and cognitive functions based on extero- andsensory areas. These nuclei also project to the interoceptive signals arriving from the insula,subcortical nuclei of the amygdala, which, in brainstem, basal nucleus of the amygdala, andturn, project to autonomic and somatic effec- hypothalamus. In addition, certain areas of thetors. Signals from the amygdala modulate atten- ACC (24, 25, and ventral 32) receive strongtion, perception, and memory and are carried by inputs from the mediodorsal nucleus of the tha-excitatory connections that terminate in layer II lamus. All subregions of the ACC receive inputsof the neocortex. There, they compete for from the anterior thalamic nuclei (Vogt et al.,synaptic sites with adjacent or more distant 1987) that receive, in turn, input from the mam-cortical areas (Freese & Amaral, 2006). millary bodies of the hypothalamus—a

CIRCUITS OF EMOTION IN THE PRIMATE BRAIN 299memory-related structure. The diversity of the amygdala (for review, see Barbas, 2007). Theinputs is mirrored by a diversity of functions intercalated nuclei separate the basolateralwithin the ACC (for review, see Bush et al., (cortical) groups of nuclei from the centrome-2000; Joseph, 2000). dial (subcortical groups) through their Among these functions are the control of GABAergic interneurons. These interneuronsvisceral, skeletal, and endocrine outflow in may inhibit the transfer of signals from the baso-response to emotional stimuli. The outflow of lateral complex to the lateral division of thethe ACC targets the amygdala, periaqueductal central nucleus, thus blocking the initiation ofgray, and several cranial nerve nuclei in the autonomic responses (Pare & Smith, 1993;brainstem (V, VII, IX, and X). Via the Rempel-Clower, 2007). The recently describedoutput to the brainstem nuclei, emotional functions of intercalated nuclei (Likhtik et al.,states set up in the ACC translate into 2008) might explain why a snake in the outdoorschanges of heart rate, blood pressure, and is more fear producing than when experiencedvocalizations associated with expressing internal in a terrarium. In theory, the orbitofrontalstates (e.g., crying, moaning). Note that spe- activity could prevent the basolateral signal ofcies-specific vocalizations are also controlled by danger from activating the central nuclei,the PAG in the brainstem. ‘‘heading it off at the pass,’’ thereby eliminating Like the PAG and the insula, the anterior unnecessary behavioral and autonomiccingulate cortex also functions as a pain center. responses.Compared to the insula, where bodily sensations Collectively, the intermediate structuressuch as pain are mapped according to their sub- show partially overlapping, yet complementaryjective quality, the pain representation in the functions. For example, the similar connectivityanterior cingulate is more abstract. The same of the ACC and OFC suggest that they processareas of the cingulate cortex are activated by reward signals in conjunction with internal andsocial pain (the feeling of being ignored or external stimuli and can influence autonomicrejected) and by physical pain. Finally, mother- and somatic responses that contribute toinfant interactions are also under the influence normal social and emotional behavior. Thereof a division of the anterior cingulate. are important differences, however, between Cognitive functions mediated by the ACC these two areas in terms of their contributioninclude reward anticipation and social decision to the evaluation of stimuli and the elaborationmaking. The control signals for these functions of responses. Whereas the ACC integratesoriginate from dorsal areas of the anterior cin- reward-related signals with species-specificgulate that contain a motor region. The motor motor behaviors, the OFC learns to associateregion influences motor areas of the brainstem and dissociate internal and external stimuli andand spinal cord, thereby contributing to the rewards. Together, they form a unit critical forselection of the most appropriate action pattern. bringing into register the internal and external state of the world with behavioral choices, theOrbitofrontal Cortex building blocks of emotion and social decisionBased on patterns of connectivity, the orbito- making.frontal cortex resembles the amygdala, a struc-ture to which it is also reciprocally connected. Affiliate StructuresHigh-level sensory inputs from all modalitiesconverge there, and the outputs target the core Affiliate regions are primarily neocortical andstructures of the emotion circuits including the are more strongly connected to the intermediatehypothalamus (from which it also receives structures than to the core structures. They areinputs) and brainstem autonomic areas. capable of processing complex, high-dimen- The orbitofrontal cortex may exert control sional signals, linking multiple aspects of emo-over the output of the primate amygdala by tion, such as memory, decision making, andway of projections to the intercalated nuclei of action planning, to ongoing stimuli and events.

300 PRIMATE NEUROETHOLOGYCompared to the core and intermediate struc- volume (Pruessner et al., 2005). Given its cou-tures, their role in generating autonomic and pling with stress responses and the associationreflexive emotional outputs is less direct, often between high stress levels, anxiety, and heigh-demonstrating their involvement only for a spe- tened responses to acute stressors, the hippo-cific set of stimuli and contexts. Though this list campus should be brought back into the fold ofwill surely lengthen as our understanding of emotional circuits, at least when considering theemotion circuits increases, for now, we consider long-term trait influences on emotionthe hippocampus, the lateral temporal lobe, processing.ventrolateral prefrontal cortex, the lateral Lateral Temporal Lobeintraparietal area, and the medial pulvinar. The continuous stretch of cortex from the upperHippocampus bank of the superior temporal sulcus (STS),Historically, both Papez (1937) and MacLean through the lower bank, and onto the inferotem-(1949) included the hippocampus in the emo- poral (IT) gyrus processes complex visual images,tional centers of the ‘‘limbic system.’’ Refined including faces (Bruce et al., 1981; Gross et al.,lesion techniques later suggested that the hippo- 1972; Perrett et al., 1982; Tanaka, 1992; Yamanecampus does not play a major role in emotion; et al., 1988). This area of the temporal cortexhowever, as a central learning and memory receives inputs from ‘‘upstream’’ visual areasstructure, it is at the interface between emotion such as MT, TEO, and V4, and projects to theand memory. Episodic memories that are hip- orbitofrontal cortex, lateral prefrontal cortex,pocampal dependent can have an emotional medial temporal lobe neocortex and amygdalacomponent that is encoded and stored together (Baizer et al., 1991; Felleman & Van Essen, 1991;with the neutral components. Conversely, emo- Seltzer & Pandya, 1978). Neurons in these areastional states are known to influence memory show responses to different dimensions of socialformation, whether by enhancing vigilance and stimuli, such as face identity, facial expression,attention at the time of encoding (Adolphs et al., head or gaze direction, and body movements2005; Easterbrook, 1959) or through the effects (for review, see Rolls, 2007; see also Tsao et al.,of arousal on memory consolidation (McGaugh, this volume). In addition, the upper bank STS2000, 2004). The strong connections of the hip- shows multisensory responses that can combinepocampus (or its gateway structures) with the visual and auditory information, such as commu-amygdala, orbitofrontal cortex, and medial nication signals (Barraclough et al., 2005;prefrontal cortex constitute the anatomical fra- Ghazanfar et al., 2008).mework on which emotional memories are built Some evidence that these areas are important(LaBar & Cabeza, 2006; McGaugh, 2004; for processing socioemotional cues comes fromPackard & Teather, 1998). lesions made in infancy to area TE (inferior tem- An additional role of the hippocampus in poral cortex) that lead to temporary deficits inemotion and memory derives from its high con- socioemotional behavior in infants (Bachevaliercentration of glucocorticoid receptors. This et al., 2001; Malkova et al., 1997b). The STS ismakes the hippocampus vulnerable to the neu- required for normal gaze discrimination, which isrotoxic glucocorticoids that are released in important for interpreting the target of an expres-stressful or highly emotional situations by the sion (Heywood & Cowey, 1992), and an intactHPA axis (Sapolsky, 1996; Watanabe et al., 1992; STS and inferior temporal gyrus (ITG) areWoolley et al., 1990). Chronic stress, marked by required for discrimination of configural changeselevated levels of cortisol, is associated with hip- in faces (Horel, 1993) that are important forpocampal atrophy, reduced neurogenesis, and identification. Taken together, these areas maybehavioral changes ranging from memory loss form a key ‘‘prerequisite’’ filter for extractingto depression. In humans, personality traits such emotion and/or intention from visual socialas self-esteem and an internal locus of control cues such as faces, though the lateral temporalare positively correlated with hippocampal lobe is not the only region poised to play this role.

CIRCUITS OF EMOTION IN THE PRIMATE BRAIN 301Ventrolateral Prefrontal Cortex that triggers the presentation of a given image, the responses predict the value of that image.The ventrolateral prefrontal cortex (vlPFC), spe- Here, value was assessed by the amount of juicecifically, the inferior prefrontal convexity, pro- monkeys would forego to view an image, using acesses social-emotional information from every previously established paradigm (Deaner et al.,sensory modality. This area receives projections 2005). Thus, the LIP area might mediate pro-from high-level visual areas including the infer- cesses by which social cues are embedded in theotemporal cortex (Ungerleider et al., 1989) and decision-making networks in the brain, as wouldcontains neurons that show selective responses be important for action selection during socialto faces. (Scalaidhe et al., 1997; Wilson et al., interactions.1993) that are similar to the responses of face-selective neurons of the lateral temporal areas. Medial PulvinarAdjacent to the visual region is an area that The medial nucleus of the pulvinar is connectedshows selective responses to species-specific to the superior temporal sulcus and gyrus, thevocalizations (Cohen et al., 2004, 2006; amygdala, the insula, orbitofrontal cortex, theRomanski et al., 2005). Selectivity profiles do frontal pole, and medial prefrontal cortexnot fall strictly into featural (tonal/noisy) or including anterior cingulate cortex (Romanskifunctional (food/nonfood, aggressive/ et al., 1997). This list of structures correspondsappeasing) borders, though biases in a neuron’s to the intermediate layer of the emotional circuitresponse toward one or the other category can be proposed here.sufficient to extract this information (Thomas The medial pulvinar, like its neighboringet al., 2001). The inferior prefrontal convexity structure, the mediodorsal nucleus of the tha-was recently shown to engage in multisensory lamus, may contribute to the integration of sig-processing of audio-visual communication signals from nodes of the emotion circuit.nals (Sugihara et al., 2006), providing one clue to The unique roles played by the affiliatethe processing that may occur in this region. regions and their recruitment of widely con-Moreover, with its sensitivity for complex, spe- served core regions may be specializations tocies-specific stimuli, vlPFC activity may ulti- selective pressures that reward social savvy. Themately prove to be an important link between affiliate structures aren’t directly tied to the pro-the perception of social stimuli and the extrac- duction of a specific emotion; rather, they enabletion of their emotional significance. We refer the extraction of appropriate responses to complexreader to the chapters by Miller and Cohen and constellations of cues and scenarios that areRomanski and Ghazanfar in this volume. relevant for responding appropriately to the current social situation. In this way, the boundariesLateral Intraparietal Area segregating structures associated with emotionThe allocation of spatial attention (Bisley and and cognition become blurred (Pessoa, 2008), asGoldberg, 2003) and goal-directed selection of will become clear when considering the contri-actions (Snyder et al., 1997) are two functions bution of the aforementioned structures toascribed to the lateral intraparietal (LIP) area specific emotional processes.(see Groh & Pai, this volume). More recently, ithas also been associated with perceptual decisionmaking (Freedman & Assad, 2006), including PROCESSING EMOTION:enhanced responses when selecting preferred CIRCUITS IN ACTIONsocial cues such as images of dominant males, A basic rule of survival is to approach foodconspecific faces, or female hindquarters (Klein sources and potential mates (appetitive stimuli)et al., 2008). LIP neurons do not respond selec- and avoid danger such as predators (aversivetively in anticipation of social cues when the cues stimuli). Biologically prepared appetitive orare presented in a predictable, obligatory aversive stimuli elicit emotional/motivationalfashion, but when the monkey makes a choice states, and are processed primarily by the core

302 PRIMATE NEUROETHOLOGYof the emotional brain. The structures of the necessary for the expression of unconditioned,core circuits are highly conserved across species, but also of conditioned, startle. Monkeys withbut component neurons are often tuned to lesions of the central nucleus also lack thestimuli with species-specific significance normal apprehension to the sight of a snake(e.g., odors of predators, pheromones, etc.). (real or fake), as determined by the latency toThe appetitive-aversive dichotomy, although retrieve food in its presence (Davis et al., 2008;not mapped onto different circuitry, has Kalin et al., 2004). In addition, monkeys withoutproven to have great heuristic value for deci- a central nucleus fail to demonstrate normalphering the neural circuitry of emotion in freezing (Kalin et al., 2004) that typically accom-animals. panies the appearance of a human intruder. Our most complete understanding to date of Finally, lesions of the central nucleus reducethe most basic and most shared emotion across cortisol levels, indicating that an intact amygdalaall species—fear—comes from aversive condi- is necessary for generation of normal stresstioning in rats. Fear has been studied using an responses.aversive stimulus such as foot shock, often inassociation with a cue such as a tone (Davis & Conditioned Fear ResponsesWhalen, 2001; Fanselow & Gale, 2003; LeDoux,2000). A subset of neurons in the monkey amygdala Approach behaviors, which do not corre- respond selectively to stimuli that predict anspond to a state as clearly defined as fear, have aversive, unconditioned stimulus (US), such asbeen studied using food reinforcement (Everitt a puff of air directed at the face or eyes (Patonet al., 2003; Holland & Gallagher, 2004). In rats, et al., 2006). Neuronal responses to the air puffthe amygdala alone can support fear condi- and the stimuli that predict it (conditioned sti-tioning, whereas appetitive conditioning and muli [CS]) ramp up during learning and fall offreward devaluation paradigms require the joint during extinction, suggesting they may reflectcontribution of the amygdala and frontal cortex. the updated value of the association to the aver- Pavlovian conditioning in primates suggests sive unconditioned stimulus. Although patientsa conservation of neural structures (Baxter & with amygdala damage are able to learn CS-USMurray, 2002) and possibly mechanisms associations, they are unable to express fear or(Salzman et al., 2007) that support this basic the anxious anticipation of the noxious stimulusform of emotional learning. The main difference (Bechara et al., 1995). Moreover, conditionedmight be that emotional processing from the stimulus learning and extinction are associatedcore structures radiates in primates to a larger with changes in blood level oxygen–dependentarray of partially or fully corticalized structures (BOLD) responses in the ventromedial pre-that reprocess emotion and link it to memory, frontal cortex and amygdala of humansplanning, and decision making. The essential ¨ (Buchel et al., 1998; LaBar et al., 1998; Phelpsdifference derives from the major role of social et al., 2004). The following reviews consider thestimuli to elicit emotions in primates. We will fear conditioning literature in more detailconsider first the evidence for aversive and appe- (LeDoux, 2000; Phelps, 2006).titive conditioning in primates before describing The predominance of fear conditioning as ahow social stimulus evaluation is intrinsically model for emotional learning has led to therelated to emotional circuits. implicit assumption that the structures necessary for processing fearful stimuli are specializedAversive Emotional States for fear. To determine whether structures such as the amygdaloid nuclei function as ‘‘fear mod-Unconditioned Fear ules,’’ involving only negative affect, it is impor-Some stimuli are known to produce fearful or tant to demonstrate that they do not act morestartle responses in monkeys (Davis et al., 2008). generally in emotional learning, involving eitherThe central nucleus of the monkey amygdala is positive or negative affect.

CIRCUITS OF EMOTION IN THE PRIMATE BRAIN 303Appetitive Emotional States discrimination task. Yet when a preferred food is devalued, lesions to either the amygdala and/Conditioned Appetitive Responses or the contralateral orbitofrontal cortex lead toClassical conditioning can also be used to study continued selection of objects associated withpositive emotional states by pairing an initially the satiated (devalued) reward, when a normalneutral stimulus, such as a visual stimulus, with response would be to switch, selecting the foodreward. For example, a study in marmoset mon- reward that had not been devalued. Curiously,keys showed that the amygdala is not necessary upon seeing the underlying food reward, onlyfor appetitive conditioning per se (Braesicke monkeys with orbitofrontal lesions actually tooket al., 2005). In this study, the overt behaviors the reward. These monkeys selected the non-indicating anticipation of reward (looking and devalued food when given a choice between thescratching at a food barrier) persisted in amyg- two, indicating that satiation indeed changed thedala-lesioned animals, despite a reduction in the value representation of the food. These monkeysphysiological signs of arousal during that antici- only fail when given a choice between the cuespatory phase. It was as though the habit of predicting the foods, falling back on theirorienting toward a conditioned stimulus learned behaviors to previous conditioned asso-remained, even when the underlying incentive ciations. Having selected the single, devaluedwas no longer present. This suggests that some food, they take it despite satiety for it, unlikeaspects of positive affect require an intact monkeys with amygdala lesions. Thus, both theamygdala. orbitofrontal cortex and the amygdala share a Consistent with this observation, neural role in updating behaviors toward previouslyresponses in the macaque amygdala follow the learned associations based on current prefer-time course of learning the association between ences or motivational state, and the integrity ofan image and juice reward, just as a previously both structures is required for updating thementioned population of amygdala neurons value of a reinforcer (Baxter et al., 2000). These‘‘tracks’’ learning of image-punishment associa- results are also consistent with a study of instru-tions (Paton et al., 2006). For a more detailed mental extinction (Izquierdo & Murray, 2005).examination, we refer the reader to reviews of In contrast to the nuanced distinctions in thethe role of the amygdala in reward and positive roles played by the amygdala and orbitofrontalemotion (Baxter & Murray, 2002; Everitt et al., lesions in encoding the value of reinforcers, the2003; Murray, 2007). anterior insula may be more directly involved in Lesion studies implicate both the amygdala representing the reward value of foods. As inand orbitofrontal cortex in the flexible selection secondary olfactory and primary gustatoryof cues that predict specific rewards, though cortex, cells in the insula show reduced firingeach plays a distinct role in the process. Overall rates to preferred odors or foods following satia-preferences for desirable foods are unchanged tion (Rolls, 2000). When orbitofrontal lesionsafter lesions or disruption of the amygdala include part of agranular insular cortex, mon-(Machado & Bachevalier, 2007b; Malkova keys continue to select foods after being fed toet al., 1997a; Murray & Izquierdo, 2007; satiety (Machado & Bachevalier, 2007a,b). Thus,Wellman et al., 2005) or of the orbitofrontal the insula may be a central source of behavioralcortex (Machado & Bachevalier, 2007b), indi- responses to some rewarding (appetitive)cating that these structures are not necessary to stimuli, such as food.encode the primary reinforcement value of The use of similar appetitive and aversivefoods. Indeed, behavioral responses are appro- conditioning paradigms in primates and inpriately diverted away from food that has been rodents reveals a large overlap in the structuresdevalued through satiation (Machado & involved. This implies a conservation of theBachevalier, 2007a). Moreover, neither structure potential mechanisms of emotional learningis necessary to discriminate which objects are (Everitt et al., 2003; Holland & Gallagher, 2004;associated with food reward in a concurrent Murray, 2007; Phelps, 2006). Although the

304 PRIMATE NEUROETHOLOGYemotional behavior of rodents cannot be suggested in the macaque expression literaturereduced to food and predators, and social sti- (Deputte, 2000; Mason, 1985; Partan, 2002)muli are likely processed by the circuit that eval- are avoidance, which maps directly onto otheruates reinforcers, little is known about the fearful or avoidance behaviors; affiliation,‘‘social brain’’ in rodents (Panksepp, 1998). In which corresponds to appetitive behaviors;primates, however, it is clear that the majority of and the additional pole of aggression, a ‘‘strictlyemotional states are centered on social interac- social’’ addition that arises as a consequence oftions; for many primate species, individuals live dominance status.within elaborate social dominance hierarchies. Generating ExpressionsHere, appropriate responses to members of thegroup can reduce the threat of attack or increase Avoidance (Fear Grimace, Scream) Inaccess to food, reproductive partners, or allies macaques, a cluster of expressions indicate thethat indirectly reduce threats and increase access presence of an aversive stimulus, including theto rewards. Indeed, the majority of emotional fear grimace and scream. The fear grimace isstates in primate species occur during social characterized by horizontally retracted lips,interactions. These socio-emotional interactions revealing upper and lower teeth, and retracteddepend on the identity and dominance status of ears. It can also be accompanied by a high-the participants, as well as on the recent history pitched, shrill vocalization, or ‘‘scream’’of aggression/affiliation, and perhaps most criti- (Partan, 2002).cally on the social signals (facial expressions) Affiliation (Lip Smack, Coo, Groomdisplayed by interacting conspecifics. Present) Expressions indicating a desirable or appetitive stimulus include the lip smack, coo,The Social Envelope: The Relationship and grunt. The lip smack is produced by puck-Between Expressions and Emotions ering the lips and smacking them together, with the chin held up and ears retracted. The coo is aExpressions can be regarded both as the exter- single tonal vocalization made while making annalization of the emotional state of the dis- ‘‘oo’’ shape with the lips. The groom present,player but also as intentional signals aimed at common to many primate species, is a bodyanother individual. At the receiving end, gesture made by exposing a vulnerable part ofexpressions can induce emotional states in the the body by extension, such as by raising theobserver. Importantly, expressions map most arm, turning the head away to expose the neck,closely onto the state of the sender; how it is or turning one’s back, all in close proximity to aninterpreted by an observer will depend on individual invited to groom. The act ofadditional factors. For example, an aggressive grooming that may follow has been used as aexpression indicates the perturbed state of the measurement of the degree of affiliationsender, but if the sender is a meek juvenile (Seyfarth & Cheney, 1984). Affiliative behaviorsmonkey, or if the expression is directed at trigger neural reward states. Grooming isanother individual, it would have little effect accompanied by b-endorphin release in cere-on an adult monkey observer. Thus, we will brospinal fluid, blocking the m-opioid receptorbegin by describing the expressions generated increases groom invitations and grooming dura-by an individual based on that individual’s tions, and opiate delivery decreases groomingemotional state before considering how invitations (Keverne et al., 1989).expressions can evoke emotional responses inobservers. We will focus on macaque expres- Aggression and Alarm (Open-Mouth Stare,sions, allowing us to describe neural structures Pant-Threat, Shrill Bark) The most typicalassociated with the generation and perception expression of aggressive or threatening gesturesof expressions, expanding to incorporate what in the macaque is the open-mouth threatis known in humans or other members of the (or stare), characterized by directed gaze andprimate order, where possible. The three poles an opening of the mouth so that lips form an

CIRCUITS OF EMOTION IN THE PRIMATE BRAIN 305‘‘o’’ shape, with the upper teeth covered by the intact amygdala (Kalin et al., 2001). They alsolips. Ears are forward or are flapped, the head is barked less and showed less submissive behavior.lowered, the eyes are fixated on the receiver of In contrast, when paired with a peer monkey,the threat, and the facial display is often accom- monkeys with neonatal amygdala lesions pro-panied by a body lunge toward the target. The duced more affiliative expressions than intactpant-threat is a staccato noisy (broadband) monkeys (Bauman et al., 2004b). This is consis-vocalization, often occurring as a triplet. tent with the suggestion that the amygdala is notConsistent with the presumed emotional corre- required for the generation of expressions,lates of vocalizations, the shrill bark alarm call regardless of emotional state; rather, it seems toappears to be mediated by stress levels. A reduc- play a role in determining the appropriatetion of stress hormone output leads to fewer gestures in a particular context. We will returnshrill bark vocalizations under stressful to this point in our discussion of the perceptionconditions (Bercovitch et al., 1995). of expressions and their link to emotional state.Neural Basis for the Generation of Aggression In marmosets, stimulation of theExpressions ventromedial hypothalamus produces aggressive vocalizations, and lesions to the anteriorFacial expressions reflect the internal state of hypothalamus or the preoptic area lead to aapproach, avoidance, or aggression of the dis- reduction in aggressive vocalizations toward anplaying animals. Whereas the involvement of intruder (Lloyd & Dixson, 1988). In macaques,some neural structures appears to be restricted stimulation of the anterior hypothalamus, pre-to the generation of a subset of expressions (e.g., optic area, and BNST lead to an increase inaggressive displays), other structures, such as the aggressive vocalizations (Robinson, 1967) andamygdala, are involved in generating a wide aggressive behaviors toward subordinate mon-array of expressions, possibly selecting which keys (Alexander & Perachio, 1973). Thus, properclass of responses is appropriate for a given expression of aggressive behaviors involves, at asituation. minimum, the hypothalamus, BNST, and, asAvoidance Monkeys that were given neonatal previously mentioned, periaqueductal gray andamygdala lesions show more fear responses possibly anterior cingulate cortex.(more grimaces and screams) toward a novel Perceiving and Evaluating Expressionspeer monkey than do intact monkeys (Baumanet al., 2004b), but show fewer fear responses than Although expressions indicate the emotionalintact monkeys do when separated from their state and/or the intentions of an individual, themother (e.g., fewer screams; Bauman et al., dominance hierarchy of macaque societies2004a). Likewise, in a study of young monkeys imposes additional factors on the interpretationwho had just received amygdala lesions, there of an expression, such as the rank, sex, andwas diminished fear expression toward a sti- identity of the sender and of the receiver, asmulus that should have been threatening: the well as the ranks of the close affiliates of both.introduction of a novel adult male (Kalin et al., Recent social exchanges and their outcomes may2001). The ability of these monkeys to respond, also weigh into the interpretation of a givenbut at inappropriate times or with abnormal expression. What is the evidence that monkeysfrequency, suggests that the amygdala may be are sensitive to expressions and to associatedone structure important for appropriate evalua- social factors? In this section, we explore thetion or expression of fearful situations, but not neural substrates for perceiving and properlyfor the generation of the response per se. interpreting expressions.Affiliation When faced with a novel adult When a Threat Is Threatening In general,male (a threatening stimulus), young amyg- when observing a social encounter, an aggressivedala-lesioned monkeys delivered fewer affiliative response is one that escalates an encounter, ulti-vocalizations (coos) than did monkeys with an mately predicting an attack, whereas an

306 PRIMATE NEUROETHOLOGYaffiliative response is one that invites or predicts proximity, and gestures of other individuals, orthose interactions that decrease the probability of a sender’s history of behaviors associated withof attack. An avoidance response is one that an expression, as well as the recent activity of thedisarms or prevents an encounter, leading to a colony. We will explore the evidence for neuralbreak in the interaction. It follows that aggres- sensitivity to these contextual factors and thesive gestures directed at an observer are generally effects they may have on the evaluation ofarousing and threatening to the observer; emotion-laden stimuli.directed affiliative gestures may be arousing, Head and Gaze Directionbut not threatening; and directed avoidancegestures should not be arousing or threatening. Expressions directed away from the observer Images of expressions elicit in the macaque toward an unseen target are ambiguous.brain stimulus-specific responses in several areas Without knowing the target of the expression,of the temporal cortex, as well as in the hippo- the appropriate response is unclear and requirescampus, amygdala, and prefrontal cortices, as additional information gathering, such asshown by imaging studies (Hadj-Bouziane et al., orienting in the direction of display. Indeed,2008; Hoffman et al., 2007; Tsao et al., 2008; see averted expressions in monkeys are moreTsao et al., this volume). Neurons in the infero- arousing than are expressions directed at thetemporal cortex or STS (Hasselmo et al., 1989) observer (Hoffman et al., 2007). Because anand amygdala (Gothard et al., 2007; Kuraoka & important factor in the perception of expres-Nakamura, 2007) respond selectively to facial sions is an assessment of the target of an expres-expressions. In the amygdala, the same fraction sion, the direction of head and gaze of aof neurons respond to threatening, appeasing, or displaying monkey is important to the properneutral stimuli, but the firing rates are higher for interpretation of an expression.threatening faces. The heightened response to Neurons sensitive to body, head, and eye gazethreats is a matter of degree; all types of expres- direction are found in the superior temporalsions can elicit selective amygdala responses sulcus of the temporal lobe (Perrett et al.,(Gothard et al., 2007). Of note, many amygdala 1985), and lesions to this region produce gaze-neurons responded most strongly to a specific discrimination deficits (Heywood & Cowey,combination of identity and expression, not to 1992). In addition, the central nucleus of theexpressions in an identity-invariant manner. This amygdala, which is known to influence auto-would provide a neural means of perceiving nomic output, shows greater BOLD activationexpressions embedded in the appropriate social for averted- than for directed-gaze faces, consis-context, something monkeys with lesions to the tent with physiological measures of arousal inamygdala or orbitofrontal cortex are unable to response to the same stimuli (Hoffman et al.,do. Proper interpretation of an expression is a 2007). The STS sends strong projections to theprerequisite for generating context-appropriate basolateral amygdaloid nuclei, which, in turn,expressions. Recall that the generation of appro- project to the central nucleus, thereby providingpriate expressions was impaired in monkeys with a putative circuit for perceiving and redirectingamygdala lesions (Bauman et al., 2004a,b; Kalin attention based on the intended target of a seenet al., 2001). It’s possible that the process of expression. These results are consistent with aevaluating expressions simply does not occur more general role for the central nucleus in theindependently of the context in which the expres- detection of saliency and the reallocation ofsion is displayed. Here, ‘‘context’’ can be ascer- attention (Whalen, 2007).tained from physical attributes and attitudes of Dominance Rankthe sender that co-occur with the expression,such as gaze and body direction, age, gender, The asymmetrical allocation of valued resourcesand dominance rank, deduced from recognizing such as space, food, and agonistic interactionsthe sender’s identity and physical markers of fit- within groups of macaques is evidence of a dom-ness. Context could also reflect the presence, inance hierarchy, with the commonly studied

CIRCUITS OF EMOTION IN THE PRIMATE BRAIN 307rhesus macaque demonstrating one of the most unlesioned monkeys. Monkeys with neonatallinear, despotic hierarchies of all macaque species amygdala lesions also show enhanced contact,(Flack & de Waal, 2004; Thierry, 2004). in this case with their mothers. These two obser-Sensitivity to social dominance is a pervasive vations support the idea that the amygdala isfacet of macaque behaviors, from its effects on involved in the inhibition of certain types ofthe latency to approach foods to female orgasm behaviors, primarily affiliative behaviors (Sallyrate, which increases with the dominance rank of Mendoza, personal communication). In addi-the males in a pair (Troisi & Carosi, 1998). Even tion, despite the ability to generate the full reper-in an isolated, experimental setting, responses to toire of expressions and vocalizations, duringpictures of familiar conspecifics demonstrate sen- perturbations in the social environment, mon-sitivity to dominance rank. Low-ranking maca- keys with neonatal amygdala lesions generateques will follow the gaze of a monkey looking inappropriate responses, either vocalizing moreaway, regardless of the rank of the stimulus or less than controls, depending on the situation,monkey. High-ranking monkeys, however, will or failing to return to their mothers after forcedonly follow the gaze of other high-ranking mon- separation (Bauman et al., 2004a,b; Kalin et al.,keys (Shepherd et al., 2006). Thus, the aforemen- 2001). In contrast, hippocampal-lesioned mon-tioned neural sensitivity to head and gaze keys showed none of these behaviors. It is notdirection could also be influenced by the status clear what components of the amygdala areof both the observing and stimulus monkeys. responsible for these aberrations, but the resultsConsideration of rank will be important in are consistent with the suspected role of thefuture studies of macaque circuits of emotion. basolateral amygdala in evaluating the proper constellation of cues to update behavioralKin and Affiliation responses. Clearly, more information is neededMacaque mothers differentiate between threats to understand what, if any, additional structuresdirected at them and those directed toward their are implicated in identifying and alteringinfants; the presence of a dominant female is responses to expressions based on kin relation-threatening to both, a subordinate female is a ships and social bonds.threat to her infants, and young daughters are Traits and Statesnonthreatening to both groups (Maestripieri,1995). Understanding the neural substrates for In addition to functional architecture that allowsmacaques’ sensitivity to social status is difficult, the evaluation of stimuli, the emotional circuitshowever; most of our knowledge of social beha- of the brain are influenced by genetic makeupviors comes from field studies devoid of neural and by early life experience that establish themanipulations. Those studies measuring social ‘‘baseline’’ or tonic output even in the absencebehaviors in monkeys following lesions have of a changing input. The temperament,focused on the role of the amygdala. affective style, or personality traits of theQualitative descriptions of the effects of amyg- animal depend on this output. Anxiety-relateddala lesions or lesions extending into other personality traits in monkeys, for example, havemedial temporal lobe regions included with- been linked to the polymorphism of the regula-drawal, submission, and drop in rank when tory region of the gene for transporter-facilitatedplaced in large-group settings (Dicks et al., uptake of serotonin (Suomi, 2006). Secure1969; Rosvold et al., 1954). Somewhat different attachment, or lack thereof, to the mother, com-results were obtained through the quantitative bined with the presence of the short or long allelebehavioral assessment of amygdala-lesioned of the serotonin transporter gene or of themonkeys placed in randomized dyads (Emery monoamine oxidase A gene promoter is predic-et al., 2001). Under these conditions, introduc- tive of risk-taking behaviors, resiliency totion to another, unlesioned monkey produced adverse social situations, and the ability of mon-more affiliative approach behaviors than those keys to control aggression (Newman et al., 2005;seen for introductions between pairs of Suomi, 2006). Individual differences in tonic

308 PRIMATE NEUROETHOLOGYautonomic output or ‘‘trait’’ behaviors will in firing rate for threatening faces compared toinfluence the degree and possibly character of neutral or appeasing faces (Gothard et al., 2007).emotional states evoked by changing inputs. Neurons in the monkey amygdala that areLikewise, keeping track of which individuals selective for facial expressions respond byare highly reactive will assist the interpretation either increasing or decreasing their firing rate.of the likely consequences of a given expression. Positive facial expressions (appeasing faces) are more often encoded by significant decreases of firing rates than threatening faces that causeMultidimensionality and Pluripotency most often significant increases in firing rateWithin Emotion circuits (Gothard et al., 2007). The number of neuronsThe literature on appetitive conditioning allocated to process positive, negative, and evensuggests there are no ‘‘dedicated’’ fear circuits; neutral stimuli is roughly equivalent, and therather, emotional stimuli with negative and global activation is significantly above baselinepositive valence are evaluated by the same or for all facial expressions. There was indeed ahighly overlapping circuits. Likewise, the struc- short-lived excess of population firing rate fortures implicated in the evaluation of social threatening faces, but the size of the differencestimuli overlap, at least partially, with those of between the threatening and appeasing faces wasthe process-conditioned approach/avoidance smaller than the size of responses to any type ofbehaviors, demonstrating a co-opting of facial expressions compared to baseline.approach/avoidance processing into the social Nevertheless, the observed difference betweendomain. In addition, all stimuli with uncertain aggressive and appeasing facial expressions isor ambiguous emotional significance engage the sufficient to account for neuroimaging resultsamygdala and related structures (Whalen, 2007). obtained by subtraction analyses using theThis raises the question: To what extent do same facial expressions (Hoffman et al., 2007).structures within circuits of emotion evaluate Our observation that the monkey amygdala pro-‘‘emotional’’ stimuli exclusively? cesses aversive and appetitive stimuli equally is A comparison of the neural responses elicited in line with earlier neurophysiological and beha-by unfamiliar objects, food items, fear- vioral studies of the monkey amygdala (Fuster &producing objects, and a variety of facial expres- Uyeda, 1971; Nakamura et al., 1992; Ono &sions indicated that all classes of stimuli elicit Nishijo, 1992; Paton et al., 2006; Sangherastimulus-selective response in the monkey et al., 1979; Wilson & Rolls, 1993).amygdala (Gothard et al., 2007). Whether these It appears, therefore, that the monkey amyg-images engage the amygdala because they are dala performs the operations of aversive condi-unfamiliar with ambiguous value or their ‘‘neu- tioning and appetitive conditioning elucidated intrality’’ is encoded as part of a more general rats but, in addition, carries out important socialprocess of stimulus evaluation remains to be processing that is not biased in favor of stimuliclarified. The quasi-equanimous allocation of with a particular emotional valence. If a structurethe processing resources of the amygdala to traditionally placed at the heart of emotionalnegative and positive stimuli is best illustrated processing demonstrates processing of stimuliby the response to social stimuli, such as facial irrespective of emotional valence, other structuresexpressions. Although fearful faces are the least in circuits of emotion may do the same.discernable expression for patients with amyg-dala damage (Adolphs et al., 1994;Sprengelmeyer et al., 1998; but see Rapcsak, CONCLUSIONS2003) and hemodynamic changes in the humanamygdala appear larger in response to fearful 1. A variety of neural structures contribute to theand angry faces than to neutral or happy faces processing of emotions. For didactic purposes,(for review, see Zald, 2003), neurons in the the neural circuits of emotion can be dividedmonkey amygdala show only a small increase into core structures that are the proximal

CIRCUITS OF EMOTION IN THE PRIMATE BRAIN 309 sources of autonomic and motor responses REFERENCES associated with the expression of emotion. Adolphs, R., Tranel, D., & Buchanan, T. W. (2005). These emotional expressions are closely tied Amygdala damage impairs emotional memory to—and in certain cases include—vigilance, for gist but not details of complex stimuli. orienting, and the reallocation of attention to Nature Neuroscience, 8, 512–518. emotionally salient stimuli. Intermediate Adolphs, R., Tranel, D., Damasio, H., & Damasio, structures are those that generate more A. (1994). Impaired recognition of emotion in complex representations that place in facial expressions following bilateral damage to emotional register the external and internal the human amygdala [see comment]. Nature, milieu and flexibly link perceptual and motor 372, 669–672. components of an emotional response. Several Alexander, M., & Perachio, A. A. (1973). The additional structures serve an extended influence of target sex and dominance on emotional circuit, providing further evoked attack in rhesus monkeys. American Journal of Physical Anthropology, 38, 543–547. information about the emotional content of Amaral, D. G., & Sinnamon, H. M. (1977). The stimuli, expected outcomes, and contextual locus coeruleus: Neurobiology of a central contingencies that determine which noradrenergic nucleus. Progress in emotional response should be delivered and Neurobiology, 9, 147–196. under what circumstances. All regions, taken ¨ Amaral, D. G., Price, J. L., Pitkanen, A., & together, afford adaptive responses to changes Carmichael, S. T. (1992). Anatomical in the social environment. organization of the primate amygdaloid2. The amygdala is a key structure for the complex. In: J. P. Aggleton (Ed.), The evaluation of emotional stimuli. Based on amygdala: Neurobiological aspects of emotion, interaction with affiliate structures, the memory, and mental dysfunction (pp. 1–66). amygdala controls the transmission of signals New York: Wiley-Liss. Appenzeller, O., & Oribe, E. (1997). The autonomic toward the core structures that orchestrate the nervous system: An introduction to basic and expressions of emotions. Much of our current clinical concepts (5th ed.). Amsterdam: Elsevier understanding is based on the treatment of the Health Sciences. amygdala as a unitary structure due, in part, to Augustine, J. R. (1996). Circuitry and functional methodological limitations. Nevertheless, the aspects of the insular lobe in primates including subcortical and cortical nuclei, when humans. Brain Research, earch Reviews, 22, scrutinized, fall into distinct roles in 229–244. emotional circuits on both anatomical and Bachevalier, J., Malkova, L., & Mishkin, M. (2001). functional grounds. Effects of selective neonatal temporal lobe3. Emotional responses can be broken down into lesions on socioemotional behavior in infant the immediate evaluation and response to rhesus monkeys (Macaca mulatta). Behavioral Neuroscience, 115, 545–559. changes in the environment and to long-term Baizer, J. S., Ungerleider, L. G., & Desimone, R. modifications that can shape future responses, (1991). Organization of visual inputs to the or even regulatory states of the animal (e.g., inferior temporal and posterior parietal cortex stress hormone levels). Whereas core and in macaques. Journal of Neuroscience, 11, intermediate areas may afford rapid 168–190. responses, the high-dimensional information Bandler, R., Keay, K. A., Floyd, N., & Price, J. available to intermediate and affiliate areas (2000). Central circuits mediating patterned autonomic activity during active vs. passive may be useful for constructing multiple emotional coping. Brain Research Bulletin,. 53, contingencies for subtly varying sets of 95–104. inputs. They may also support memory for Barbas, H. (2007). Flow of information for these contingencies and set into motion emotions through temporal and orbito- longer-term changes, thereby shaping both frontal pathways. Journal of Anatomy, 211, traits and states related to emotions. 237–249.

310 PRIMATE NEUROETHOLOGYBarraclough, N. E., Xiao, D., Baker, C. I., macaque. Journal of Neurophysiology, 46, Oram, M. W., & Perrett, D. I. (2005). 369–384. Integration of visual and auditory information ¨ Buchel, C., Morris, J., Dolan, R. J., & Friston, K. J. by superior temporal sulcus neurons responsive (1998). Brain systems mediating aversive to the sight of actions. Journal of Cognitive conditioning: An event-related fMRI study. Neuroscience, 17, 377–391. Neuron, 20, 947–957.Bartels, A., & Zeki, S. (2004). The neural correlates Bush, G., Luu, P., & Posner, M. I. (2000). of maternal and romantic love. NeuroImage, 21, Cognitive and emotional influences in 1155–1166. anterior cingulate cortex. Trends in CognitiveBauman, M. D., Lavenex, P., Mason, W. A., Sciences, 4, 215–222. Capitanio, J. P., & Amaral, D. G. (2004a). The Cohen, Y. E., Hauser, M. D., & Russ, B. E. (2006). development of mother-infant interactions Spontaneous processing of abstract categorical after neonatal amygdala lesions in rhesus information in the ventrolateral prefrontal monkeys. Journal of Neuroscience, 24, 711–721. cortex. Biology Letters, 2, 261–265.Bauman, M. D., Lavenex, P., Mason, W. A., Cohen, Y. E., Russ, B. E., Gifford, G. W., 3rd, Capitanio, J. P., & Amaral, D. G. (2004b). The Kiringoda, R., & MacLean, K. A. (2004). development of social behavior following Selectivity for the spatial and nonspatial neonatal amygdala lesions in rhesus monkeys. attributes of auditory stimuli in the Journal of Cognitive Neuroscience, 16, ventrolateral prefrontal cortex. Journal of 1388–1411. Neuroscience, 24, 11307–11316.Baxter, M. G., & Murray, E. A. (2002). The Cole, C. E., Gale, J. T., Gale, K., Holmes, A. L., amygdala and reward. Nature Reviews Malkova, L., & Zarbalian, G. (2006). GABA Neuroscience, 3, 563–573. receptor manipulation in the primate deepBaxter, M. G., Parker, A., Lindner, C. C., layers of superior colliculus: Effects on Izquierdo, A. D., & Murray, E. A. (2000). emotional behavior. In: Neuroscience Meeting Control of response selection by reinforcer Planner; Society for Neuroscience, 2006. value requires interaction of amygdala and Craig, A. D. (2002). How do you feel? orbital prefrontal cortex. Journal of Interoception: The sense of the physiological Neuroscience, 20, 4311–4319. condition of the body. Nature ReviewsBechara, A., Tranel, D., Damasio, H., Adolphs, R., Neuroscience, 3(8), 655–666. Rockland, C., & Damasio, A. R. (1995). Double Davis, M., & Whalen, P. J. (2001). The amygdala: dissociation of conditioning and declarative Vigilance and emotion. Molecular Psychiatry, 6, knowledge relative to the amygdala and 13–34. hippocampus in humans. Science, 269, Davis, M., Antoniadis, E. A., Amaral, D. G., & 1115–1118. Winslow, J. T. (2008). Acoustic startle reflex inBercovitch, F. B., Hauser, M. D., & Jones, J. H. rhesus monkeys: A review. Reviews in (1995). The endocrine stress response and Neurosciences, 19, 171–185. alarm vocalizations in rhesus macaques. Davis, P. J., Zhang, S. P., Winkworth, A., & Animal Behaviour, 49, 1703–1706. Bandler, R. (1996). Neural control ofBisley, J. W., & Goldberg, M. E. (2003). Neuronal vocalization: Respiratory and emotional activity in the lateral intraparietal area and influences. Journal of Voice, 10, 23–38. spatial attention. Science, 299, 81–86. de Gelder, B., Vroomen, J., Pourtois, G., &Blessing, W. W. (1997). The lower brainstem and Weiskrantz, L. (1999). Non-conscious bodily homeostasis. New York: Oxford recognition of affect in the absence of striate University Press. cortex. Neuroreport, 10, 3759–3763.Braesicke, K., Parkinson, J. A., Reekie, Y., Man, Dean, P., Redgrave, P., & Westby, G. W. (1989). M-S., Hopewell, L., Pears, A., et al. (2005). Event or emergency? Two response systems in Autonomic arousal in an appetitive context in the mammalian superior colliculus. Trends in primates: A behavioural and neural analysis. Neuroscience, 12, 137–147. European Journal of Neuroscience, 21, 1733–1740. Deaner, R. O., Khera, A. V., & Platt, M. L. (2005).Bruce, C., Desimone, R., & Gross, C. G. (1981). Monkeys pay per view: Adaptive valuation of Visual properties of neurons in a polysensory social images by rhesus macaques. Current area in superior temporal sulcus of the Biology, 15, 543–548.

CIRCUITS OF EMOTION IN THE PRIMATE BRAIN 311Deputte, B. (2000). Primate socialization revisited: to visual cortical areas TE and V1 in the Theoretical and practical issues in social macaque monkey. Journal of Comparative ontogeny. Advances in the Study of Behavior, Neurology, 496, 655–667. 29, 99–157. Fuster, J. M., & Uyeda, A. A. (1971). ReactivityDicks, D., Myers, R. E., & Kling, A. (1969). Uncus of limbic neurons of the monkey to and amygdala lesions: Effects on social behavior appetitive and aversive signals. Electroence- in the free-ranging rhesus monkey. Science, phalography and Clinical Neurophysiology, 165, 69–71. 30, 281–293.Easterbrook, J. A. (1959). The effect of emotion on Gaffan, D., & Parker, A. (2000). Mediodorsal cue utilization and the organization of thalamic function in scene memory in rhesus behavior. Psychological Review, 66, 183–201. monkeys. Brain, 123, 816–827.Emery, N. J., Capitanio, J. P., Mason, W. A., Gallagher, M. (2000). The amygdala and Machado, C. J., Mendoza, S. P., & Amaral, associative learning. In: J. P. Aggleton (Ed.), D. G. (2001). The effects of bilateral lesions of The amygdala: A functional analysis the amygdala on dyadic social interactions in (pp. 311–330). Oxford: Oxford University rhesus monkeys (Macaca mulatta). Behavioral Press. Neuroscience, 115, 515–544. Ghazanfar, A. A., Chandrasekaran, C., &Engell, A. D., Haxby, J. V., & Todorov, A. (2007). Logothetis, N. K. (2008). Interactions between Implicit trustworthiness decisions: Automatic the superior temporal sulcus and auditory coding of face properties in the human cortex mediate dynamic face/voice integration amygdala. Journal of Cognitive Neuroscience, in rhesus monkeys. Journal of Neuroscience, 28, 19, 1508–1519. 4457–4469.Everitt, B. J., Cardinal, R. N., Parkinson, J. A., & Gothard, K. M., Battaglia, F. P., Erickson, C. A., Robbins, T. W. (2003). Appetitive behavior: Spitler, K. M., & Amaral, D. G. (2007). Neural Impact of amygdala-dependent mechanisms of responses to facial expression and face identity emotional learning. Annals of the New York in the monkey amygdala. Journal of Academy of Sciences, 985, 233–250. Neurophysiology, 97, 1671–1683.Fanardjian, V. V., & Manvelyan, L. R. (1987). Gregg, T., & Siegel, A. (2001). Brain structures and Mechanisms regulating the activity of facial neurotransmitters regulating aggression in cats: nucleus motoneurons-III. Synaptic influences Implications for human aggression. Progress in from the cerebral cortex and subcortical Neuropsychopharmacology, Biology, and structures. Neuroscience, 835–843. Psychiatry, 25, 91–140.Fanselow, M. S., & Gale, G. D. (2003). The Gross, C. G., Rocha-Miranda, C. E., & Bender, amygdala, fear, and memory. In: The amygdala D. B. (1972). Visual properties of neurons in in brain function: Basic and clinical approaches inferotemporal cortex of the Macaque. Journal (pp. 125–134). Oxford: Oxford University of Neurophysiology, 35, 96–111. Press. Hadj-Bouziane, F., Bell, A. H., Knusten, T. A.,Felleman, D. J., & Van Essen, D. C. (1991). Ungerleider, L. G., & Tootell, R. B. H. Distributed hierarchical processing in the (2008). Perception of emotional expressions is primate cerebral cortex. Cerebral Cortex, 1, independent of face selectivity in monkey 1–47. inferior temporal cortex. Proceedings of theFlack, J. C., & de Waal, F. B. M. (2004). Dominance National Academy of Sciences, 105, 5591–5596. style, social power, and conflict management: Hasselmo, M. E., Rolls, E. T., & Baylis, G. C. (1989). A conceptual framework In: B. Thierry, M. The role of expression and identity in the face- Singh, & W. Kaumanns (Eds.), Macaque selective responses of neurons in the temporal societies: A model for the study of social visual cortex of the monkey. Behavioural Brain organization (pp. 157–181). Cambridge: Research, earch, 32, 203–218. Cambridge University Press. Heywood, C. A., & Cowey, A. (1992). The role ofFreedman, D. J., & Assad, J. A. (2006). Experience- the ‘face-cell’ area in the discrimination and dependent representation of visual categories in recognition of faces by monkeys. Philosophical parietal cortex. Nature, 443, 85–88. Transactions of the Royal Society of London PartFreese, J. L., & Amaral, D. G. (2006). Synaptic B: Biological Sciences, 335, 31–37; discussion organization of projections from the amygdala 37–38.

312 PRIMATE NEUROETHOLOGYHoffman, K. L., Gothard, K. M., Schmid, M. C., & LaBar, K. S., & Cabeza, R. (2006). Cognitive Logothetis, N. K. (2007). Facial-expression and neuroscience of emotional memory. Nature gaze-selective responses in the monkey Reviews Neuroscience, 7, 54–64. amygdala. Current Biology, 17, 766–772. LaBar, K. S., Gatenby, J. C., Gore, J. C., LeDoux,Holland, P. C., & Gallagher, M. (2004). J. E., & Phelps, E. A. (1998). Human amygdala Amygdala-frontal interactions and reward activation during conditioned fear acquisition expectancy. Current Opinion in Neurobiology, and extinction: A mixed-trial fMRI study. 14, 148–155. Neuron, 20, 937–945.Horel, J. A. (1993). Retrieval of a face LeDoux, J. (2000). Emotion circuits in the brain. discrimination during suppression of monkey Annual Review of Neuroscience, 23,155–184. temporal cortex with cold. Neuropsychologia, Liddell, B. J., Brown, K. J., Kemp, A. H., Barton, 31, 1067–1077. M. J., Das, P., Peduto, A., et al. (2005). A directIzquierdo, A., & Murray, E. A. (2005). Opposing brainstem-amygdala-cortical ‘alarm’ system for effects of amygdala and orbital prefrontal cortex subliminal signals of fear. Neuroimage, 24, lesions on the extinction of instrumental 235–243. responding in macaque monkeys. European Likhtik, E., Popa, D., Apergis-Schoute, J., Fidacaro, Journal of Neuroscience, 22, 2341–2346. G. A., & Pare, D. (2008). Amygdala intercalatedJones, E. G. (Ed.). (2007). The thalamus (2nd ed.). neurons are required for expression of fear Cambridge: Cambridge University Press. extinction. Nature, 454, 642–645.Joseph, R. (2000). Cingulate gyrus. In: R. Joseph Lloyd, S. A., & Dixson, A. F. (1988). Effects of (Ed.), Neuropsychiatry, neuropsychology, clinical hypothalamic lesions upon the sexual and neuroscience (3rd ed.). New York: Academic social behaviour of the male common Press. marmoset (Callithrix jacchus). Brain Research, ¨Jurgens, U. (1994). The role of the periaqueductal 463, 317–329. grey in vocal behaviour. Behavioural Brain Machado, C. J., & Bachevalier, J. (2007a). The Research, earch, 62, 107–117. effects of selective amygdala, orbital frontalKalin, N. H., Shelton, S. E., & Davidson, R. J. cortex or hippocampal formation lesions on (2004). The role of the central nucleus of the reward assessment in nonhuman primates. amygdala in mediating fear and anxiety in European Journal of Neuroscience, 25, the primate. Journal of Neuroscience, 24, 2885–2904. 5506–5515. Machado, C. J., & Bachevalier, J. (2007b).Kalin, N. H., Shelton, S. E., Davidson, R. J., & Kelley, Measuring reward assessment in a semi- A. E. (2001). The primate amygdala mediates naturalistic context: The effects of selective acute fear but not the behavioral and amygdala, orbital frontal or hippocampal physiological components of anxious tempera- lesions. Neuroscience, 148, 599–611. ment. Journal of Neuroscience, 21, 2067–2074. Maclean, P. D. (1949). Psychosomatic disease andKeay, K. A., Redgrave, P., & Dean, P. (1988). the ‘‘visceral brain’’: Recent developments Cardiovascular and respiratory changes elicited bearing on the Papez theory of emotion. by stimulation of rat superior colliculus. Brain Psychosomatic Medicine, 11, 338–353. Research, earch Bulletin, 20, 13–26. Maestripieri, D. (1995). Assessment of danger toKeverne, E. B., Martensz, N. D., & Tuite, B. themselves and their infants by rhesus macaque (1989). Beta-endorphin concentrations in (Macaca mulatta) mothers. Journal of cerebrospinal fluid of monkeys are influenced Comparative Psychology, 109, 416–420. by grooming relationships. Psychoneuro- Malkova, L., Gaffan, D., & Murray, E. A. (1997a). endocrinology, 14, 155–161. Excitotoxic lesions of the amygdala fail toKlein, J. T., Deaner, R. O., & Platt, M. L. (2008). produce impairment in visual learning for Neural correlates of social target value in auditory secondary reinforcement but interfere macaque parietal cortex. Current Biology, 18, with reinforcer devaluation effects in rhesus 419–424. monkeys. Journal of Neuroscience, 17, 6011–6020.Kuraoka, K., & Nakamura, K. (2007). Responses Malkova, L., Mishkin, M., Suomi, S. J., & of single neurons in monkey amygdala to Bachevalier, J. (1997b). Socioemotional facial and vocal emotions. Journal of behavior in adult rhesus monkeys after early Neurophysiology, 97, 1379–1387. versus late lesions of the medial temporal lobe.

CIRCUITS OF EMOTION IN THE PRIMATE BRAIN 313 Annals of the New York Academy of Sciences, 807, Packard, M. G., & Teather, L. A. (1998). Amygdala 538–540. modulation of multiple memory systems:Mason, W. A. (1985). Experiential influences Hippocampus and caudate-putamen. on the development of expressive behaviors Neurobiology of Learning and Memory, 69, in rhesus monkeys. In: G. Zivin (Ed.), 163–203. The development of expressive behavior: Panksepp, J. (1998). Affective neuroscience: The Biology-environment interactions (pp. 117–152). foundations of human and animal emotions. New York: Academic Press. New York: Oxford University Press.McGaugh, J. L. (2000). Memory – a century of Papez, J. W. (1937). A proposed mechanism of consolidation. Science, 287, 248–251. emotion. Journal of Neuropsychiatry andMcGaugh, J. L. (2004). The amygdala modulates Clinical Neuroscience, 7, 103–112. the consolidation of memories of emotionally Pare, D., & Smith, Y. (1993). Distribution of GABA arousing experiences. Annual Review of immunoreactivity in the amygdaloid complex Neuroscience, 27, 1–28. of the cat. Neuroscience, 57, 1061–1076.Morris, J. S., Ohman, A., & Dolan, R. J. Partan, S. R. (2002). Single and multichannel signal (1999). A subcortical pathway to the right composition: Facial expressions and amygdala mediating ‘‘unseen’’ fear. vocalizations of rhesus macaques (Macaca Proceedings of the National Academy of mulatta). Behaviour, 139, 993–1027. Sciences, 96, 1680–1685. Paton, J. J., Belova, M. A., Morrison, S. E., &Morris, J. S., DeGelder, B., Weiskrantz, L., & Dolan, Salzman, C. D. (2006). The primate amygdala R. J. (2001). Differential extrageniculostriate represents the positive and negative value of and amygdala responses to presentation of visual stimuli during learning. Nature, 439, emotional faces in a cortically blind field. 865–870. Brain, 124, 1241–1252. Perrett, D. I., Rolls, E. T., & Caan, W. (1982). VisualMurray, E. A. (2007). The amygdala, reward and neurones responsive to faces in the monkey emotion. Trends in Cognitive Sciences, 11, temporal cortex. Experimental Brain Research, 489–497. 47, 329–342.Murray, E. A., & Izquierdo, A. (2007). Perrett, D. I., Smith, P. A., Potter, D. D., Mistlin, Orbitofrontal cortex and amygdala A. J., Head, A. S., Milner, A. D., et al. (1985). contributions to affect and action in primates. Visual cells in the temporal cortex sensitive to Annals of the New York Academy of Sciences, face view and gaze direction. Proceedings of the 1121, 273–296. Royal Society of London – Series B: BiologicalNakamura, K., Mikami, A., & Kubota, K. (1992). Sciences, 223, 293–317. Activity of single neurons in the monkey Pessoa, L. (2008). On the relationship between amygdala during performance of a visual emotion and cognition. Nature Review in discrimination task. Journal of Neuroscience, 9, 148–158. Neurophysiology, 67, 1447–1463. Pessoa, L., Japee, S., & Ungerleider, L. G. (2005).Newman, T. K., Syagailo, Y. V., Barr, C. S., Visual awareness and the detection of fearful Wendland, J. R., Champoux, M., Graessle, M., faces. Emotion, 5, 243–247. et al. (2005). Monoamine oxidase A gene Phelps, E. A. (2006). Emotion and cognition: promoter variation and rearing experience Insights from studies of the human amygdala. influences aggressive behavior in rhesus Annual Review of Psychology, 57, 27–53. monkeys. Biological Psychiatry, 57, 167–172. Phelps, E. A., Delgado, M. R., Nearing, K. I., &O. Scalaidhe, S. P., Wilson, F. A., & Goldman- LeDoux, J. E. (2004). Extinction learning in Rakic, P. S. (1997). Areal segregation of face- humans. Neuron, 43, 897–905. processing neurons in prefrontal cortex. Phillips, M. L., Young, A. W., Senior, C., Brammer, Science, 278, 1135–1138. M., Andrew, C., Calder, A. J., et al. (1997).Ono, T., & Nishijo, H. (1992). Neurophysiological A specific neural substrate for perceiving facial basis of the Kluver-Bucy syndrome: Responses expressions of disgust. Nature, 389, 495–498. of monkey amygdaloid neurons to biologically Pruessner, J. C., Baldwin, M. W., Dedovic, K., significant objects. Amygdala: Neurobiological Renwick, R., Mahani, N. K., Lord, C., et al. Aspects of Emotion, Memory, and Mental (2005). Self-esteem, locus of control, Dysfunction, 167–190. hippocampal volume, and cortisol regulation

314 PRIMATE NEUROETHOLOGY in young and old adulthood. Neuroimage, 28, Seltzer, B., & Pandya, D. N. (1978). Afferent 815–826. cortical connections and architectonics of theRapcsak, S. Z. (2003). Face memory and its superior temporal sulcus and surrounding disorders. Current Neurology and Neuroscience cortex in the rhesus monkey. Brain Research, Reports, 3, 494–501. 149, 1–24.Redgrave, P., McHaffie, J. G., & Stein, B. E. (1996). Seyfarth, R. M., & Cheney, D. L. (1984). Grooming, Nociceptive neurones in rat superior colliculus. alliances and reciprocal altruism in vervet I. Antidromic activation from the contralateral monkeys. Nature, 308, 541–543. predorsal bundle. Experimental Brain Research, Shepherd, S. V., Deaner, R. O., & Platt, M. L. 109, 185–196. (2006). Social status gates social attention inRempel-Clower, N. L. (2007). Role of orbitofrontal monkeys. Current Biology, 16, R119–R120. cortex connections in emotion. Annals of the Singer, T., Seymour, B., O’Doherty, .J, Kaube, New York Academy of Sciences, 1121, 72–86. H., Dolan, R. J., & Frith, C. D. (2004).Robinson, B. W. (1967). Vocalization evoked Empathy for pain involves the affective but from forebrain in Macaca mulatta. not sensory components of pain. Science, Physiology & Behavior, 2, 345–346, IN341– 303, 1157–1162. IN344, 347–354. Snyder, L. H., Batista, A. P., & Andersen, R. A.Rolls, E. T. (2000). The orbitofrontal cortex and (1997). Coding of intention in the posterior reward. Cerebral Cortex, 10, 284–294. parietal cortex. Nature, 386, 167–170.Rolls, E. T. (2007). The representation of Sparks, D. L., & Hartwich-Young, R. (1989). The information about faces in the temporal and deep layers of the superior colliculus. Review of frontal lobes. Neuropsychologia, 45, 124–143. Oculomotor Research, 3, 213–255.Romanski, L. M., & LeDoux, J. E. (1992). Sprengelmeyer, R., Rausch, M., Eysel, U. T., & Equipotentiality of thalamo-amygdala and Przuntek, H. (1998). Neural structures thalamo-cortico-amygdala circuits in auditory associated with recognition of facial fear conditioning. Journal of Neuroscience, 12, expressions of basic emotions. Proceedings of 4501–4509. the Royal Society of London – Series B:Romanski, L. M., Averbeck, B. B., & Diltz, M. Biological Sciences, 265, 1927–1931. (2005). Neural representation of vocalizations Stefanacci, L., & Amaral, D. G. (2002). Some in the primate ventrolateral prefrontal cortex. observations on cortical inputs to the macaque Journal of Neurophysiology, 93, 734–747. monkey amygdala: An anterograde tracingRomanski, L. M., Giguere, M., Bates, J. F., & study. Journal of Comparative Neurology, 451, Goldman-Rakic, P. S. (1997). Topographic 301–323. organization of medial pulvinar connections Stoerig, P., & Cowey, A. (1997). Blindsight in man with the prefrontal cortex in the rhesus and monkey. Brain, 120, 535–559. monkey. Journal of Comparative Neurology, Sugihara, T., Diltz, M. D., Averbeck, B. B., & 379, 313–332. Romanski, L. M. (2006). Integration ofRosvold, H. E., Mirsky, A. F., & Pribram, K. H. auditory and visual communication (1954). Influence of amygdalectomy on social information in the primate ventrolateral behavior in monkeys. Journal of Comparative prefrontal cortex. Journal of Neuroscience, 26, and Physiological Psychology, 47, 173–178. 11138–11147.Salzman, C. D., Paton, J. J., Belova, M. A., & Suomi, S. J. (2006). Risk, resilience, and gene X Morrison, S. E. (2007). Flexible neural environment interactions in rhesus monkeys. representations of value in the primate brain. Annals of the New York Academy of Sciences, Annals of the New York Academy of Sciences, 1094, 52–62. 1121, 336–354. Tanaka, K. (1992). Infereotemporal cortex andSanghera, M. K., Rolls, E. T., & Roper-Hall, A. higher visual functions. Current Opinion in (1979). Visual responses of neurons in the Neurobiology, 2, 502–505. dorsolateral amygdala of the alert monkey. Thierry, B. (2004). Social epigenesis In: B. Thierry, Experimental Neurology, 63, 610–626. M. Singh, & W. Kaumanns (Eds.), MacaqueSapolsky, R. M. (1996). Why stress is bad for your societies: A model for the study of social brain.(overproduction of glucocorticoids organization (pp. 267–289). Cambridge: damage the hippocampus). Science, 273, 749. Cambridge University Press.

CIRCUITS OF EMOTION IN THE PRIMATE BRAIN 315Thomas, E., Van Hulle, M. M., & Vogels, R. hippocampal CA3 pyramidal neurons. Brain (2001). Encoding of categories by Research, 588, 341–345. noncategory-specific neurons in the inferior Wellman, L. L., Gale, K., & Malkova, L. (2005). temporal cortex. Journal of Cognitive GABAA-mediated inhibition of basolateral Neuroscience, 13, 190–200. amygdala blocks reward devaluation inTroisi, A., & Carosi, M. (1998). Female orgasm rate macaques. Journal of Neuroscience, 25, increases with male dominance in Japanese 4577–4586. macaques. Animal Behaviour, 56, 1261–1266. Whalen, P. J. (2007). The uncertainty of it all.Tsao, D. Y., Schweers, N., Moeller, S., & Freiwald, Trends in Cognitive Sciences, 11, 499–500. W. A. (2008). Patches of face-selective cortex in Wilson, F. A., & Rolls, E. T. (1993). The effects of the macaque frontal lobe. Nature Neuroscience, stimulus novelty and familiarity on neuronal 11, 877–879. activity in the amygdala of monkeysUngerleider, L. G., Gaffan, D., & Pelak, V. S. performing recognition memory tasks. (1989). Projections from inferior temporal Experimental Brain Research, 93, 367–382. cortex to prefrontal cortex via the uncinate Wilson, F. A., Scalaidhe, S. P., & Goldman-Rakic, fascicle in rhesus monkeys. Experimental Brain P. S. (1993). Dissociation of object and spatial Research, 76, 473–484. processing domains in primate prefrontalVogt, B. A., Pandya, D. N., & Rosene, D. L. (1987). cortex. Science, 260, 1955–1958. Cingulate cortex of the rhesus monkey: Winston, J. S., Strange, B. A., O’Doherty, J., & I. Cytoarchitecture and thalamic afferents. Dolan, R. J. (2002). Automatic and intentional Journal of Comparative Neurology, 262, brain responses during evaluation of 256–270. trustworthiness of faces. Nature Neuroscience,von Euler, C., & Trippenbach, T. (1976). 5, 277–283. Excitability changes of the inspiratory ‘‘off- Woolley, C. S., Gould, E., & McEwen, B. S. (1990). switch’’ mechanism tested by electrical Exposure to excess glucocorticoids alters stimulation in nucleus parabrachialis in the dendritic morphology of adult hippocampal cat. Acta Physiologica Scandinavia, 97, 175–188. pyramidal neurons. Brain Research, 531,Vuilleumier, P., Armony, J. L., Driver, J., & 225–231. Dolan, R. J. (2003). Distinct spatial Yamane, S., Kaji, S., & Kawano, K. (1988). frequency sensitivities for processing faces What facial features activate face neurons and emotional expressions. Nature in the inferotemporal cortex of the Neuroscience, 6, 624–631. monkey? Experimental Brain Research, 73,Waddell, J., Heldt, S., & Falls, W. A. (2003). 209–214. Posttraining lesion of the superior colliculus Young, M. P., Scannell, J. W., Burns, G. A., & interferes with feature-negative discrimination Blakemore, C. (1994). Analysis of connectivity: of fear-potentiated startle. Behavioral Brain Neural systems in the cerebral cortex. Review of Research, 142, 115–124. Neuroscience, 5, 227–250.Wallace, M. T., McHaffie, J. G., & Stein, B. E. Zald, D. H. (2003). The human amygdala and the (1997). Visual response properties and emotional evaluation of sensory stimuli. Brain visuotopic representation in the newborn Research Reviews, 41, 88–123. monkey superior colliculus. Journal of Zhao, Z., & Davis, M. (2004). Fear-potentiated Neurophysiology, 78, 2732–2741. startle in rats is mediated by neurons in theWard, R., Danziger, S., & Bamford, S. (2005). deep layers of the superior colliculus/deep Response to visual threat following damage to mesencephalic nucleus of the rostral the pulvinar. Current Biology, 15, 571–573. midbrain through the glutamate non-Watanabe, Y., Gould, E., & McEwen, B. S. (1992). NMDA receptors. Journal of Neuroscience, Stress induces atrophy of apical dendrites of 24, 10326–10334.

CHAPTER 17Neurophysiological Correlates of Reward LearningWolfram SchultzLEARNING AS CHANGE IN The behavioral reactions studied classicallyPREDICTIONS by Pavlov are vegetative responses governed by smooth muscle contraction and gland discharge,Predictions whereas more recent Pavlovian tasks also involveLearning is manifested as reproducible change in reactions of striated muscles. In the latter case,behavior in identical situations and is a major the final reward usually needs to be collected byfactor in the behavioral adaptation to changing an instrumental contraction of striated muscle,environments. Many forms of behavior are based but the behavioral reaction to the conditionedon anticipating the future by predicting the out- stimulus itself is not required for the reward tocomes, such as rewards and punishers. Changed occur (e.g., anticipatory licking). Thus, indivi-environmental situations lead to changed predic- dual stimuli in instrumental tasks come to pre-tions, which then induce changes in behavior. dict rewards, and the acquisition of this rewardThus, learning can be viewed as behavioral prediction is considered to be Pavlovian condi-changes that follow changes in predictions. These tioned. In this sense Pavlovian conditioning canconsiderations are particularly pertinent for con- be broadly defined as the acquisition of outcomeditioning or associative learning for motivational predictions, irrespective of the specific beha-outcomes, such as rewards and punishers. vioral reaction being evoked. These distinctions Rewards are positive outcomes that induce are helpful when searching for neural mechan-changes in observable behavior and serve as posi- isms of reward prediction.tive reinforcers by increasing the frequency of the Conditioning is governed by contiguity andbehavior that results in reward. In Pavlovian, or contingency. Contiguity refers to the requirementclassical, conditioning, the outcome follows of near simultaneity (Fig. 17.1a). Specifically, athe conditioned stimulus irrespective of any beha- reward needs to follow a conditioned stimulus orvioral reaction, and repeated pairing of stimuli response by an optimal interval of a few seconds,with outcomes leads to a representation of whereas rewards occurring before a stimulus orthe outcome that is evoked by the stimulus response do not contribute to learning (no orand elicits the behavioral reaction. By contrast, poor backward conditioning). Contingencyinstrumental, or operant, conditioning requires describes that a reward needs to occur more fre-the subject to execute a behavioral response; quently in the presence of a stimulus as comparedwithout such response there will be no reward. to its absence in order to induce ‘‘excitatory’’Instrumental conditioning increases the frequency conditioning of the stimulus (Fig. 17.1b); theof those behaviors that are followed by reward by occurrence of the conditioned stimulus predictsreinforcing stimulus-response links. Instrumental a higher probability of reward compared to noconditioning allows subjects to influence their stimulus, and the stimulus becomes a rewardenvironment and determine their rate of reward. predictor. By contrast, if a reward occurs less 316

NEUROPHYSIOLOGICAL CORRELATES OF REWARD LEARNING 317 A Contiguity CS react US DRIVE B Contingency C Prediction error 1.0 Learning p (reward | stimulus) Associative strength 0.8 0.6 0.4 Conditioned 0.2 inhibition ∆V = α β(λ = V) 0.2 0.4 0.6 0.8 1.0 p (reward | no stimulus) Trials --->Figure 17.1 Basic assumptions of animal learning theory defining the reinforcing functions of rewards.(A) Contiguity refers to the temporal proximity of a conditioned stimulus (or action) and the reward. (B)Contingency refers to the conditional probability of reward occurring in the presence of a conditionedstimulus as opposed to its absence (modified from Dickinson, 1980). (C) Prediction error denotes thediscrepancy between an actually received reward and its prediction. Learning (DV, associative strength) isproportional to the prediction error (g – V) and reaches its asymptote when the prediction error approacheszero after several learning trials. Used with permission.frequently in the absence of a reward, compared process. Once the discrepancy has declined to lowto its presence, the occurrence of the stimulus levels, there would be little further updating ofpredicts a lower incidence of reward, and the predictions, little behavioral changes, and thusstimulus becomes a conditioned inhibitor, even little learning. Thus, the progress of learningthough the contiguity requirement is fulfilled. depends on reducing the discrepancy between the actual and predicted outcomes. Formalizations of the role of discrepanciesPrediction Errors between outcomes and predictions have recog-Whereas sensory systems in principle use infor- nized the similarity to general error-drivenmation that is directly received from the outside learning rules, and accordingly named the dis-world through specific organs, adaptive behavior crepancy an error in reward predictionbased on predictions requires continuous com- (Rescorla & Wagner, 1972; Sutton & Barto,parisons with the actually received outcomes 1981). Outcomes that are different than pre-so that predictions can be kept valid through dicted influence behavior in a direction thatupdating. Thus, to make accurate choices reduces the prediction error between the out-between predicted outcomes, the brain would come and its prediction until outcome andset up predictions, compare the actual outcomes prediction match. This concept applies to clas-with the predictions, and update the predictions sical (Pavlovian) conditioning in whichaccording to this discrepancy. Learning, viewed as learning consists of changing the predictionbehavioral adaptation to environmental change, until it matches the outcome, and behavioralwould be the simple consequence of this updating changes are consequential to the change in

318 PRIMATE NEUROETHOLOGYprediction. In instrumental (operant) condi- unifying concept of a single striatal function hastioning, the behavioral action is performed emerged (and may never emerge). The lateralwith some prediction of outcome. Learning putamen is activated during movements, andconsists of changing the behavior until the out- Parkinsonian patients with dopamine deficienciescome matches the prediction. Thus, setting up in the putamen have severe deficits in motorand modifying predictions is essential for activity. The rostral caudate and putamen haveadapting behavior to the requirements of the prominent connections with most areas of theorganism according to the resources available association cortex (‘‘associative striatum’’).in the environment. Many parts of the striatum, including the ventral The crucial role of prediction error is derived striatum (nucleus accumbens), are involved infrom Kamin’s blocking effect (1969), which pos- reward processing. Synaptic connections in thetulates that a reward that is fully predicted does striatum show use-dependent plasticitynot contribute to learning, even when it follows (Reynolds et al., 2001), suggesting involvementthe conditions of contiguity and contingency. in various forms of learning and memory, such asThis is conceptualized in the associative learning procedural learning, sensorimotor and skillrules (Rescorla & Wagner, 1972), according to learning (Hikosaka et al., 1989; Wise, 1996),which learning advances only to the extent to habit learning (Malamut et al., 1984; Packard &which a reinforcer is unpredicted, and learning Knowlton, 2002), goal-directed instrumental andslows progressively as the reinforcer becomes reward association learning (Gaffan & Eacott,more predicted. Thus, learning can be graphed 1995), and emotional learning (Linden et al.,as an asymptotic learning curve (Fig. 17.1c) and 1990). Without attempting to resolve all mys-follows the equation: teries of striatal functions, this section will describe the neuronal activity of striatal neurons D V ¼ a bðlÀVÞ while animals learn reward predictions in con-with V as associative strength of conditioned trolled behavioral tasks.stimulus (prediction), a and b as learning con-stants, and l as maximal associative strength Behavioral Tasks Testing Striatal and Frontalpossibly sustained by an outcome (e.g., reward). Cortical Reward Functions The omission of a predicted reinforcerreduces the strength of the conditioned stimulus Experimental psychologists have developedand produces extinction of behavior. So-called specific behavioral tasks to test particular beha-attentional learning rules in addition relate the vioral processes in isolation and with acapacity to learn (associability) in certain situa- minimum of confounding variables, and neu-tions to the degree of attention evoked by the roscientists use these tasks to study the involve-conditioned stimulus or reward (Mackintosh, ment of specific brain structures in individual1975, Pearce & Hall, 1980). behavioral processes. A good behavioral paradigm for assessing the function of the frontal cortex, and of the closely associated basalSTRIATUM ganglia (see above), is the delayed response task, which tests spatial processing, workingBackground memory, movement preparation and execution,The mammalian striatum (caudate nucleus, prediction of reward, and goal-directed behaviorputamen, ventral striatum including nucleus (Divac et al., 1967; Jacobsen & Nissen, 1937). Inaccumbens) is an important brain structure order to relate neurophysiological findings to pre-involved in controlling behavioral output. It is vious lesion data using these tests, researchers useclosely associated, through multiple, convergent, different versions of spatial and conditionaland partly closed loops, with all areas of the delayed response tasks and investigate the motiva-cerebral cortex. Despite 40 years of intense tional properties of single neurons in macaqueresearch and numerous suggestions, still no monkeys. In such tasks, an initial visual stimulus

NEUROPHYSIOLOGICAL CORRELATES OF REWARD LEARNING 319indicates the spatial position of the target of an scratch and acquire the associated reward pre-arm movement for a brief period. Following a dictions. The behavioral tasks usually employdelay of several seconds, during which the spatial an initial stimulus that predicts the rewardinformation is absent, a neutral stimulus appears and informs the animal about the requiredthat tells the animal to make its response to the behavioral reaction.remembered target in order to receive a liquid or Our own studies use a learning set paradigmfood reward. In adaptations of delayed response in which animals first learn a behavioral tasktasks for investigating reward functions, and subsequently learn only a single new taskresearchers use initial stimuli that, in addition to component at a time that changes repeatedly.spatial information, predict which specific reward We use a delayed go/no-go response task withthe animal would receive for responding correctly. three trial types, namely, rewarded movementIntroduction of a delay of 2 to 3 seconds between trials, rewarded nonmovement trials, and unre-the correct response and the delivery of reward warded, sound-reinforced movement trialsprovides the animal with a specific reward expec- (Fig. 17.2A). Different initial pictures presentedtation period following the behavioral reaction. on a computer monitor predict whether theThus, the delayed response task is an instrumental animal would receive a liquid reward or the(operant) task in which the reward is contingent auditory reinforcer, and which behavioral reac-upon the correct response of the animal. The tion (movement or no movement) is required toassociation of the initial stimulus with the obtain that outcome. Animals first learn thereward is of a Pavlovian nature, such that the familiar stimuli. Then the three stimuli for thestimulus becomes a predictor for a specific three trial types are replaced by three novel pic-reward. tures, whereas all other task events and the global The presentation of a learned stimulus leads task structure remain unchanged. Animals learnto the recall of its behavioral and motivational the reward predictions and behavioral signifi-significance from long-term memory. The cances of the stimuli by trial and error. Whenstimulus is stored in working memory asymptotic performance is reached with a set of(remember what just happened) and induces three learning stimuli, those stimuli are discardedshort-term preparations of behavioral actions and three novel stimuli are introduced. Thus,and expectations of future rewards. learning consists of associating each new pictureExpectation and preparation have common (1) with liquid reward or conditioned auditoryanticipatory components, as they precede reinforcement and (2) with executing or with-rather than follow predictable events, but they holding the movement. The repeated learning ofrefer to different predictable events (stimuli, tens and hundreds of novel pictures results in aaction, reward) and occur during different learning set in which learning occurs largelyepochs of the task. The prospective (forward in within the first few trials and approaches asymp-time) expectation and preparation processes totic performance within 5 to 10 trials of eachdiffer schematically from retrospective (back- trial type (Fig. 17.2C), similar to previous beha-ward in time) working memory processes that vioral studies (Gaffan et al., 1988; Harlow, 1949).concern the previous event and do not require The procedure resembles everyday situations inpredictive coding, although they are somewhat which only a minimum of new information isentangled and could be considered common learned and most other task components remainconstituents of working memory (Honig & valid or are only slightly altered. We use theThompson, 1982). learning set paradigm to investigate the electrophysiological activity of single neurons during complete learning episodes and to compareLearning Tasks their activity with performance in trials usingNeurophysiological studies investigate the familiar, well-learned pictures. We are parti-acquisition of neural responses to novel sti- cularly interested in the responses to reward-muli while animals learn a new task from predicting stimuli and in the more sustained

320 PRIMATE NEUROETHOLOGY A Movement -> reward Stimulus pictures Familiar Learn 1 Learn 2 No movement -> reward Movement -> no reward –2 0 2 4 6s Stimulus React C 100BReward prediction 80 Correct (%)Movement preparation 60Movement initiation 40 Rewarded movementReward expectation 20 Rewarded non movementReward detection Unrewarded movement Instruction Movement Reward 0 trigger 1 3 5 7 9 11 13 15 Learning trialsFigure 17.2 Learning task and basic neuronal activity. (A) Delayed response task. An initial pictureinstructs the animal to perform a rewarded movement, unrewarded movement, or rewarded nonmovementreaction following a subsequent uniform trigger stimulus. Small drops of juice, or a sound, serve asreinforcer if the trial is correctly performed. For learning (right), three novel pictures indicate the threetrial types, and the animal finds out by trial and error which of the novel pictures corresponds to the knowntrial types. (B) Schematics of main forms of behavioral relationships of striatal neurons. (C) Learning curvesindicating rapid learning of the three trial types.activations occurring during the expectation of specific epochs of this task in response toreward. reward-predicting stimuli and in advance of the expected delivery of reward. The activations differentiate between reward and no reward,Basic Neural Activity between different kinds of liquid and foodDuring the performance of delay tasks, neurons reward, and between different magnitudes ofin the striatum and frontal cortex are activated reward. They occur in all trial types in whichdifferentially and specifically during individual reward is expected, irrespective of the type oftask components (Fig. 17.2B) involving the ret- behavioral action. Such reward expectation–rospective encoding, maintenance, and recall of related activations are found not only in theworking memory and the prospective prepara- striatum (Apicella et al., 1992; Bowman et al.,tion of movement (Alexander & Crutcher, 1990; 1996; Cromwell & Schultz, 2003; Hikosaka et al.,Funahashi et al., 1989, 1993). 1989; Hollerman et al., 1998; Schultz et al., 1992) Neuronal activations related to the predic- but also in the amygdala, orbitofrontal cortex,tion and expectation of reward arise during dorsolateral prefrontal cortex, anterior cingulate,

NEUROPHYSIOLOGICAL CORRELATES OF REWARD LEARNING 321and supplementary eye field (Amador et al., 2000; reward-predicting stimuli are abolished byPratt & Mizumori, 2001; Schoenbaum et al., interference with dopamine neurotransmission1998; Shidara & Richmond, 2002; Tremblay & (Aosaki et al., 1994a). Responses are reinstatedSchultz, 1999, 2000a). by systemic administration of the dopamine In some striatal and dorsolateral prefrontal agonist apomorphine. Associating an aversiveneurons, the differential reward expectation– stimulus, such as a loud or air puff–relatedrelated activity discriminates in addition noise, with a liquid reward changes thebetween different behavioral responses, such response of TANs to the stimulus (Ravel etas eye and limb movements toward different al., 2003). During T-maze learning, neurons inspatial targets, and movement versus non- rat striatum show progressively earlier, predic-movement reactions (Cromwell & Schultz, tive activity in the maze as learning advances2003; Hassani et al., 2001; Hollerman & (Jog et al., 1999). Behavioral reversal of sti-Schultz, 1998; Kawagoe et al., 1998; mulus-outcome associations results in reversedKobayashi et al., 2002; Matsumoto et al., neural responses in neurons discriminating2003; Watanabe, 1996; Watanabe et al., between reward- and punisher-predicting olfac-2002). The activations do not simply represent tory stimuli (Setlow et al., 2003). During theoutcome expectation, as they differentiate performance of learning set tasks, striatal neu-between different behavioral reactions despite rons develop activating responses to reward-the same outcome, and they do not simply predicting stimuli during individual learningreflect different behavioral reactions, as they episodes or show transient responses onlydifferentiate between the expected outcomes. during the learning of novel reward-predictingThus, the neurons show differential, beha- stimuli (Tremblay et al., 1998). Transientvior-related activations that depend on the responses disappear after less than 10 oroutcome of the trial, namely, reward or no within a few tens of trials (Fig. 17.3).reward and different kinds and magnitudes of Responses occur occasionally also in additionalreward. These activities provide evidence for a trial types during initial learning, suggesting aneural representation of the outcome during transient loss of selectivity during learning.the preparation and execution of the beha- Compatible with the motor and sensory func-vioral reactions performed in order to obtain tions of the striatum, neurons in this structurethe rewarding goal. They may be parts of do not only respond to motivationally signifi-neural mechanisms underlying goal-directed cant stimuli but can also acquire responses tobehavior. stimuli indicating the direction of behavioral reactions, as seen in a conditional oculomotor task (Pasupathy & Miller, 2005). Neurons inAcquisition of Responses to the posterior striatum respond stronger toReward-Predicting Stimuli novel visual stimuli as compared to familiarStriatal neurons in monkeys and rats acquire stimuli, although these neurons generally faildiscriminating responses to reward- or pun- to respond to rewards and reward-predictingisher-predicting stimuli during associative stimuli (Brown et al., 1995). Taken together,Pavlovian or operant learning. The depressant, striatal neurons readily acquire responses thatdiscriminant responses of tonically active striatal are sensitive to the outcome and movementinterneurons (TANs) to reward-predicting audi- contingencies of external stimuli.tory stimuli are rare before learning and becomesubstantial following 15 minutes of repeated Adaptation of Reward Expectationpairing with liquid (Aosaki et al., 1994b).Behavioral extinction of conditioned stimuli The presentation of a reward-predicting stimulusby withholding reward results in reduction of evokes an expectation of the forthcoming reward.neural responses within 10 minutes. Acquisition Novel stimuli of unknown reward prediction willand expression of the neuronal response to not evoke specific reward expectations, although

322 PRIMATE NEUROETHOLOGY Familiar Learning 2 seconds StimulusFigure 17.3 Transient response of caudate neuron to reward-predicting stimulus during learning. Theresponse disappears gradually after a few initial learning trials. Dots in rasters denote the time of neuronalimpulses. Each line of dots represents one trial. The sequence of trials is plotted chronologically from top tobottom, separately for familiar and learning trials, learning rasters beginning with the first presentationsof the new stimulus. Adapted from Tremblay, L., Hollerman, J. R., & Schultz, W. (1998). Modifications ofreward expectation-related neuronal activity during learning in primate striatum. Journal ofNeurophysiology, 80, 964–977, with permission by The American Physiological Society.the previous experience of the animal would indi- and longer return times from the lever back tocate that some of these stimuli should be asso- the resting key, as compared to unrewardedciated with a reward. Consequently, a default movements (Fig. 17.4A,B). This behavioralreward expectation might precede any specific marker allows us to infer the animal’s rewardexpectation arising from learning the specific, expectation and its changes during learning.reward-predicting stimuli. With new stimuli, return times are initially Using our learning set task we investigate typical for rewarded movements in bothchanges in existing reward expectations during rewarded and unrewarded movement trials.the learning of novel stimuli, using rewarded Return times differentiate after the first fewmovements, rewarded nonmovement reactions, trials between rewarded and unrewarded move-and unrewarded movements (Tremblay et al., ments (Fig. 17.4C). Erroneous movements in1998). Rewarded movements in familiar trials nonmovement trials are usually performedshow consistently shorter reaction times of with return times typical for rewarded move-movements from a resting key toward a lever ments. Similar changes occur with another

NEUROPHYSIOLOGICAL CORRELATES OF REWARD LEARNING 323 A B Familiar ms 2500 Rewarded movements 1500 Unrewarded movements 500 Wait Reward Return Target Learning C ms Shift to 2500 Rewarded movements no reward expectation 1500 Unrewarded movements 500 1 3 5 7 9 11 13 trialsFigure 17.4 Adaptation of reward expectation during learning, as evidenced by behavioral reactions. (A)Movement phase of the behavioral task. Following release of the resting key, the animal’s hand touches thetarget key (left), upon which the trigger stimulus disappears (center) and reward or an auditory reinforcer isdelivered, and the animal subsequently returns its hand to the resting key (right). (B) In familiar trials, theanimal’s return to the resting key differs depending on delivery of reward (top) or auditory reinforcer(bottom). Ordinate indicates median return times in milliseconds (from release of resting key until return toresting key). (C) During learning, the animal’s behavioral reaction after touching the target key initiallyresembles that in familiar rewarded trials (trials 1–3) and differentiates subsequently according to rewardversus auditory reinforcement. The shift after trial 3 is the result of averaging and varies in individuallearning problems. The trial contains also an initial reinforcer-predicting stimulus and a subsequentmovement trigger stimulus (not shown). Data from Tremblay, L., Hollerman, J. R., & Schultz, W. (1998).Modifications of reward expectation-related neuronal activity during learning in primate striatum. Journalof Neurophysiology, 80, 964–977, with permission by The American Physiological Society.behavioral marker, the electromyographic predicted by each picture. The animal’s preex-activity of forearm muscles. During initial isting reward expectation appears to adapt to thelearning, forearm muscle activity is similar in currently expected outcome according to theall movement trials between key release and experience of the animal with the new stimuli.return to resting key and resembles the muscle During learning, all forms of reward expecta-activity seen in familiar rewarded movements. tion–related activity of striatal neurons change inLater during each learning episode, muscle the same manner (Tremblay et al., 1998). Duringactivity approaches a pattern in unrewarded initial learning trials, in which the novel stimulimovement trials that is typical for familiar unre- are not yet firmly associated with specific rewardwarded movements. Thus, both movement expectations and animals rely on default rewardparameters and muscle activity in initial learning expectations, striatal reward expectation–relatedtrials are typical for rewarded movements and neuronal activations occur in all trial types.subsequently differentiate between rewarded Gradually, the neuronal activations becomeand unrewarded movements. We conclude restricted to rewarded as opposed to unrewardedfrom these observations that the animals have a movement trials, concurrent with, or slightly ear-default expectation of reward in initial learning lier than, the animal’s shift in behavioral indicestrials, probably acquired from previous experi- of expectation. These changes occur in neuronsence in the same task, which differentiates sub- showing pure reward expectation-related activitysequently according to the type of reinforcer immediately preceding the reward and in

324 PRIMATE NEUROETHOLOGY Familiar Rewarded movement Unrewarded movement Instruction Movement Reward Instruction Movement Sound 1s Learning Rewarded movement Unrewarded movement unrewarded movements Instruction Movement Reward Instruction Movement Sound wait for reward: 2000 ms wait for reward: 2000 ms -> no waitFigure 17.5 Adaptation of neural reward expectation during learning. During performance with familiarstimuli (top), this caudate neuron shows a sustained response to the initial reward-predicting stimulus inrewarded movement trials and a transient response in nonmovement trials (not shown). During learning(bottom), the sustained response occurs initially also in unrewarded movement trials, which are performedwith parameters of rewarded movements. The sustained response disappears when movement parametersbecome typical for unrewarded movements (arrows to the right). From Tremblay, L., Hollerman, J. R., &Schultz, W. (1998). Modifications of reward expectation-related neuronal activity during learning inprimate striatum. Journal of Neurophysiology, 80, 964–977, with permission by The AmericanPhysiological Society.neurons in which the reward expectation–related initial stimuli and the movement that are typicalactivity occurs during the preparation period for for dorsolateral prefrontal cortex (Funahashi etthe behavioral reaction (Fig. 17.5). Thus, striatal al., 1989, 1993; Fuster, 1973). Although even thereward expectation activity is nondifferential most elaborate versions of delayed response tasksduring initial learning trials with undifferentiated are unlikely to test the whole spectrum of thereward expectations and subsequently adapts to wide-ranging orbitofrontal functions, orbito-the newly learned, valid expectations. frontal neurons consistently show pronounced relationships to reward (Padoa-Schioppa & Assad, 2006; Rolls et al., 1989, 1996;ORBITOFRONTAL CORTEX Schoenbaum et al., 1998; Tremblay & Schultz,In delayed response tasks adapted for reward stu- 1999).dies, orbitofrontal neurons display several forms Several studies examine orbitofrontal neu-of task-related activity, including responses to the rons during learning. During the performancereward-predicting stimuli, activations during the of go/no-go tasks, animals emit a behavioralexpectation of reward, and responses to the response to one conditioned stimulus andreward itself (Tremblay & Schultz 2000a). refrain from moving to a different stimulus inHowever, orbitofrontal neurons show only rarely order to receive a predicted reward or avoid asustained activations during the delay between the punisher. Rolls and colleagues (1996) used

NEUROPHYSIOLOGICAL CORRELATES OF REWARD LEARNING 325differential odor and visual stimuli for licking responses during learning. If the increases andmovements rewarded with sweet liquid (go) and decreases are added, hardly any net changefor withholding licking to avoid aversive saline would result (Fig. 17.6, bottom).(no-go). The study tests the reversal of existing These data suggest that orbitofrontal neuronsconditioned stimulus–outcome associations in respond in a flexible manner to stimuli signalingmonkeys and found that many orbitofrontal rewards and aversive outcomes. In the intui-neurons reverse their differential, reinforcer-dis- tively most understandable form, orbitofrontalcriminating responses at about the same time as neurons acquire differential responses to visualthe animal reverses its go/no-go behavioral and olfactory stimuli associated with differentresponses. Schoenbaum and colleagues (1998, types of reinforcer to be expected or avoided. A1999) used odors as outcome-predicting stimuli conceptually more interesting learning changein rats for the differential acquisition of associa- consists of adaptation of reward expectation attions with rewarding sucrose and aversive qui- behavioral and neural levels. The changes ofnine solutions in go/no-go nose-poking overt behavioral reactions reveal how the ani-responses. The studies report that orbitofrontal mal’s internal reward expectations adapt to theresponses to the odors come to discriminate current values of reward prediction while thebetween the two outcomes during learning, new stimuli are being learned across successiveand most of them become differential only at trials. The behavioral changes are preceded orthe same time as the animal reaches the beha- paralleled by changes in neural activations,vioral learning criterion. Furthermore, a sepa- which may constitute a neural correlate for therate group of orbitofrontal neurons develops behavioral adaptation of expectations to newdifferential activations during the expectation task contingencies. These observations suggestof the two types of reinforcers, and these changes a neural mechanism for adaptive learning inoccur before the animal reaches the learning which existing expectation-related activity iscriterion. matched to the new condition rather than Our own study employs the same delayed go/ acquiring all task contingencies from scratch.no-go learning set task for orbitofrontal cortexas for the striatum (Tremblay & Schultz, 2000b).Only the three initial stimuli change between DOPAMINE NEURONSlearning episodes, and animals learn the new Backgroundreward predictions and behavioral significanceswithin a few trials by trial and error. Midbrain dopamine neurons show phasic exci-Orbitofrontal neurons show very similar tatory responses (activations) following primarylearning-related changes as striatal neurons. food and liquid rewards; reward-predictingNotably, the neurons acquire responses to the visual, auditory, and somatosensory stimuli;reward-predicting stimuli in each learning epi- and physically intense visual and auditory sti-sode, and their existing reward expectation– muli. These activations occur in 65% to 80% ofrelated activity adapts to the novel reward con- dopamine neurons in cell groups A9 (pars com-tingencies in parallel with behavioral adapta- pacta of substantia nigra), A10 (ventral teg-tions of reward expectation. However, mental area [VTA]), and A8 (dorsolateralcompared to the striatum, orbitofrontal neurons substantia nigra). The activations have latenciesshow more transient changes during learning of less than 100 ms and durations of less than 200that are absent during performance with familiar ms. The same neurons are briefly depressed instimuli. These changes consist of increases or their activity by reward omission and by stimulidecreases of responses to the novel stimuli predicting the absence of reward. The activa-during the learning period (Fig. 17.6), and tions are only rarely seen following aversive sti-often during the subsequent consolidation muli (Mirenowicz & Schultz, 1996) and not at allperiod of several tens of trials. By contrast, after inedible objects and known neutral stimuliother orbitofrontal neurons show reduced unless they are very intense or large. With these

Response increases Familiar Learning Response decreases Familiar Learning Changes combined Familiar Learning Stimulus 2sFigure 17.6 Changes of population responses in orbitofrontal cortex to reward-predicting stimuli duringlearning. Top: Increase of responses in 38 neurons with responses in familiar trials. Center: Decrease ofresponses in 26 neurons. Bottom: Averaged changes in neurons showing responses exclusively in learningtrials, increased responses during learning, and decreased responses during learning (82 neurons). Only datafrom rewarded movement trials are shown. In each display, histograms from the first 15 rewardedmovement trials recorded with each neuron are added, and the resulting sum is divided by the number ofneurons. From Tremblay, L., Hollerman, J. R., & Schultz, W. (1998). Modifications of reward expectation-related neuronal activity during learning in primate striatum. Journal of Neurophysiology, 80, 964–977, withpermission by The American Physiological Society. 326

NEUROPHYSIOLOGICAL CORRELATES OF REWARD LEARNING 327characteristics the phasic dopamine responses to 1994; Pan et al., 2005), reminiscent of arewards are compatible with the notion of a Pavlovian behavioral response transfer. Thereward prediction error and may serve as a responses to these different events coexist tran-teaching signal for reinforcement learning. siently at intermediate levels of learning. Thus,Dopamine neurons in groups A8 through A10 the two responses change in opposite directionsproject their axons to the dorsal and ventral at the time of the events, without involvingstriatum, dorsolateral and orbital prefrontal activity backpropagating across the stimulus-cortex, and some other cortical and subcortical reward interval. During learning, the responsesstructures. The subsecond dopamine reward of dopamine neurons become discriminativeresponse may be responsible for the reward- and gradually stabilize, consisting of activationsinduced dopamine release seen with voltam- following reward-predicting stimuli, depres-metry (Roitman et al., 2004) but would not sions or activation-depression sequences fol-easily explain the 300 to 9,000 times slower lowing conditioned inhibitors that explicitlydopamine fluctuations with rewards and pun- predict the absence of reward, and no orishers seen in microdialysis (Datla et al., 2002; minor generalizing responses to neutral stimuliYoung, 2004). Separate from the rapid reward (Fig. 17.8) (Tobler et al., 2003).response, slower, mostly depressant electrophy-siological responses occur in dopamine neurons Crucial Role of Prediction Errorsfollowing strong aversive stimuli underanesthesia. Behavioral studies suggest that prediction errors may provide an important mechanismAcquisition of Responses to for the learning and updating of reward pre-Reward-Predicting Stimuli dictions. The response acquisition of dopamine neurons might underlie this behavioral func-Differential Responses tion if it could be shown that neuronalDopamine neurons acquire responses to learning follows the same basic tests for pre-reward-predicting visual and auditory condi- diction errors as behavioral learning. Thetioned stimuli through pairing with food or blocking paradigm developed by Kaminliquid rewards (Ljungberg et al., 1992; (1969) can be easily adapted to neurophysio-Mirenowicz & Schultz, 1994). The responses logical studies. In essence, the test involves thecovary with the expected value of reward, irre- pairing of a novel stimulus with a reward thatspective of spatial position, sensory stimulus is already fully predicted (by another sti-attributes, and arm, mouth, and eye move- mulus), such that the fully predicted rewardments (Tobler et al., 2005). The responses are fails to elicit a prediction error in the presencemodulated by the motivation of the animal, the of the novel stimulus. The assumption of pre-time course of predictions and the animal’s diction error learning postulates that the novelchoice among rewards (Morris et al., 2006; stimulus fails to acquire reward predictionNakahara et al., 2004; Satoh et al., 2003). (blocking). The absence of learning can beAlthough discriminating between reward-pre- attributed to the lack of prediction errordicting stimuli and neutral stimuli, dopamine rather than unspecific factors by using a con-activations have considerable propensity for trol stimulus that is paired with an unpredictedgeneralization (Waelti et al., 2001). While the reward, such that the control stimulus becomesanimals undergo Pavlovian or operant condi- a reward predictor.tioning, dopamine neurons are initially acti- In the neurophysiological experiment, thevated by the reward, but the reward response novel stimulus indeed fails to acquire a neuronaldeclines gradually and a response to the response, whereas the control stimulus elicits areward-predicting stimulus appears and grows substantial response, very much in parallel togradually over successive trials (Fig. 17.7) behavioral learning (Fig. 17.9) (Waelti et al.,(Ljungberg et al., 1992; Mirenowicz & Schultz, 2001). The blocked neuronal learning of the

328 PRIMATE NEUROETHOLOGY <--- Chronology Conditioned Reward 0.5 s stimulusFigure 17.7 Transfer of neural response from reward to conditioned stimulus in a single dopamineneuron during a full learning episode. Each line of dots represents a trial, each dot represents the time of thedischarge of the dopamine neuron, the vertical lines indicate the time of the stimulus and juice reward, andthe picture above the raster shows the visual conditioned stimulus presented to the monkey on a computerscreen. Chronology of trials is from top to bottom. The top trial shows the activity of the neuron while theanimal saw the stimulus for the first time in its life, whereas it had previous experience with the liquidreward. Data from Waelti, P., Dickinson, A., & Schultz, W. (2001). Dopamine responses comply with basicassumptions of formal learning theory. Nature, 412, 43–48.novel stimulus occurs despite its pairing with the notion that dopamine neurons emit a positivereward, demonstrating the crucial role of pre- signal (activation) when an appetitive event isdiction error in learning. It thus appears that the better than predicted, no signal (no change inlearning of dopamine neurons is governed by the activity) when an appetitive event occurs as pre-same basic principle that underlies behavioral dicted, and a negative signal (decreased activity)conditioning, suggesting a neural correlate for when an appetitive event is worse than predictedthe role of prediction errors in the acquisition of (Schultz et al., 1997). In line with this argument,reward prediction. dopamine neurons report both positive and negative prediction errors during discrimination learning when animals find out by trial and errorPrediction Error Coding which one of two novel stimuli is rewarded andIn addition to neuronal learning being sensitive which one is not (Hollerman & Schultz, 1998).to prediction errors, dopamine neurons encode More stringent tests for the neural coding ofan explicit prediction error signal. Dopamine prediction errors include formal paradigms ofneurons show reward activations only when the animal learning theory in which predictionreward occurs unpredictably and fail to respond errors occur in specific situations. In the blockingto well-predicted rewards; their activity is paradigm, the blocked stimulus fails to predict adepressed when the predicted reward fails to reward. Accordingly, the absence of reward fol-occur (Ljungberg et al., 1992; Mirenowicz & lowing that stimulus does not produce a predic-Schultz, 1994). This result has prompted the tion error nor a response in dopamine neurons at

NEUROPHYSIOLOGICAL CORRELATES OF REWARD LEARNING 329 Conditioned stimulus predicting reward Conditioned stimulus predicting absence of reward Known neutral stimulus 0.5 sFigure 17.8 Differential responses of a single dopamine neuron to three types of stimuli. Top: Activatingresponse to a reward-predicting stimulus. Middle: Depressant response to a different stimulus predictingthe absence of reward (Pavlovian conditioned inhibition task). Bottom: Neutral stimulus. Vertical line andarrow indicate time of stimulus. Data from Tobler, P. N., Dickinson, A., & Schultz, W. (2003). Coding ofpredicted reward omission by dopamine neurons in a conditioned inhibition paradigm. Journal ofNeuroscience, 23, 10402–10410the usual time of the omitted reward, and the reward, and the absence of reward after this sti-delivery of a reward does produce a positive mulus does not produce a prediction error nor aprediction error and a dopamine response response in dopamine neurons, even when(Fig. 17.10A, left) (Waelti et al., 2001). By con- another, otherwise reward-predicting stimulus istrast, after a well-trained, reward-predicting sti- added (Fig. 17.10B) (Tobler at al., 2003). By con-mulus, reward omission produces a negative trast, the occurrence of reward after a conditionedprediction error and a depressant neural inhibitor produces a strong positive predictionresponse, whereas the fully predicted delivery of error due to the large difference between thea reward does not lead to a prediction error nor a received reward and the negative predictionresponse in dopamine neurons (Fig. 17.10A, from the inhibitor. Dopamine neurons show aright). In a conditioned inhibition paradigm, stronger response in conditioned inhibitor trialsthe conditioned inhibitor predicts the absence of (Fig. 17.10B, bottom left) compared to trials with

330 PRIMATE NEUROETHOLOGY Established reward prediction Reward No reward Novel stimulus added Reward Reward 0.5 s Neuronal learning test Stimulus StimulusFigure 17.9 Acquisition of dopamine response to reward-predicting stimulus is governed by predictionerror. Neural learning is blocked when the reward is predicted by another stimulus (left) but is intact inthe same neuron when reward is unpredicted in control trials with different stimuli (right). The neuronhas the capacity to respond to reward-predicting stimuli (top left) and discriminates against unrewardedstimuli (top right). Addition of a second stimulus results in maintenance and acquisition of response,respectively (middle). Testing the added stimulus reveals absence of learning when the reward is alreadypredicted by a previously conditioned stimulus (bottom left). Data from Waelti, P., Dickinson, A., &Schultz, W. (2001). Dopamine responses comply with basic assumptions of formal learning theory.Nature, 412, 43–48 .neutral control stimuli (Fig. 17.10B, bottom right). This equation may constitute a neural equivalentTaken together, these data suggest that dopamine for the prediction error term of (l – V) of theneurons show bidirectional coding of reward pre- Rescorla-Wagner learning rule. With these char-diction errors, following the equation: acteristics, the bidirectional dopamine error response would constitute an ideal teaching Dopamine response = Reward occurred – signal for inducing neural learning through Reward predicted synaptic plasticity.

NEUROPHYSIOLOGICAL CORRELATES OF REWARD LEARNING 331 A No reward No reward Reward Reward B No reward No reward R No reward R No reward Reward Reward 0.5 sFigure 17.10 Coding of prediction errors by dopamine neurons in paradigms derived from animallearning theory. (A) Blocking test. Lack of response to absence of reward following the blocked stimulus,but positive signal to delivery of reward (left), in contrast to control trials with a learned stimulus (right).Data from Waelti, P., Dickinson, A., & Schultz, W. (2001). Dopamine responses comply with basicassumptions of formal learning theory. Nature, 412, 43–48. (B) Conditioned inhibition test. Lack ofresponse to absence of reward following the stimulus predicting no reward (top), even if the stimulus ispaired with an otherwise reward-predicting stimulus (R, middle, summation test), but strong activation toreward following a stimulus predicting no reward (bottom). These responses contrast with those following aneutral control stimulus (right). Data from Tobler, P. N., Dickinson, A., & Schultz, W. (2003). Coding ofpredicted reward omission by dopamine neurons in a conditioned inhibition paradigm. Journal ofNeuroscience, 23, 10402–10410.CONCLUSIONS comparison, a different, control stimulus or a control action is not associated with the sameLearning Situations outcome. Behavioral measures of learning consistStandard behavioral learning experiments of assessing the number of behavioral responsesinvolve the acquisition of Pavlovian or operant over the course of learning and of comparing theresponding through association of an intrinsi- responses to the target stimulus with those to thecally neutral stimulus or a spontaneously emitted control stimulus. However, neurophysiologicalaction with an outcome, such as a reward. For experiments have substantial time constraints,

332 PRIMATE NEUROETHOLOGYas the recording from individual neurons is Change of Expectationsusually limited to a few tens of minutes. In addi- The learning set experiments demonstrate the pre-tion, these experiments require studying more sence of neuronal reward expectation activitythan one neuron in each animal for reasons of from the first learning trial on. Explicit informa-reproducibility and for better use of the animal. tion eliciting these activations obviously cannot beThe time and the repeated measures require- derived from the novel, yet unknown stimuli.ments often compromise the neurophysiological During learning, a novel stimulus can only pro-study of straightforward learning situations in vide a point in time for recalling default rewardwhich a whole task is being acquired. The expectations from long-term memory aboutlearning set paradigm offers tremendous advan- the task being rewarded. Thus, the reward expec-tages in respect to these constraints. In these tation activity in each initial learning trial may beexperiments, usually only one or very few com- evoked in time by the stimulus and in content byponents of a task have to be learned, which is also recall from long-term memory of the task struc-beneficial for the behavioral analysis of well-con- ture. Once the novel stimuli have been learned, thetrolled learning. The repeated change of the novel reward expectations would be adapted andtask component increases the speed of learning to restricted to only those trials that currently leadthe point where a whole learning episode takes to reward. Only then would the newly learnedonly a few trials. This is an ideal situation for stimuli evoke the specific information necessaryneurophysiology, as both the time constraints for expecting the reward. This learningand the requirements for repeated, reproducible mechanism consists of two steps, an initial setuplearning episodes in the same animals are met. In of default expectations about reward acquiredaddition, the activity of the same neurons can be from task experience, and the subsequentcompared between novel and familiar task com- updating to the currently valid, more specificponents. Hopefully future neurophysiological and restricted expectations. The updating wouldlearning studies will profit more from this enor- be based on the deviation of the initial, defaultmous potential. expectation from the actually occurring reward, namely, a prediction error. The observed activityResponse Acquisition changes during learning appear to simply reflectThe most obvious neurophysiological phenom- this updating of expectation. This mechanism isenon in learning is the gradual acquisition of obviously more efficient than a continuous settingneuronal responses to the novel stimuli or up of complete representations of reward expecta-actions. As Pavlovian conditioning offers the tions from scratch and thus has higher chances formost simple and easily interpretable learning evolutionary survival.situation, all reviewed studies reported increasedresponses to reward-predicting stimuli com- Role of Prediction Errorspared to non–reward-predicting stimuli. Suchresponses are found in the striatum, orbito- One of the key postulated mechanisms of out-frontal cortex, and dopamine neurons, as well come-directed learning is based on predictionas in a number of other brain structures not errors. This hypothesis explains the effects of con-described here, such as the dorsolateral pre- tingency on learning (Kamin, 1969) and the gra-frontal cortex and various other parts of the dual response acquisition and extinction whenbasal ganglia besides the striatum. The response outcomes are better or worse than expected,acquisition appears to be similar between novel respectively. Prediction errors form the basis forlearning situations and learning set tasks. The a number of associative learning theories thatexample of dopamine neurons suggests that explain learning in various behavioral situationsneuronal response acquisitions follow similar (Mackintosh, 1975; Pearce & Hall, 1980; Rescorlarules as behavioral learning, namely, prediction & Wagner, 1972). In the crucial blocking test forerrors. the role of prediction errors in contingency and

NEUROPHYSIOLOGICAL CORRELATES OF REWARD LEARNING 333learning, dopamine neurons closely follow the plasticity mechanisms may link some of thesefundamental conditions of prediction error brain regions during learning.coding (Waelti et al., 2001). It is reasonable toassume that prediction errors drive the adapta-tions of reward expectations described in learning ACKNOWLEDGMENTSset situations, and it could be speculated, but is in Our work was supported by the Wellcome Trust,no way shown, that this influence is derived from NSF (US), NIH (US), Swiss NSF, Humaninputs from dopamine neurons carrying predic- Frontiers Science Program, and several othertion error information. grant and fellowship agencies.Possible Synaptic MechanismsIt is possible, but by no means shown, that inputs REFERENCESfrom some of the described brain regions induce Alexander, G. E., & Crutcher, M. D. (1990).responses to reward-predicting stimuli in the Preparation for movement: Neuralother regions, such as the dopamine responses representations of intended direction in threedriving striatal and orbitofrontal responses. The motor areas of the monkey. Journal oftransient responses of dopamine neurons may Neurophysiology, 64, 133–150.induce synaptic changes at striatal synapses, Amador, N., Schlag-Rey, M., & Schlag, J. (2000).which could underlie the learning of stimuli with Reward-predicting and reward-detectingmotivational significance and the adaptation of neuronal activity in the primate supplementary eye field. Journal ofexpectations by changed contingencies. Increased Neurophysiology, 84, 2166–2170.cortical inputs to striatal neurons may lead to Aosaki, T., Graybiel, A. M., & Kimura, M. (1994a).long-lasting synaptic changes, following a Effect of the nigrostriatal dopamine system onHebbian learning mechanism with conjoint pre- acquired neural responses in the striatum ofand postsynaptic activity. Corticostriatal synapses behaving monkeys. Science, 265, 412–415.show long-term potentiation and depression Aosaki, T., Tsubokawa, H., Ishida, A., Watanabe,(Calabresi et al., 1992; Reynolds et al., 2001). The K., Graybiel, A. M., & Kimura, M. (1994b).standard Hebbian learning mechanism may be Responses of tonically active neurons in theextended to a three-factor Hebbian process with primate’s striatum undergo systematica dedicated reward-predicting dopamine signal as changes during behavioral sensorimotorthe determining factor (Schultz & Dickinson, conditioning. Journal of Neuroscience, 14, 3969–3984.2000). Dopamine is known to induce synaptic Apicella, P., Scarnati, E., Ljungberg, T., & Schultz,plasticity in the striatum and cortex (Bao et al., W. (1992). Neuronal activity in monkey2001; Blond et al., 2002; Gurden et al., 2000), and striatum related to the expectation ofinterference with dopamine neurotransmission predictable environmental events. Journal ofimpairs the induction of long-term synaptic Neurophysiology, 68, 945–960.changes (Calabresi et al., 1992; Gurden et al., Bao, S., Chan, V. T., & Merzenich, M. M. (2001).2000; Kerr & Wickens 2001). The use of the dopa- Cortical remodelling induced by activity ofmine prediction error in models of reinforcement ventral tegmental dopamine neurons. Nature,learning demonstrates that the dopamine signal 412, 79–83.not only replicates some of the basic assumptions Blond, O., Crepel, F., & Otani, S. (2002). Long-termof temporal difference learning (Montague et al., potentiation in rat prefrontal slices facilitated by phased application of dopamine. European1996) but also is able to serve as an effective Journal of Pharmacology, 438, 115–116.teaching signal for such typical striatal and frontal Bowman, E. M., Aigner, T. G., & Richmond, J.cortical tasks as delayed responding (Suri & (1996). Neural signals in the monkey ventralSchultz, 1999). Thus, although reward informa- striatum related to motivation for juice andtion may reach the different components of the cocaine rewards. Journal of Neurophysiology,reward system through separate routes, the listed 75, 1061–1073.

334 PRIMATE NEUROETHOLOGYBrown, V. J., Desimone, R., & Mishkin, M. (1995). potentiation at hippocampal-prefrontal cortex Responses of cells in the caudate nucleus during synapses in vivo. Journal of Neuroscience, 20 RC visual discrimination learning. Journal of 106, 1–5. Neurophysiology, 74, 1083–1094. Harlow, H. F. (1949). The formation of learningCalabresi, P., Maj, R., Pisani, A., Mercuri, N. B., & sets. Psychology Review, 56, 51–65. Bernardi G. (1992). Long-term synaptic Hassani, O. K., Cromwell, H. C., & Schultz, W. depression in the striatum: Physiological and (2001). Influence of expectation of different pharmacological characterization. Journal of rewards on behavior-related neuronal activity Neuroscience, 12, 4224–4233. in the striatum. Journal of Neurophysiology, 85,Cromwell, H. C., & Schultz, W. (2003). Effects 2477–2489. of expectations for different reward Hikosaka, O. (1999). Parallel neural networks for magnitudes on neuronal activity in primate learning sequential procedures. Trends in striatum. Journal of Neurophysiology, 89, Neuroscience, 22, 464–471. 2823–2838. Hikosaka, O., Sakamoto, M., & Usui, S. (1989).Datla, K. P., Ahier, R. G., Young, A. M. J., Functional properties of monkey caudate Gray, J. A., & Joseph, M. H. (2002). Condi- neurons. III. Activities related to expectation tioned appetitive stimulus increases extra- of target and reward. Journal of cellular dopamine in the nucleus accumbens of Neurophysiology, 61, 814–832. the rat. European Journal of Neuroscience, 16, Hollerman, J. R., & Schultz, W. (1989). Dopamine 1987–1993. neurons report an error in the temporalDickinson, A. (1980). Contemporary animal prediction of reward during learning. Nature learning theory. Cambridge: Cambridge Neuroscience, 1, 304–309. University Press, 1980. Hollerman, J. R., Tremblay, L., & Schultz, W.Divac, I., Rosvold, H. E., & Szwarcbart, M. K. (1989). Influence of reward expectation on (1967). Behavioral effects of selective ablation behavior-related neuronal activity in primate of the caudate nucleus. Journal of Comparative striatum. Journal of Neurophysiology, 80, Physiology and Psychology, 63, 184–190. 947–963.Funahashi, S., Bruce, C. J., & Goldman-Rakic, P. S. Honig, W., & Thompson, R. K. R. (1982). (1989). Mnemonic coding of visual space in the Retrospective and prospective processing in monkey’s dorsolateral prefrontal cortex. Journal animal working memory. In: G. H. Bower of Neurophysiology, 61, 331–349. (Ed.), The psychology of learning andFunahashi, S., Chafee, M. V., & Goldman-Rakic, P. S. motivation: Advances in research and theory (1993). Prefrontal neuronal activity in rhesus (pp. 239–283). New York: Academic Press. monkeys performing a delayed anti-saccade task. Kamin, L. J. (1969). Selective association and Nature, 365, 753–756. conditioning. In: N. J. Mackintosh & W. K.Fuster, J. M. (1973). Unit activity of prefrontal Honig (Eds.), Fundamental issues in cortex during delayed-response performance: instrumental learning (pp. 42–64). Dalhousie: Neuronal correlates of transient memory. Dalhousie University Press. Journal of Neurophysiology, 36, 61–78. Kawagoe, R., Takikawa, Y., & Hikosaka, O. (1998).Gaffan, D., & Eacott, M. J. (1995). Visual learning Expectation of reward modulates cognitive for an auditory secondary reinforcer by signals in the basal ganglia. Nature macaques is intact after uncinate fascicle Neuroscience, 1, 411–416. section: Indirect evidence for the involvement Kerr, J. N., & Wickens, J. R. (2001). Dopamine of the corpus striatum. European Journal of D-1/D-5 Receptor activation is required for Neuroscience, 7, 1866–1871. long-term potentiation in the rat neostriatumGaffan, E. A., Gaffan, D., & Harrison, S. (1988). in vitro. Journal of Neurophysiology, 85, Disconnection of the amygdala from visual 117–124. association cortex impairs visual reward Kobayashi, S., Lauwereyns, J., Koizumi, M., association learning in monkeys. Journal of Sakagami, M., & Hikosaka, O. (2002). Neuroscience, 8, 3144–3150. Influence of reward expectation onGurden, H, Takita, M., & Jay, T. M. (2000). visuospatial processing in macaque lateral Essential role of D1 but not D2 receptors in prefrontal cortex. Journal of Neurophysiology, the NMDA receptor-dependent long-term 87, 1488–1498.

NEUROPHYSIOLOGICAL CORRELATES OF REWARD LEARNING 335Jacobsen, C. F., & Nissen, H. W. (1937). Studies of Padoa-Schioppa, C., & Assad, J. A. (2006). Neurons cerebral function in primates: IV. The effects of in the orbitofrontal cortex encode economic frontal lobe lesions on the delayed alternation value. Nature, 441, 223–226. habit in monkeys. Journal of Comparative Pan, W-X., Schmidt, R., Wickens, J. R., & Hyland, Physiology and Psychology, 23, 101–112. B. I. (2005). Dopamine cells respond toJog, M. S., Kubota, Y., Connolly, C. I., Hillegaart, predicted events during classical conditioning: V., & Graybiel, A. M. (1999). Building neural Evidence for eligibility traces in the reward- representations of habits. Science, 286, 1745–1749. learning network. Journal of Neuroscience, 25,Linden, A., Bracke-Tolkmitt, R., Lutzenberger, W., 6235–6242. Canavan, A.G.M., Scholz, E., Diener, H.C., & Pasupathy, A., & Miller, E. K. (2005). Different Birbaumer, N. (1990). Slow cortical potentials time courses of learning-related activity in the in Parkinsonian patients during the course of an prefrontal cortex and striatum. Nature, 433, associative learning task. Journal of 873–876. Psychophysiology, 4, 145–162. Pearce, J. M., Hall, G. (1980). A model for PavlovianLjungberg, T., Apicella, P., & Schultz, W. (1992). conditioning: Variations in the effectiveness of Responses of monkey dopamine neurons conditioned but not of unconditioned stimuli. during learning of behavioral reactions. Psychology Review, 87, 532–552. Journal of Neurophysiology, 67, 145–163. Pratt, W. E., & Mizumori, S. J. Y. (2001). NeuronsMackintosh, N. J. (1975). A theory of attention: in rat medial prefrontal cortex show Variations in the associability of stimulus with anticipatory rate changes to predictable reinforcement. Psychology Review, 82, 276–298. differential rewards in a spatial memory task.Malamut, B. L., Saunders, R. C., & Mishkin M. Behavioral Brain Research, 123, 165–183. (1984). Monkeys with combined amygdalo- Ravel, S., Legallet, E., & Apicella, P. (2003). hippocampal lesions succeed in object Responses of tonically active neurons in the discrimination learning despite 24-hour monkey striatum discriminate between intervals. Behavioral Neuroscience, 98, 759–769. motivationally opposing stimuli. Journal ofMatsumoto, K., Suzuki, W., & Tanaka, K. (2003). Neuroscience, 23, 8489–8497. Neuronal correlates of goal-based motor Rescorla, R. A., & Wagner, A. R. (1972). A theory of selection in the prefrontal cortex. Science, 301, Pavlovian conditioning: Variations in the 229–232. effectiveness of reinforcement andMirenowicz, J., & Schultz, W. (1994). Importance nonreinforcement. In: A. H. Black & W. F. of unpredictability for reward responses in Prokasy (Eds.), Classical conditioning II: primate dopamine neurons. Journal of Current research and theory (pp. 64–99). Neurophysiology, 72, 1024–1027. New York: Appleton Century Crofts.Mirenowicz, J., & Schultz, W. (1996). Preferential Reynolds, J. N. J., Hyland, B. I., & Wickens, J. R. activation of midbrain dopamine neurons by (2001). A cellular mechanism of reward-related appetitive rather than aversive stimuli. Nature, learning. Nature, 413, 67–70. 379, 449–451. Roitman, M. F., Stuber, G. D., Phillips, P. E. M.,Montague, P. R., Dayan, P., & Sejnowski, T. J. Wightman, R. M., & Carelli, R. M. (2004). (1996). A framework for mesencephalic Dopamine operates as a subsecond modulator dopamine systems based on predictive Hebbian of food seeking. Journal of Neuroscience, 24, learning. Journal of Neuroscience, 16, 1936–1947. 1265–1271.Morris, G., Nevet, A., Arkadir, D., Vaadia, E., & Rolls, E. T., Critchley, H. D., Mason, R., & Bergman, H. (2006). Midbrain dopamine Wakeman, E. A. (1996). Orbitofrontal cortex neurons encode decisions for future action. neurons: Role in olfactory and visual Nature Neuroscience, 9, 1057–1063. association learning. Journal ofNakahara, H., Itoh, H., Kawagoe, R., Takikawa, Neurophysiology, 75, 1970–1981. Y., & Hikosaka, O. (2004). Dopamine neurons Rolls, E. T., Sienkiewicz, Z. J., & Yaxley, S. can represent context-dependent prediction (1989). Hunger modulates the responses to error. Neuron, 41, 269–280. gustatory stimuli of single neurons inPackard, M. G., & Knowlton, B. J. (2002). Learning the caudolateral orbitofrontal cortex of and memory functions of the basal ganglia. the macaque monkey. European Journal of Annual Review of Neuroscience, 25, 563–593. Neuroscience, 1, 53–60.

336 PRIMATE NEUROETHOLOGYSatoh, T., Nakai, S., Sato, T., & Kimura, M. (2003). paradigm. Journal of Neuroscience, 23, 10402– Correlated coding of motivation and outcome 10410. of decision by dopamine neurons. Journal of Tobler, P. N., Fiorillo, C. D., & Schultz, W. (2005). Neuroscience, 23, 9913–9923. Adaptive coding of reward value by dopamineSchoenbaum, G., Chiba, A. A., & Gallagher, M. neurons. Science, 307, 1642–1645. (1998). Orbitofrontal cortex and basolateral Tremblay, L., Hollerman, J. R., & Schultz, W. amygdala encode expected outcomes during (1998). Modifications of reward expectation- learning. Nature Neuroscience, 1, 155–159. related neuronal activity during learning inSchoenbaum, G., Chiba, A. A., & Gallagher, M. primate striatum. Journal of Neurophysiology, (1999). Neural encoding in orbitofrontal 80, 964–977. cortex and basolateral amygdala during Tremblay, L., & Schultz, W. (1999). Relative reward olfactory discrimination learning. Journal of preference in primate orbitofrontal cortex. Neuroscience, 19, 1876–1884. Nature, 398, 704–708.Schultz, W., Apicella, P., Scarnati, E., & Ljungberg, Tremblay, L., & Schultz, W. (2000a). Reward- T. (1992). Neuronal activity in monkey ventral related neuronal activity during go-nogo task striatum related to the expectation of reward. performance in primate orbitofrontal cortex. Journal of Neuroscience, 12, 4595–4610. Journal of Neurophysiology, 83, 1864–1876.Schultz, W., Dayan, P., & Montague, R. R. (1997). Tremblay, L., & Schultz, W. (2000b). A neural substrate of prediction and reward. Modifications of reward expectation-related Science, 275, 1593–1599. neuronal activity during learning in primateSchultz, W., & Dickinson, A. (2000). Neuronal orbitofrontal cortex. Journal of coding of prediction errors. Annual Review of Neurophysiology, 83, 1877–1885. Neuroscience, 23, 473–500. Waelti, P., Dickinson, A., & Schultz, W. (2001).Setlow, B., Schoenbaum, G., & Gallagher, M. Dopamine responses comply with basic (2003). Neural encoding in ventral striatum assumptions of formal learning theory. Nature, during olfactory discrimination learning. 412, 43–48. Neuron, 38, 625–636. Watanabe, M. (1996). Reward expectancy in primateShidara, M., & Richmond, B. J. (2002). Anterior prefrontal neurons. Nature, 382, 629–632. cingulate: Single neuron signals related to Watanabe, M., Hikosaka, K., Sakagami, M., & degree of reward expectancy. Science, 296, Shirakawa, S. I. (2002). Coding and 1709–1711. monitoring of behavioral context in theSuri, R., & Schultz, W. (1999). A neural network primate prefrontal cortex. Journal of with dopamine-like reinforcement signal that Neuroscience, 22, 2391–2400. learns a spatial delayed response task. Wise, S. P. (1996). The role of the basal ganglia in Neuroscience, 91, 871–890. procedural memory. Seminars in Neuroscience,Sutton, R. S., & Barto, A. G. (1981). Toward a 8, 39–46. modern theory of adaptive networks: Young, A. M. J. (2004). Increased extracellular Expectation and prediction. Psychology Review, dopamine in nucleus accumbens in response 88, 135–170. to unconditioned and conditioned aversiveTobler, P. N., Dickinson, A., & Schultz, W. (2003). stimuli: Studies using 1 min microdialysis in Coding of predicted reward omission by rats. Journal of Neuroscience Methods, 138, dopamine neurons in a conditioned inhibition 57–63.

CHAPTER 18Associative Memory in the Medial Temporal LobeYuji Naya and Wendy A. SuzukiThe ability to form new long-term memories for memory impairment was global, it became clearfacts, events, and relationships is referred to as later that the impairment was selective for declara-declarative memory in humans (Squire et al., tive memory for facts and events while other forms2004) and relational memory in animals of memory including visuo-motor skill learning(Eichenbaum & Cohen, 2001; Fig. 18.1A). (Corkin, 1968; Milner, 1962) and sensory primingDeclarative/relational memory is a fundamental (Gabrieli et al., 1990) remained intact. These motorcognitive function that allows us to make appro- and perceptual forms of plasticity are typicallypriate choices based on past experience. In both expressed through performance rather than recol-humans and animals, it is critical for many lection, and have been referred to collectively asaspects of everyday life, including social interac- nondeclarative or procedural memory (Fig.tions, foraging behaviors, and escape from 18.1A). The relationship between declarativedanger. In humans, declarative memory defines memory and the MTL has been further exploredour personal histories and in that way shapes our with the development of animal models of humanvery personalities. amnesia in both nonhuman primates (Mishkin, While early studies assumed memory was a 1978; Zola-Morgan & Squire, 1985) and rodentsunitary process, likely subserved by the interaction (Eichenbaum & Cohen, 2001).of large areas of the cerebral cortex (Lashley, 1929), While it is now clear that the MTL is criticalthe description of the well-known amnesic patient for normal declarative/relational memory, theH.M. (Scoville & Milner, 1957) showed for the first question of the relative contribution of indivi-time that memory could be localized to particular dual MTL structures to particular forms ofbrain areas. Patient H.M. underwent experimental declarative/relational memory remains contro-bilateral medial temporal lobe (MTL) (Fig. 18.1B) versial. While some reports argue that all MTLresection in an attempt to relieve very severe epi- structures can contribute in a complementarylepsy. The bilateral resection was described as way to many forms of declarative memoryincluding both the amygdala and the hippocampus (Squire et al., 2007), other reports have empha-as well as some of the surrounding cortical areas sized the differential contributions of individual(Scoville & Milner, 1957). Although the operation MTL structures to distinct aspects of memorymarkedly reduced the number and severity of his (Eichenbaum et al., 2007). In the latter frame-seizures, H.M. suffered from extensive anterograde work, the hippocampus is thought to be criticaland temporally graded retrograde amnesia. He for recollection-based recognition, whereasretained nothing of day-to-day happenings after perirhinal cortex is necessary for familiarity-the surgery, and could not recognize any of the based recognition (Eichenbaum et al., 2007).hospital staff except for Dr. Scoville, his neurosur- On the other hand, declarative memory is alsogeon whom he had known for many years (Milner, separable according to the types of items to be2005). While early reports suggested that H.M.’s memorized such as object, place, and time or 337

338 PRIMATE NEUROETHOLOGY A B Long-term memory Hippocampus Entorhinal Other direct cortex projections Declarative (explicit) Nondeclarative (implicit) Perirhinal Parahippocampal cortex cortex Facts Events Skills Priming Simple Nonassociative (semantic) (episodic) and classical learning Unimodal and polymodal habits conditioning Association areas (frontal, temporal, and parietal lobes)Figure 18.1 (A) Classification of long-term memory. Declarative (explicit) memory refers to consciousrecollections of facts (semantic) and events (episodic) and depends on the integrity of the medial temporallobe. Declarative memory is also referred to as relational memory in animals. Nondeclarative (implicit)memory refers to a collection of abilities and is independent of the medial temporal lobe. Nonassociativelearning includes habituation and sensitization. In the case of nondeclarative memory, experience altersbehavior unconsciously without providing access to any memory content. (B) A schematic view of themedial temporal lobe memory system and anatomically related areas. The medial temporal lobe regionsimportant for declarative/relational memory include the hippocampus and entorhinal, perirhinal, andparahippocampal cortices (thick outlines). The entorhinal cortex is the major source of corticalprojections to the hippocampus. Two thirds of the cortical input to the entorhinal cortex originates inthe adjacent perirhinal and parahippocampal cortices, which in turn receive projections from unimodal andpolymodal areas in the frontal, temporal, and parietal lobes (thin dashed outlines). The entorhinal cortexalso receives other direct inputs from orbital frontal cortex, cingulate cortex, insular cortex, and superiortemporal gyrus (thin dashed outlines). All these projections are reciprocal. Based on Squire and Zola-Morgan (1991). The medial temporal lobe memory system. Science, 253, 1380–1386. Used with permission.their combination. To elucidate the representa- in the retrieval of learned visual-visual associa-tion of memory between separate objects in tions, and how does this area interact with othera single modality, a visual stimulus-stimulus sensory areas that provide the ‘‘raw’’ sensoryassociation paradigm was introduced into the input to the perirhinal cortex? To address theseexperimental animal lesion literature (Buckley & questions, behavioral neurophysiological studiesGaffan, 1998; Bunsey & Eichenbaum, 1993; have focused on recording in both the perirhinalMurray et al., 1993). Results from these studies cortex and area TE during the performance ofindicate that the perirhinal cortex is critical for well-learned visual-visual paired associate (VPA)the formation and long-term representation of tasks. In Part I of this chapter, we detail the resultsstimulus-stimulus association memory. While from these studies.findings from lesion studies provide evidence The critical contribution of the hippocampusthat the perirhinal cortex is important for sti- to declarative/relational memory in animals hasmulus-stimulus association memory (Buckley & been demonstrated using conditional motorGaffan, 1998; Bunsey & Eichenbaum, 1993; association (CMA) learning, in which stimuliMurray et al., 1993), another critical question and spatially directed actions are associated inconcerns understanding the neural representa- memory (Murray & Wise, 1996). Monkeys withtion of this critical form of memory. How do hippocampal ablations could acquire CMA, butneurons in the perirhinal cortex represent a their learning speed was significantly slower thanlearned visual-visual association? What is its role normal control animals (Wise & Murray, 1999).

ASSOCIATIVE MEMORY IN THE MEDIAL TEMPORAL LOBE 339This finding suggested that the hippocampus 2003). Specifically, these studies show that themay be particularly important for the initial perirhinal cortex and area TE have a dynamicdevelopment of new conditional motor associa- and interrelated relationship in the long-termtions when animals need to rapidly bind object representation of associative memory; the peri-information with an action or spatial location. rhinal cortex receives perceptual signals (cueIn Part II of this chapter, we will review the stimuli) from area TE and feeds back mne-neurophysiological studies in the medial tem- monic signals (paired associates) (Naya et al.,poral lobe that have examined the development 2001, 2003b).of new associative representations. These studiesreveal dynamic changes in both the perirhinal Neuronal Correlates of Associative Long-cortex and hippocampus during the acquisition Term Memory in the Temporal Cortexof new associative memories. In one of the first descriptions of long-term associative memory signals in monkey temporalPART I: NEURONAL lobe, Miyashita (1988) recorded the activity ofORGANIZATION OF LONG-TERM neurons in IT cortex (Fig. 18.2B) as macaqueASSOCIATIVE MEMORY monkeys performed a delayed matching-to-The semantic memory system has been sample (DMS) task in which a sample imagedescribed as ‘‘a mental thesaurus’’ (Tulving, was first shown in a video monitor followed by1972) formed by overlapping collections of cell a matching stimulus (Fig. 18.2C). During theassemblies of related objects. From this point of training phase of the task, a set of 97 visualview, the inferior temporal (IT) cortex, consid- fractal patterns (‘‘learned stimuli’’; Fig. 18.2A)ered to be a long-term memory (LTM) store- was repeatedly used and these stimuli were pre-house of visual objects (Mishkin, 1982; sented to the animal as the sample stimulus in aMiyashita & Chang, 1988; Penfield & Perot, fixed sequence according to an arbitrary serial1963), is a particularly appropriate place to position number. During the recording phase ofencode the relationships between semantically the experiment, a sample stimulus was selectedlinked items. IT cortex includes both the ventral not only from the 97 learned patterns but alsoparts of perirhinal cortex (area 36, A36) and the from a new set of 97 patterns (‘‘new stimuli’’),laterally adjacent visual area TE. The two sub- independent of the serial position numbers.areas are cytoarchitectonically distinct but Surprisingly, Miyashita (1988) reported thatmutually interconnected (Saleem & Tanaka, the effective response to a given learned stimulus1996; Suzuki & Amaral, 1994; von Bonin & was most highly correlated to those stimuli thatBailey, 1947). The neural correlates of semanti- had been presented in close temporal ordercally or temporally linked items have been stu- during the training phase (stippled columns indied in monkey IT cortex with the use of various Fig. 18.2D,E). Thus, this study showed that thelong-term associative memory tasks. In the responses of single IT neurons can reflect thefollowing section, we describe studies that long-term associations between temporallyassess how LTM signals are organized and acti- related sets of stimuli even if that informationvated in macaque IT cortex following associa- is not required for the task.tion learning at a time when the long-term To examine the effect of long-term associa-associations are well established. The results tion on IT neurons when animals were explicitlysuggest that the relationships between the required to use those associations to solve thesemantically or temporally associated items are task, Sakai and Miyashita (1991) recorded therepresented by the responses of single neurons activity of IT neurons as animals performed a(Miyashita, 1988; Sakai & Miyashita, 1991) and VPA task. Paired associate tasks are one of thethe long-term association memory is stored in most widely used neuropsychological tasks inthe synaptic connections between TE and peri- humans to test for elementary pair-wise associa-rhinal cortex (Naya et al., 2003a; Yoshida et al., tions (Wechsler, 1987). Sakai and Miyashita

340 PRIMATE NEUROETHOLOGYFigure 18.2 Stimulus-stimulus association among the learned fractal patterns. (A) Examples of colorfractal patterns. (B) Location of recording sites. Top: A lateral view of macaque brain. Bottom: A sectionindicated by a vertical line on the lateral view. The stippled area represents the range of recording sites. rs,rhinal sulcus; amts, anterior middle temporal sulcus; a, anterior; p, posterior; m, medial; l, lateral. (C)Sequence of events in a trial of delayed matching-to-sample task. lev, lever press by the monkey; war,warning green image; sam, sample stimulus; mat, match stimulus following a 16-second delay; cho, choicesignal of white image. (D) Average delay discharge rate for each sample stimulus in a cell against ‘‘serialposition number’’ of the stimuli (see text). Stippled columns, learned stimuli; black columns, new stimuli.(E) As D, but for a different cell. Based on Miyashita, Y. (1988). Neuronal correlate of visual associative long-term memory in the primate temporal cortex. Nature, 335, 817–820. Used with permission.trained monkeys in the VPA task using 12 pairs negligibly when stimuli from any of the otherof Fourier descriptors (Fig. 18.3). In each trial, a pairs were presented as cue stimuli. This prop-cue stimulus was presented on a video monitor. erty indicates that memory storage is organizedAfter a delay period, two choice stimuli, the such that single neurons can code both pairedpaired associate of the cue (correct choice) and associates in the VPA task. Figure 18.4B showsa distracter from the other pairs (incorrect the response of a pair-recall neuron. For thischoice), were shown. The identity of the correct neuron, in the trial when the paired associatecombination of any given paired associate of the cue-optimal stimulus was presented as acannot be predicted without memorizing the cue stimulus, it exhibited the highest delayspecific pairs beforehand; in addition, the VPA activity among the stimuli. This stimulus-selec-task demands memory retrieval and thus gen- tive activity during the delay period is thought toeration of images from LTM. In this study, they represent the retrieval of a sought targetfound two types of task-related neurons in IT retrieved from LTM through the cue stimulus.cortex, ‘‘pair-coding’’ and ‘‘pair-recall’’ neu- To test this hypothesis directly, Naya and collea-rons. Figure 18.4A shows the responses of a gues (1996) manipulated the requirement andpair-coding neuron. Note that one stimulus eli- timing for retrieval explicitly using a modifiedcited the strongest response from this neuron VPA task with a ‘‘color-switch’’ signal shown induring the cue period (cue-optimal stimulus). the delay interval that indicated if the trialThis neuron was also activated when the paired required memory retrieval (VPA trial) or notassociate of the cue-optimal stimulus was pre- (DMS trial). When the paired associate of thesented. In contrast to the robust responses to this cue-optimal stimulus was presented as a cue, ITstimulus pair, the neuron responded only neurons started to fire just after the color switch

ASSOCIATIVE MEMORY IN THE MEDIAL TEMPORAL LOBE 341 A Paried Associates 1 1’ 5 5’ 9 9’Figure 18.3 Visual pairassociation (VPA) task used toassess long-term memory in 2 2’ 6 6’ 10 10’monkeys. (A) Twelve pairs ofFourier descriptors (1-1’ and12-12’) were used in the VPAtask. (B) Cue stimuli were 3 3’ 7 7’ 11 11’presented at the center of a videomonitor. Choice stimuli werepresented randomly in two of 4 4’ 8 8’ 12 12’four positions on the videomonitor. One is the pairedassociate of the cue (‘‘Correct’’);the other is a distracter from adifferent pair (‘‘Error’’). Basedon Sakai, K., & Miyashita, Y.(1991). Neural organization for Pair association trialthe long-term memory of pairedassociates. Nature, 354, 152–155; Correct Band Naya, Y., Yoshida, M., &Miyashita, Y. (2003a). Forwardprocessing of long-term Errorassociative memory in monkeyinferotemporal cortex. Journal of CueNeuroscience, 23, 2861–2871.Used with permission. Choicethat signaled the necessity for memory retrieval component of the medial temporal lobe memoryfor their cue-optimal stimuli (VPA trial). In system (Zola-Morgan & Squire, 1990), TE is acontrast, if there was no color switch, the same unimodal visual neocortical area located at theneurons showed no activation (DMS trial). This final stage of the ventral visual pathway, impor-finding suggests that the delay activity of IT tant for object vision (Janssen et al., 2000;neurons in the VPA task correspond to the Tanaka, 1996).requirement for the retrieval of a visual image Naya and his colleagues (2003a) mapped thefrom LTM. two subdivisions; a total of 2,368 neurons were recorded from A36 (510 neurons) and TE (1,858Forward Processing of Associative Memory neurons) in the three monkeys performingfrom Visual Cortex to Limbic Cortex the VPA task. Of those, 423 neurons (76 neuronsAfter the discovery of pair-coding and pair- in A36 and 347 neurons in TE) showedrecall neurons, the next critical challenge was to stimulus-selective responses (P <0.01, analysiselucidate the neuronal circuits that underlie the of variance [ANOVA]) during the cue periodresponse properties of these memory signals (cue-selective neurons). Their response latenciesduring a VPA trial. To address this problem, were significantly shorter in TE than A36 (mean,Naya and colleagues (2001, 2003a) examined 93.8 ms in A36 vs. 86.2 ms in TE, P <0.05),the interaction between the two major subdivi- confirming that the visual signal reached TEsions of IT cortex: A36 and TE. Note that while before it reached A36. This finding was consis-A36 is a limbic polymodal association area and a tent with their anatomical hierarchy: A36 is

342 PRIMATE NEUROETHOLOGY A Pair-coding neuron 50 50 spikes/s 0 0 6′ 6 6 6′ 1s Cue Choice Cue Choice B Pair-recall neuron 50 50 spikes/s 0 0 12 12′ 12′ 12 1s Cue Choice Cue ChoiceFigure 18.4 Responses of ‘‘pair-coding neuron’’ and ‘‘pair-recall neuron.’’ (A) A pair-coding neuron.Raster display and peri-stimulus time histogram (PSTH) in trials for cue 6’ (left) and for cue 6 (right). Thetrials were aligned at the cue onset. Note that cue 6’ and 6 elicited the strongest and the second strongest cueresponses from the neuron. (B) A pair-recall neuron. The neuron showed the strongest cue responses intrials for cue 12. Note the tonic increasing activity during the delay period in trials for cue 12’, which is muchhigher than the cue response. Based on Sakai, K., & Miyashita, Y. (1991). Neural organization for the long-term memory of paired associates. Nature, 354, 152–155; and Sakai, K., Naya, Y., & Miyashita, Y. (1994).Neuronal tuning and associative mechanisms in form representation. Learning & Memory, 1, 83-105. Usedwith permission.situated one synapse downstream of TE At the single neuron level, a substantial number(Felleman & Van Essen, 1991). The responses of neurons showed significantly (P <0.01) corre-of cue-selective neurons to paired associates lated responses to the paired associates (pair-were significantly correlated at the population coding neuron) in A36. The percentage of thelevel in both A36 and TE, but this correlation pair-coding neurons was much higher inwas much stronger in A36 than TE (median, 0.51 A36 than in area TE (33% in A36 vs. 4.9% inin A36 vs. 0.14 in TE) (Fig. 18.5A). TE, of the cue-selective neurons) (Fig. 18.5B).

ASSOCIATIVE MEMORY IN THE MEDIAL TEMPORAL LOBE 343The spatial distribution of the pair-coding percentage of the pair-coding neurons drama-neurons demonstrated that the pair-coding tically increased from TE to A36.neurons in TE were not necessarily distributed These striking differences between TE and A36in the region near the border with A36 raise the question of whether the pair-coding(Fig. 18.5C). This suggests that the percentage response of A36 neurons was elicited by a feedfor-of the pair-coding neurons did not increase in a ward input from TE or by a feedback input fromgradual manner from lateral to medial in IT other higher centers. To address this question,cortex. Moreover, within TE, there was no sub- initial transient responses after the cue stimulusregion where the percentage of the pair-coding presentations were examined for the pair-codingneurons was comparable with that in A36 neurons in A36. The analysis revealed that they(Fig. 18.5C). These anatomical observations were separable into two subtypes, and type 1supported the physiological result that the neurons (68%), but not type 2 (32%) neurons, A B Number of neurons [ratio] 0.2 Pair-coding / cue-selective A36 0.14* 0.51* 1.0 P < 0.001 TE *, P< 0.001 0.5 25 / 76 17 / 347 0.0 –1.0 0.0 1.0 0.0 Correlation coefficient Area 36 Area TE C Category of neurons Pair-coding Cue-selective Nonselective A M L P 5.0 mmFigure 18.5 Comparison of pair-coding responses between area 36 and area TE. (A) Response correlationto paired associates of cue-selective neurons in A36 (N = 76; green) and TE (N = 347; red). The correlationcoefficients for A36 neurons were significantly higher than those for TE neurons (P <0.001; Kolmogorov-Smirnov test). (B) The ratio of the pair-coding neurons among the cue-selective neurons was significantlyhigher (P <0.001; w2 test) in A36 (0.33; green) than TE (0.05; red). The pair-coding neurons were defined asthe cue-selective neurons that showed significantly positive PCI at the single-neuron level: P <0.01 (i.e.,correlation coefficient >0.71). (C) Spatial distributions of pair-coding neurons. The positions of the pair-coding (orange-filled diamond), cue-selective (black open square), and other recorded (black dot) neuronsare shown on two-dimensional unfolded maps for one monkey. Black lines, area borders; gray lines, fundusor lips of sulci; amts, anterior middle temporal sulcus; ots, occipital temporal sulcus; pmts, posterior middletemporal sulcus; rs, rhinal sulcus; sts, superior temporal sulcus; vl, ventral lip; A, anterior; P, posterior;L, lateral; M, medial. Scale bar, 5.0 mm. Based on Naya, Y., Yoshida, M., & Miyashita, Y. (2003a). Forwardprocessing of long-term associative memory in monkey inferotemporal cortex. Journal of Neuroscience, 23,2861–2871. Used with permission.

344 PRIMATE NEUROETHOLOGYbegan to encode associations between paired sti- converge onto an individual type 1 A36 neuronmuli as soon as they exhibited a stimulus-selective (Fig. 18.6B, the "selective-convergence" model).response (Fig. 18.6A). Thus, the representation of In other words, a single type 1 neuron in A36 canlong-term memory encoded by type 1 neurons in receive visual inputs from either of TE neuronA36 is likely generated without feedback input groups coding different but semantically linkedfrom other higher centers. Taken together, this objects (e.g., ‘‘glove’’ and ‘‘ball’’); therefore, theevidence suggests that the representation of sti- type 1 neuron can code both objects and/or theirmulus-stimulus association memory develops relationship.from TE to A36. Naya and colleagues (2003a)hypothesize that the forward processing of pair- Backward Spreading of Memory Retrievalassociation memory most likely requires selective Signal from Limbic Cortex to Visual Cortexconvergence such that the perceptual informationabout the individual elements of a learned paired After images are associated in LTM, what kind ofassociate is coded by separate TE neurons that signal is engaged during memory retrieval? A Type 1 neurons ( ) Type 2 neurons (O) Normalized response optimal pair optimal pair 1.0 1.0 other (average) other 0.0 0.0 0 320 0 320 Time from Cue Onset [ms] Time from Cue Onset [ms] B Selective convergence model TE A36 G or GB BFigure 18.6 Initial responses of pair-coding neurons and ‘‘selective convergence’’ model. (A) Population-averaged peri-stimulus time histogram (PSTH) for the type 1 (N = 17; left) and type 2 (N = 8; right)neurons, showing the normalized responses in the cue-optimal (green), pair (light green), and other (darkgreen) trials. Note that two thirds of pair-coding neurons in A36 showed initial transient responses in bothcue-optimal and pair trials. (B) Selective convergence model of association memory in inferior temporal(IT) cortex. Suppose that a glove and a ball are associated together in memory. In TE, these images arerepresented by different neurons (neuron G for a glove and neuron B for a ball). The neurons G and Bproject their fibers to the same A36 neuron, in particular, to type 1 neuron (neuron GB). In this way neuronGB can code the paired items regardless of their different visual/geometrical properties. Based on Naya, Y.,Yoshida, M., & Miyashita, Y. (2003a). Forward processing of long-term associative memory in monkeyinferotemporal cortex. Journal of Neuroscience, 23, 2861–2871. Used with permission.

ASSOCIATIVE MEMORY IN THE MEDIAL TEMPORAL LOBE 345Figure 18.7 shows examples of pair-recall neu- what kind of additional process might berons in A36 and TE (Naya et al., 2001). The A36 involved in this slow, backward spreading ofneuron exhibited sustained activity earlier than mnemonic signal. Theoretically, there are twothe TE neuron when the paired associates of the kinds of signals thought to be conveyed in thecue-optimal stimuli were presented as cue sti- delay interval of the VPA task. The first is amuli (Fig. 18.7A, bottom). The time course of retrospective signal that is closely coupledthe pair-recall activity of each neuron was exam- with the to-be-retained cue stimulus. Theined by considering the responses to all cue second is a prospective signal that is coupledstimuli. In this study, a partial correlation ana- with the to-be-retrieved target stimulus. Tolysis was used to evaluate the response correla- compare retrospective and prospective signalstion with the target to be retrieved. This measure quantitatively, the signal contents of the delay-was useful because IT neurons have a tendency period activity were characterized by partialto encode both of the paired stimuli and the correlation coefficients of delay-period activ-correlation between cues and their targets must ities for each cue stimulus with the cue-periodbe removed from the analysis. The partial corre- responses to that stimulus (cue-holding indexlation coefficients of instantaneous firing rates at [CHI]) and with the cue-period responses to itstime t for each cue stimulus were calculated with paired associate (pair-recall index [PRI]). Thethe visual responses to its paired associate (Fig. delay-period activity of TE neurons preferen-18.7C, pair-recall index, PRI(t)). To characterize tially represented the paired associate (PRI,the time course of PRI(t), two parameters, tran- median = 0.54) rather than the cue stimulussition time (TRT) and transition duration itself (CHI, 0.23) (P <0.001, PRI vs. CHI,(TRD), were determined for each single N = 70), while the delay-period activity of A36neuron. TRT was defined as the period from neurons retained both the cue stimulus and itsthe cue onset to the instant when the PRI(t) paired associate equivalently (CHI, 0.44; PRI,curve reached 50% of its full increase, and TRD 0.46) (P = 0.78, N = 38) (Naya et al., 2003b).was defined as the duration between the instants These results indicate that TE mostly representswhen the curve reached 10% and 90% of its full a sought target that is retrieved from long-termincrease. The TRT values for the pair-recall neu- memory during the delay interval, while A36 inrons in A36 were significantly smaller than those addition retains a cue stimulus that is trans-in TE (A36, median 206 ms; TE, median 570 ms; mitted from earlier visual areas.Kolmogorov-Smirnov test, P <0.005) (Fig. If back-projections from A36 to TE drive the18.8A, left). On the other hand, the distributions delay activity seen in TE, it is surprising that A36of TRD values did not differ between the two signals both sensory-related retrospective as wellareas (A36, median 115 ms; TE, median 145 ms; as prospective information while TE only signalsP >0.8) (Fig. 18.8A, right). These results indi- prospective information. One interpretation ofcated that the memory-retrieval signal emerges this finding may be that the backward signalearlier in A36, and TE neurons were then gradu- transmission from A36 to TE is equipped withally recruited to represent the retrieved image a selective gating mechanism, which preferen-(Fig. 18.8B) (Naya et al., 2001). The median tially passes information about a sought targetretrieval time was over 300 ms longer in TE (Naya et al., 1996). Another interpretation maythan in A36. Given the fact that TE neurons be that there are some mechanisms of activelyreceive numerous back-projections from A36, a inhibiting irrelevant information in TE. Thereasonable interpretation is that the mnemonic finding of the strong sought-target–relatedsignal spreads backward from A36 to TE. activity of TE neurons during the VPA task is Compared with the forward transmission of consistent with the report of Sheinberg andthe visual signal from TE to A36, the backward Logothetis (1997) showing that TE reflected per-mnemonic signal from A36 to TE exhibited a ceptually relevant activity during a binocularsurprisingly long latency (%10 ms vs. %300 rivalry task, in which visual ambiguity wasms). Naya and colleagues (2003b) next asked induced by presenting incongruent images to

346 PRIMATE NEUROETHOLOGYFigure 18.7 Comparison of pair-recall responses between A36 (left) and TE (right). (A) Raster displaysand peri-stimulus time histograms (PSTHs) were aligned at the cue onset in trials with the cue-optimalstimulus as a cue (upper) and in trials with its paired associate as a cue (lower). In the PSTHs, black linesindicate responses to the cue-optimal stimulus (upper) or its paired associate (lower), and gray linesindicate mean responses to all 24 stimuli. (B) Mean discharge rates during the cue (upper) and delay(lower) periods are shown for each cue presentation (mean – SEM). A stimulus-selective delay activity wasclosely coupled with a strong cue response to its paired associate. (C) Temporal dynamics of responsecorrelation are shown; the values of the pair-recall index (PRI) are plotted against the time axis and are fittedwith sigmoid functions (solid). The vertical lines, intersecting the best-fit sigmoid functions, indicate thetransition times (TRTs). The shaded areas indicate the transition durations (TRDs). Based on Naya, Y.,Yoshida, M., & Miyashita, Y. (2001). Backward spreading of memory-retrieval signal in the primatetemporal cortex. Science, 291, 661–664. Used with permission.the two eyes. During the binocular rivalry con- exclusively when the driving stimulus is seen.dition, one image is seen at a time while the They suggested that TE is the place where theother is perceptually suppressed. In their study, neural activity reflects the brain’s internal viewalmost all TE neurons (%90%) discharge of objects. The report by Sheinberg and

ASSOCIATIVE MEMORY IN THE MEDIAL TEMPORAL LOBE 347 A TRT TRD 1.0 A36 (N = 20) cumulative no. of cells 1.0 Proportion of TE (N = 29) * 0.5 A36 (N = 20) TE (N = 29) P < 0.005 P = 0.82 0.0 0 1000 0 1000 Transition time [ms] Transition duration [ms] B A36 TE Pair-recall signal Cue TimeFigure 18.8 Time courses of pair-recall singnals for A36 and TE neurons. (A) Cumulative frequencyhistograms of the transition time (TRT; left) and the transition duration (TRD; right) for A36 (green)and TE (red) neurons. TRTs for A36 neurons were significantly shorter than those for TE neurons(asterisk, Kolmogorov-Smirnov test, P <0.005). (B) A schematic view of the developments of pair-recallsignals in A36 and TE. The similar distribution of TRD between A36 and TE indicates the similar rate ofdevelopment of pair recall signals, but their onsets were different as illustrated by the differentdistributions of TRT between the two areas. Based on Naya, Y., Yoshida, M., & Miyashita, Y. (2001).Backward spreading of memory-retrieval signal in the primate temporal cortex. Science, 291, 661–664.Used with permission.Logothetis (1997) suggests that the dominance colleagues (1998) who recorded activity in theof sought-target–related activity in TE during perirhinal cortex during a DMS task. Theythe VPA task may also reflect the brain’s reported that perirhinal neurons convey sti-‘‘internal view’’ that derives from LTM and cor- mulus-selective sustained activity after a test sti-responds to mental imagery in the VPA task. On mulus that continued through the inter-trialthe other hand, the perception-derived activity interval, despite the fact that information aboutin A36 corresponds to a retrospective signal that the test stimulus is not relevant for the subse-is not necessarily required to solve the VPA task. quent trial. They suggested that this stimulus-This task-irrelevant retrospective signal is con- selective retrospective activity may serve tosistent with the findings of Yakovlev and generate long-term associations between visual

348 PRIMATE NEUROETHOLOGYitems which are temporally adjacent but sepa- be coupled with an explicit recall of a specificrated by some interval (Miyashita, 1988). modality. While the exclusive representation of asought target in TE implicates its critical con- Horizontal Cascades of Semantically Linkedtribution to a mental image of a retrieved Information in the Limbic Cortexobject, the additional representation of a cue The studies described previously show how anstimulus in A36 may implicate its contribution association of objects proceeds forward from TEto the formation of LTM because the retro- to A36 and retrieval signals spreads backwardspective signals can bridge the cue information from A36 to TE. The remaining question isto its pair regardless of the behavioral rele- how the pair-recall signal was generated fromvance. In addition, filtering out the automati- the association signal coded by type 1 neurons.cally driven perception-derived signal may This problem might be best addressed bycause the slow (%300 ms) backward transmis- combining a reverberation circuit modelsion of LTM-derived signal from A36 to TE. (Amit et al., 1994; Yakovlev et al., 1998) with aThis slow transmission from A36 (polymodal ‘‘selective-convergence’’ model (‘‘rb-selective-limbic area) to TE (sensory neocortex) might convergence model’’). According to the rb-Selective convergence model Type 2 Stimulus G Stimulus G G Stimulus B Stimulus B G G G G Type 1 Stimulus G GB Stimulus B B B B Type 2 Stimulus G B B Stimulus G Stimulus B Stimulus B TE Area 36Figure 18.9 Selective-convergence model with reverberation circuits. Solid and dashed arrows denotesynaptic connections within and between areas, respectively. When stimulus G is presented as a cuestimulus, TE neurons coding stimulus G are activated and the activation transmits forward to A36. Theactivated neuronal group in A36 (neuronal group G) contains type 1 neurons that code both stimulus B aswell as stimulus G. The activation of neuronal group G elicits the following activation of the neuronal groupB within A36 via type 1 neurons. Finally, the information of stimulus B spreads backward from A36 to TE.When stimulus B is presented as a cue stimulus, the signal transmits in reverse. Expected neuronal responsesare shown as peri-stimulus time histograms (PSTHs) for each cue stimulus.

ASSOCIATIVE MEMORY IN THE MEDIAL TEMPORAL LOBE 349reverberation circuit model, the selective delay memory (Eichenbaum & Cohen, 2001;activity in IT cortex is maintained by recurrent Scoville & Milner, 1957; Squire & Zola,synaptic feedback between interconnected neu- 1996). In the following section, we turn torons within a local module coding an individual studies that have investigated the dynamic sig-stimulus. Because the percentage of neurons that nals seen during new associative learning inshowed stimulus-selective sustained activity was nonhuman primates. We first describe amuch larger in A36 (53% of the cue-selective series of studies aimed at examining theneurons) than TE (21%) (Naya et al., 2003b), neural correlates of learning the same kind ofthe synaptic connections within each module VPA task used by Miyashita and colleagues.may be stronger in A36 (denoted as thick These studies provide insight into how thearrows in Fig. 18.9) than in TE (thin arrows). striking pair-coding signals described in theConsidering that type 1 neurons belong to neu- perirhinal cortex may develop during theronal ensembles coding either of the paired early learning process. We next turn to asso-associates, an activation of neuronal ensemble ciative learning signals seen during condi-coding a cue stimulus may spread to another tional motor association (CMA) tasks. Notneuronal ensemble coding its paired associate only is this latter task dependent on the integ-via type 1 neurons (Fig. 18.9). rity of MTL (Brasted et al., 2002, 2003; Murray Taken together, long-term stimulus-stimulus & Wise, 1996; Murray et al., 2000; Rupniak &association memory is stored in perirhinal Gaffan, 1987; Wise & Murray, 1999; ), but alsocortex as a form of partial overlapping neuronal multiple new associations can be learned con-ensembles coding individual items. The partial currently within the course of a singleoverlap is constructed by pair-coding neurons, recording session, affording the ability toparticularly type 1 neurons. Type 1 neurons examine in more detail the temporal relation-receive direct projections from separate TE ship between the changes in neural activity andneuron groups coding individual visual items. behavioral performance during fast new asso-Next, the retrieval from the long-term associa- ciative learning.tion memory is divided into two processes. Thefirst retrieval process occurs in perirhinal cortex Associative Learning Signals During VPAas a horizontal cascade of activation from one tasksneuronal ensemble to another. Second, theretrieved information originated in perirhinal As discussed in part I, Miyashita and his collea-cortex spreads backward to TE. The activated gues (Naya et al., 2003a; Sakai & Miyashita,TE neurons represent a mental image of the 1991) showed that visual stimuli paired togetherretrieved item. Thus, according to this model, in long-term memory become associated suchthe perirhinal cortex is critical for the storage that some neurons respond similarly to bothand retrieval of long-term association memory; associated images (i.e., pair-coding neurons).however, the interaction with neocortex (e.g., One hypothesis suggests that with learning, neu-TE) is essential for its implementation. rons that initially respond to one of the stimuli in a learned pair eventually come to respond to the learned paired associate of that original sti-PART II: NEURONAL SIGNALS mulus. This ‘‘tuning’’ of the neuron’s stimulus-UNDERLYING THE FORMATION selective activity is thought to occur through theOF NEW ASSOCIATIVE strengthening of connections between neuronsMEMORIES that initially respond to the two stimuli thatStrong and convergent evidence from both eventually become paired in memory. To testhuman neuropsychological studies as well as this hypothesis directly, Messinger and collea-experimental studies in animals suggests that gues (2001) recorded activity in the perirhinalthe medial temporal lobe is critical in the cortex and the adjacent visual area TE as animalsinitial stages of many forms of associative learned novel associations during a VPA task

350 PRIMATE NEUROETHOLOGYFigure 18.10 Visual paired associate learning task. (A) Schematic representation of the pairs of visualstimuli learned by the animal each day. In this case, airplane was always paired with basketball and corn wasalways paired with dinosaur. For each trial, after fixation, animals were shown a sample stimulus, followedby a choice array. The extinguishing of the fixation spot was the animal’s cue to make an eye movement tothe correct paired associate of the sample stimulus shown. (B) The change in neural activity of inferiortemporal (IT) cells over the course of the training session to the four different stimuli. Note that theneuron’s response to stimulus A comes to resemble the response to stimulus B (its paired associate) whileresponses to stimuli C and D remain low. Based on Messinger, A., Squire, L. R., Zola, S. M., & Albright, T. D.(2001). Neuronal representations of stimulus associations develop in the temporal lobe during learning.Proceedings of the National Academy of Sciences, 98, 12239–12244. Copyright (2001) National Academy ofSciences, U.S.A. Used with permission.(Fig. 18.10A). In their paradigm, each day, the In a follow-up study using the same data set,monkeys learned two new paired associates. Messinger and colleagues (2005) examined sig-Consistent with their initial working hypothesis, nals observed during the choice period of thethey found that over the course of the learning task when the two choice stimuli were visible onsession, neurons in both the perirhinal cortex the screen but before the behavioral responseand area TE came to respond more similarly to had been made. They reported that neural sig-two of the stimuli that were paired in memory nals during the choice phase, like the sample(Fig. 18.10B). This increase in correlated phase analyzed in Messinger and colleaguesresponse, measured by a ‘‘pair-coding index,’’ (2001), also reflected learning. Using anwas specific to those sessions in which behavioral ANOVA to contrast the effects of instructedperformance increased significantly above (i.e., correct) stimuli and chosen stimuli, 54%chance levels, though peak performance in the of their IT cells reflected information about thelast quartile of the session reached only 55.4% instructed stimulus rather than the chosen sti-correct (chance = 50% correct). Values of the mulus in the choice period of the task. In otherpair-coding index did not change in sessions words, the neurons signaled the correct associa-when the animal did not learn. Because the tion even when the animal made an incorrectchanges in correlated neural activity appeared response. Consistent with the idea that thisto parallel the time course of behavioral learning instructed stimulus signal develops with(i.e., gradually increasing over the four quartiles learning, they further showed that the instructedof the session), this suggests that these changes in stimulus signal increased significantly betweenthe neuron’s stimulus-selective response proper- the first and second half of the session.ties underlie new associative learning. Messinger and colleagues (2005) suggested that

ASSOCIATIVE MEMORY IN THE MEDIAL TEMPORAL LOBE 351Figure 18.11 Predictive go/no-go task. (A) In the predictive go/no-go task, a predictor stimulus wasshown for 500 ms followed by a delay interval and then a choice stimulus was presented for 500 ms. Animalslearned which choice stimuli required a bar release (‘‘go’’ stimulus) and which required a bar hold (‘‘no-go’’stimulus) and they could also learn the relationship between the predictor stimulus and the choice stimulus.(B) The correlation between the responses to paired predictor and choice stimuli were significantly strongerfor familiar pairs (pairs seen for more than 1 day) compared to novel pairs. In contrast, the correlationbetween predictor and choice stimuli was near zero for randomly paired stimuli (shuffled). Based onErickson, C. A., & Desimone, R. (1999). Responses of macaque perirhinal neurons during and after visualstimulus association learning. Journal of Neuroscience, 19, 10404–10416. Used with permission.this apparent dissociation between neural repre- after several days of experience with the stimulisentation of instructed stimuli on the one hand (Fig. 18.11B). In contrast, Messinger and collea-and behavioral choice on the other may be gues (2001) reported that the changes in neuralexplained by the effect of behavioral states, activity appeared to parallel behavioral learningattention, or motivation that may mask the within the time course of a single learning ses-true state of knowledge the animals accrue sion. The difference in the timing of the neuralduring learning. signals seen in the VPA task used by Messinger Erickson and Desimone (1999) recorded and colleagues (2001) and the predictive go/no-activity in the monkey perirhinal cortex as ani- go task used by Erickson suggests that the timingmals learned a different kind of associative of the neural signals seen in the perirhinal cortexlearning task (Fig. 18.11A). In their task, a pre- relative to behavior can vary depending on thedictor stimulus was followed by a choice sti- specific task demands.mulus. The choice stimulus could either signal While the studies by Messinger and collea-the animals to release a bar (‘‘go’’ condition) or gues (2001, 2005) and Erickson and Desimonecontinue holding a bar (‘‘no-go’’ condition). In (1999) have provided important new insightthis task, the animal was not required to learn into the neural correlates underlying the forma-the explicit association between the predictor tion of new paired associates, because both tasksand the choice, but knowledge of this association have a relatively slow learning curve, these taskscould allow the animal to respond more quickly are not optimal to address the kind of fast andwhen the choice was presented. Learning was flexible associative learning typically attributeddefined by a significantly faster responses on to the hippocampus (Eichenbaum & Cohen,the ‘‘valid’’ trials illustrated in Figure 18.11A 2001). To examine fast associative learning inand probe trials in which the predictor stimuli the MTL, several groups have turned to thebetween go and no-go trials were reversed. The CMA learning task in which multiple new asso-neural responses to predictor and choice stimuli ciations can be learned easily in a single trainingwere initially uncorrelated for novel stimuli used session. In contrast to the studies using the VPAfor 1 day but became significantly correlated task that have focused on neural activity in IT

352 PRIMATE NEUROETHOLOGY A Location scene sssociation (CMA) task + + + Fixation/ Scene/target Delay Eye movement baseline presentation (700 ms) response (300 ms) (500 ms) B Sustained C Baseline sustained 1 50 1 16 r = 0.96 r = –0.83 14 Response (spikes/s) 0.8 40 0.8 Percent correct 12 30 0.6 0.6 10 20 8 0.4 0.4 6 10 4 0.2 0 0.2 2 Baseline activity 0 –10 0 0 0 10 20 30 40 50 1 4 7 10 13 16 19 22 25 28 31 34 37 Trials TrialsFigure 18.12 Location scene association (CMA) task and changing cells. (A) Schematic representation ofthe location scene association task. In this task, following fixation, animals are shown a set of four identicalvisual targets superimposed on a complex visual image (images used in task were all in color). Following adelay interval during which time the targets remained on the screen, but the scene disappeared, the animal wascued to make an eye movement response (illustrated schematically by the white arrow) to one of the targets.Only one of the targets was rewarded for each particular scene. Animals learned by trial and error to associateeach new scene with a particular eye movement response. (B,C) The trial-by-trial probability correctperformance (dotted line read from the left axis) as a function of the trial-by-trial activity of cells duringeither the scene or delay period of the task (solid line read from right axis) for a sustained (B) and baselinesustained (C) cell. Note the strong positive or negative correlation between neural activity and learning. Filledcircles at the top of the graphs represent individual incorrect trial, while open circles denote correct trials.Based on Wirth, S., Yanike, M., Frank, L. M., Smith, A. C., Brown, E. N., & Suzuki, W. A. (2003). Singleneurons in the monkey hippocampus and learning of new associations. Science, 300, 1578–1581.cortex, the studies examining CMA learning in visual image presented on a computer screen) withthe medial temporal lobe have focused on neural a motor response (i.e., look right or touch right).representation in the hippocampus. Posttraining lesions to the MTL in monkeys impair the ability to learn novel CMAs, whileAssociative Learning Signals During well-learned associations remain unaffectedConditional Motor Association Tasks (Brasted et al., 2002, 2003; Murray & Wise, 1996;Both Wirth and colleagues (2003) and Cahusac Murray et al., 2000; Rupniak & Gaffan, 1987;and colleagues (1993) examined the patterns of Wise & Murray, 1999).hippocampal activity during the learning of novel In the study by Wirth and colleagues (2003),CMAs (Cahusac et al., 1993; Wirth et al., 2003). animals performed a CMA task also referred toThis category of associative learning task, also as the location scene association task (Fig.known as arbitrary sensory motor mapping or 18.12A). In this task, animals were first shownconditional visuomotor learning, requires animals four identical target stimuli superimposed on ato associate a given sensory stimulus (typically a complex visual scene that filled the video

ASSOCIATIVE MEMORY IN THE MEDIAL TEMPORAL LOBE 353monitor. Following a delay interval, during increases in neural activity also correspondedwhich the scene disappeared but the targets with an overall increase in the neuron’s selec-remained on the screen, the animal was cued to tive (i.e., differential) response to all stimulimake a single eye movement to one of the per- examined (Fig. 18.13B).ipheral targets on the screen. For each visual In the Wirth et al. (2003) study, a secondscene, only one of the four targets was associated category of learning-related cells termed ‘‘base-with reward. Each day, animals typically learned line sustained changing cells’’ made up thetwo to four new scenes by trial and error. These remaining 45% of changing cells. Baseline sus-new scenes were also randomly intermixed with tained cells started out with a scene-selectivewell-learned ‘‘reference’’ scenes that the animals response during either the scene or delayhad seen for many months before the recording period of the task even before the animal learnedexperiments began. Responses to the reference the association and signaled learning byscenes were used to control for motor-related returning to baseline activity (Fig. 18.12C).activity in the hippocampal cells. Baseline sustained cells were also characterized Wirth and colleagues (2003) reported that by a significantly decreased selectivity (i.e., dif-61% of the hippocampal cells examined ferential activity) to all stimuli examined (Fig.responded selectively (i.e., differentially) to 18.13C). While Cahusac and colleagues (1993)the different visual scenes shown in the task did not describe baseline sustained cells, they didduring the scene period, the delay period, or describe a different population of hippocampalboth periods of the task. Selectively responding neurons that only showed differential activity tocells with learning-related activity were identi- the two visual stimuli transiently, near the timefied by correlating a moving average of the raw of learning (transient cells). While Wirth andneural activity with a moving average of the colleagues (2003) looked for transient learningraw behavioral performance during learning. signals in the hippocampus, few if any wereUsing this criterion, 28% of the selectively observed (unpublished observations). Transientresponding cells showed a significant positive learning signals have also been described in theor negative correlation with learning. These striatum during a conditional motor learningcells were termed ‘‘changing cells.’’ Two cate- task (Williams & Eskandar, 2006).gories of changing cells were described. While the learning-related hippocampalSustained changing cells (54% of the popula- activity described by both Wirth and colleaguestion of changing cells) signaled learning with a (2003) and Cahusac and colleagues (1993)change in neural activity that was maintained show that cells in this region signal learning,for as long as the cell was able to be recorded another critical question concerns the timing offrom (Fig. 18.12B). These cells typically started the changing cells relative to behavioralout with little or no response during the trial learning. In other words, does the changingand signaled learning with a dramatic increase neural activity lead, lag, or occur in parallelof activity during either the scene or delay with the animal’s behavioral learning? Toperiods of the task. This change in neural address this question, Wirth and colleaguesactivity was strongly correlated with behavioral (2003) compared the estimated trial numberlearning. A similar pattern of sustained of neural change and the estimated trialchanges in neural activity was also described number of learning for each changing cell.by Cahusac and colleagues (1993) in the hip- This comparison showed that hippocampalpocampus. Importantly, because the same level cells can signal learning before (N = 18), at theof enhanced activity was not seen in response same time (N = 1), and after (N = 18) learning.to the reference scene with the same rewarded Hippocampal cells signaled learning staringtarget location in the Wirth et al. study (2003), from as much as 13 trials before learning tothis showed that the changes in neural activity 15 trials after learning (Fig. 18.14). Similar tocould not be attributed to a pure motor-based the Wirth et al. (2003) study, Cahusac andresponse. Further analysis showed that these colleagues (1993) reported that that the

354 PRIMATE NEUROETHOLOGYA 14 1 Ref 1 Ref 4 12 Ref 2 New 1 0.8 Response (spikes/s) Ref 3 New 2 Probability correct 10 Figure 18.13 Changes in 0.6 selectivity with learning.(A) 8 Average response to four Learning curve 6 for new 2 reference scenes and two new 0.4 scenes over the course of the 4 recording session for a sustained 0.2 changing cell. The learning curve 2 for New Scene 2 is illustrated in as the thick gray line. Trial number 0 0 1 11 21 31 41 51 61 71 81 91 101 111 refers to the consecutive times Trials that particular condition/scene was shown during the session. (B) B C Graph showing the significant 0.7 0.8 * * increase in selectivity index for 0.6 0.7 sustained changing cells after 0.6 learning compared to before 0.5 Selectivity index learning. (C) In contrast, baseline 0.5 sustained changing cells decreased 0.4 0.4 their selectivity after learning 0.3 compared to before learning. 0.3 Based on Wirth, S., Yanike, M., 0.2 Frank, L. M., Smith, A. C., Brown, 0.2 E. N., & Suzuki, W. A. (2003). 0.1 0.1 Single neurons in the monkey 0 0 hippocampus and learning of new Before After Before After associations. Science, 300, Learning Learning 1578–1581.learning-related signals could occur within a SUMMARY AND CONCLUSIONSwide range of lag or lead times relative to beha- The goal of this chapter was to describe ourvioral learning ranging mainly from 30 trials current understanding of the neurophysiologicalbefore learning to 40 trials after learning. properties of the MTL memory system for bothTaken together, the studies of learning-related long-term associative memory signals seen afteractivity in the hippocampus could change the association is well established and theeither before learning, in parallel with learning, dynamic associative learning signals observedand after learning. This is the general pattern during various tasks of new associative learning.one would expect from a network with feedback We have focused primarily on two key tasks ofconnections that participates in the learning association, the visual paired associate task stu-process. That is, some cells change before died most extensively in the perirhinal cortexlearning is expressed, possibly playing a role in (Naya et al., 1996, 2001, 2003a, 2003b; Sakai &driving the early learning process, but with Miyashita, 1991) and the conditional motorfeedback, other cells in the network get association task that has been studied in therecruited later in the learning process, with the hippocampus (Cahusac et al., 1993; Wirthaverage population of cells changing around et al., 2003). The results summarized in thisthe time of learning. chapter showed similarities as well as differences

ASSOCIATIVE MEMORY IN THE MEDIAL TEMPORAL LOBE 355 Neurons that change after learning 50 40 Neurons that change Neuronal change before learning 30 20 10 0 0 10 20 30 40 50 Behavioral changeFigure 18.14 Temporal relationship between neural activity and behavior. Scatter plot illustrating thetemporal relationship between trial number of behavioral change (i.e., learning) and trial number ofneuronal change. Note that about half the cells change before or at the same time as learning, while theremaining half of the cells change before learning. Based on Wirth, S., Yanike, M., Frank, L. M., Smith, A. C.,Brown, E. N., & Suzuki, W. A. (2003). Single neurons in the monkey hippocampus and learning of newassociations. Science, 300, 1578–1581.in both the behavioral correlates as well as the has been well established, strong pair-codingneural correlates underlying these two tasks. signals continue to be observed in the perirhinalOne key difference between the VPA task and cortex. The high proportion of pair-coding sig-the CMA task is the overall speed of learning. nals observed in the perirhinal cortex may reflectWhile multiple new conditional motor associa- the important role of this area in storage of thetions can be learned easily within the course of a long-term representation of the associated sti-single training session (Wirth et al., 2003), new muli. These findings are also consistent withvisual-visual paired associates are more difficult findings from lesion studies showing thatto learn quickly (Messinger et al., 2001), but with damage to the perirhinal cortex impairedexperience, large sets of visual paired associates memory for well-learned visual paired associatescan be learned to a high level of performance in addition to impairing learning of new paired(Naya et al., 2003a). In this way, the slower time associates (Murray et al., 1993). In addition,course of learning for the VPA task suggests that pair-recall signals were also described in theit may be more analogous to semantic memory perirhinal cortex with extensive previouswhile the CMA task may be more analogous to training (Naya et al., 2001). These signals havequickly developing and more flexible relational been taken as a neural signal underlying retrievalmemories thought to be dependent on the of the learned paired associates. We further sug-hippocampus (Eichenbaum & Cohen, 2001). gest that while the perirhinal cortex is critical for During learning of novel visual paired associ- storage and retrieval of visual paired associates,ates, Messinger and colleagues (2001) showed area TE may be critical for the mental represen-that perirhinal neurons change their stimulus- tation or imagery of the retrieved information.selective responses to come to respond more In contrast to the VPA task, the CMA task issimilarity to two stimuli that have been paired learned quickly and neural signals that parallelin memory. When assessed after VPA learning this relatively fast learning can be seen in the

356 PRIMATE NEUROETHOLOGYmonkey hippocampus (Cahusac et al., 1993; frontal eye field (Chen & Wise, 1995, 1997),Wirth et al., 2003). Like the signals in the peri- striatum (Brasted & Wise, 2004; Pasupathy &rhinal cortex, these signals in the hippocampus Miller, 2005; Williams & Eskandar, 2006), andappear to represent a selective tuning of the cell’s prefrontal cortex (Asaad et al., 1998) suggest thatstimulus-selective response properties that all of these other structures can also participatechanges with learning (Wirth et al., 2003). importantly in the initial learning and possiblyHippocampal learning-related cells could either the storage of CMAs. It will be of great interest tobecome more selective (sustained changing examine the contribution of these different braincells) or less selective (baseline sustained chan- areas in parallel during the course of theging cells) or provide a temporally specific signal establishment of LMTs of CMAs. Both the CMAof learning (transient cells). These dynamic and VPA task will continue to be powerful tools tochanges may represent an overall tuning of the understand both how different MTL areas interacthippocampal population with new associative during associative learning and memory tasks andlearning. how those signals develop and change over time While these findings point to an important with the development of LTM.role of the perirhinal cortex in slow semantic-like visual paired associative learning andmemory and toward the hippocampus for fast REFERENCESnew associative learning, an important questionfor future research concerns how these two areas Amit, D. J., Brunel, N., & Tsodyks, M. V. (1994). Correlations of cortical Hebbian reverberations:may interact during both the learning and long- Theory versus experiment. Journal ofterm representation of VPA and CMA tasks. For Neuroscience, 14, 6435–6445.example, while findings from lesion studies sug- Asaad, W. F., Rainer, G., & Miller, E. K. (1998).gest that the hippocampus is not needed for new Neural activity in the primate prefrontal cortexVPA learning (Murray et al., 1993), neural corre- during associative learning. Neuron, 21,lates in this structure may be involved in contex- 1399–1407.tual learning associated with the task. Similarly, Brasted, P. J., Bussey, T. J., Murray, E. A., & Wise,we now have evidence that the perirhinal cortex S. P. (2002). Fornix transection impairssignals a new CMA in a similar way to the hippo- conditional visuomotor learning in taskscampus suggesting cooperative learning signals involving nonspatially differentiated responses.across these regions (Yanike et al., 2009). It will Journal of Neurophysiology, 87, 631–633. Brasted, P. J., Bussey, T. J., Murray, E. A., & Wise,be important to record simultaneously across S. P. (2003). Role of the hippocampal system indifferent MTL structures during both VPA and associative learning beyond the spatial domain.CMA learning to better understand the relative Brain, 126, 1202–1223.roles of the MTL during new associative learning. Brasted, P. J., & Wise, S. P. (2004). Comparison of Another important but unanswered ques- learning-related neuronal activity in the dorsaltion concerns understanding the mechanisms premotor cortex and striatum. Europeanunderlying the gradual consolidation of both Journal of Neuroscience, 19, 721–740.VPA and CMA information in LTM. What are Buckley, M. J., & Gaffan, D. (1998). Perirhinal cortexthe cellular mechanisms underlying the role of ablation impairs visual object identification.the perirhinal cortex in both initial learning and Journal of Neuroscience, 18, 2268–2275.long-term storage, and how does the commu- Bunsey, M., & Eichenbaum, H. (1993). Critical role of the parahippocampal region for paired-nication of the perirhinal cortex with area TE associate learning in rats. Behavioralchange over time with consolidation? Similarly Neuroscience, 107, 740–747.for the CMA task, how does the long-term Cahusac, P. M., Rolls, E. T., Miyashita, Y., & Niki,representation of the CMAs eventually move H. (1993). Modification of the responses offrom the hippocampus early on to other as- hippocampal neurons in the monkey duringyet-unidentified storage regions? Important the learning of a conditional spatial responseparallel studies in supplementary eye field and task. Hippocampus, 3, 29–42.

ASSOCIATIVE MEMORY IN THE MEDIAL TEMPORAL LOBE 357Chen, L. L., & Wise, S. P. (1995). Supplementary Milner, B. (2005). The medial temporal-lobe eye field contrasted with the frontal eye field amnesic syndrome. Psychiatric Clinics of North during acquisition of conditional oculomotor America, 28, 599–611. associations. Journal of Neurophysiology, 73, Mishkin, M. (1978). Memory in monkeys severely 1122–1134. impaired by combined but not by separateChen, L. L., & Wise, S. P. (1997). Conditional removal of amygdala and hippocampus. oculomotor learning: Population vectors in Nature, 273, 297–298. the supplementary eye field. Journal of Mishkin, M. (1982). A memory system in the Neurophysiology, 78, 1166–1169. monkey. Philosophical Transactions of theCorkin S, (1968) Acquisition of motor skill after Royal Society of London [Biology], 298, 83–95. bilateral medial temporal-lobe excision. Miyashita, Y. (1988). Neuronal correlate of visual Neuropsychologia 6, 255-65. associative long-term memory in the primateEichenbaum, H., & Cohen, N. J. (2001). From temporal cortex. Nature, 335, 817–820. conditioning to conscious recollection. New Miyashita, Y., & Chang, H. S. (1988). Neuronal York: Oxford University Press. correlate of pictorial short-term memory inEichenbaum, H., Yonelinas, A. R., & Ranganath, C. the primate temporal cortex. Nature, 331, (2007). The medial temporal lobe and 68–70. recognition memory. Annual Review of Murray, E. A., Bussey, T. J., & Wise, S. P. (2000). Neuroscience, 30, 123–152. Role of prefrontal cortex in a network forErickson, C. A., & Desimone, R. (1999). Responses arbitrary visuomotor mapping. Experimental of macaque perirhinal neurons during and after Brain Research, 133, 114–129. visual stimulus association learning. Journal of Murray, E. A., Gaffan, D., & Mishkin, M. (1993). Neuroscience, 19, 10404–10416. Neural substrates of visual stimulus-stimulusFelleman, D. J., & Van Essen, D. C. (1991). association in rhesus monkeys. Journal of Distributed hierarchical processing in the Neuroscience, 13, 4549–4561. primate cerebral cortex. Cerebral Cortex, 1, Murray, E. A., & Wise, S. P. (1996). Role of the 1–47. hippocampus plus subjacent cortex but notGabrieli J.D., Milberg W., Keane M.M. & Corkin S. amygdala in visuomotor conditional learning (1990) Intact priming of patterns despite in rhesus monkeys. Behavioral Neuroscience, impaired memory. Neuropsychologia. 28, 110, 1261–1270. 417–427. Naya, Y., Sakai, K., & Miyashita, Y. (1996). ActivityJanssen, P., Vogels, R., & Orban, G. A. (2000). of primate inferotemporal neurons related to a Selectivity for 3D shape that reveals distinct sought target in pair-association task. areas within macaque inferior temporal cortex. Proceedings of the National Academy of Science, 288, 2054–2056. Sciences, 93, 2664–2669.Lashley, K. S. (1929). Brain mechanisms and Naya, Y., Yoshida, M., & Miyashita, Y. (2001). intelligence: A quantitative study of injuries to Backward spreading of memory-retrieval the brain. Chicago: Chicago University Press. signal in the primate temporal cortex. Science,Messinger, A., Squire, L. R., Zola, S. M., & Albright, 291, 661–664. T. D. (2001). Neuronal representations of Naya, Y., Yoshida, M., & Miyashita, Y. (2003a). stimulus associations develop in the temporal Forward processing of long-term associative lobe during learning. Proceedings of the National memory in monkey inferotemporal cortex. Academy of Sciences, 98, 12239–12244. Journal of Neuroscience, 23, 2861–2871.Messinger, A., Squire, L. R., Zola, S. M., & Albright, Naya, Y., Yoshida, M., Takeda, M., Fujimichi, R., & T. D. (2005). Neural correlates of knowledge: Miyashita, Y. (2003b). Delay-period activities in Stable representation of stimulus associations two subdivisions of monkey inferotemporal across variations in behavioral performance. cortex during pair association memory task. Neuron, 48, 359–371. European Journal of Neuroscience, 18, 2915–2918.Milner, B. (1962) Les troubles de la memoire ´ Pasupathy, A., & Miller, E. K. (2005). Different ´ accompagnant les lesions hippocampiques time courses of learning-related activity in the ´ bilaterales. In Physiologie de l’hippocampe (pp. prefrontal cortex and striatum. Nature, 433, 257–272) Paris: CNRS. 873–876.

358 PRIMATE NEUROETHOLOGYPenfield, W., & Perot, P. (1963). The brain’s record review of neuroscience (pp. 109–140). Palo Alto, of auditory and visual experience. A final CA: Annual Reviews Inc. summary and discussion. Brain, 86, 595–696. Tulving E., (1972) Episodic and semantic memory. In:Rupniak, N. M., & Gaffan, D. (1987). Monkey E. Tulving & W. Donaldson (Eds.), Organization hippocampus and learning about spatially of memory. New York: Academic Press. directed movements. Journal of Neuroscience, von Bonin, G., & Bailey, P. (1947). The neocortex of 7, 2331–2337. Macaca mulatta. Urbana, IL: University ofSakai, K., & Miyashita, Y. (1991). Neural Illinois Press. organization for the long-term memory of Wechsler, D. (1987) Wechesler memory Scale- paired associates. Nature, 354, 152–155. Revised. San Antonio, Tex.: PsychologicalSakai, K., Naya, Y., & Miyashita, Y. (1994). Corporation, Harcourt Brace Jovanovich. Neuronal tuning and associative mechanisms Williams, Z. M., & Eskandar, E. N. (2006). Selective in form representation. Learning & Memory, 1, enhancement of associative learning by 83-105. microstimulation of the anterior caudate.Saleem, K. S., & Tanaka, K. (1996). Divergent Nature Neuroscience, 9, 562–568. projections from the anterior inferotemporal Wirth, S., Yanike, M., Frank, L. M., Smith, A. C., area TE to the perirhinal and entorhinal Brown, E. N., & Suzuki, W. A. (2003). Single cortices in the macaque monkey. Journal of neurons in the monkey hippocampus and Neuroscience, 16, 4757–4775. learning of new associations. Science, 300,Scoville, W. B., & Milner, B. (1957). Loss of recent 1578–1581. memory after bilateral hippocampal lesions. Wise, S. P., & Murray, E. A. (1999). Role of the Journal of Neurology, Neurosurgery, and hippocampal system in conditional motor Psychiatry, 20, 11–21. learning: Mapping antecedents to action.Sheinberg, D. L., & Logothetis, N. K. (1997). The Hippocampus, 9, 101–117. role of temporal cortical areas in perceptual Yakovlev, V., Fusi, S., Berman, E., & Zohary, E. organization. Proceedings of the National (1998). Inter-trial neuronal activity in inferior Academy of Sciences, 94, 3408–3413. temporal cortex: A putative vehicle to generateSquire, L. R., Stark, C. E., & Clark, R. E. (2004). The long-term visual associations. Nature medial temporal lobe. Annual Review of Neuroscience, 1, 310–317. Neuroscience, 27, 279–306. Yanike, M., Wirth, S., Smith, A. C., Brown, E. N., &Squire, L. R., Wixted, J. T., & Clark, R. E. (2007). Suzuki, W. A. (2009). Comparison of Recognition memory and the medial associative learning-related signals in the temporal lobe. Nature Review Neuroscience, macaque perirhinal cortex and hippocampus. 8, 872–883. Cerebral Cortex, 19, 1064-1078.Squire, L. R., & Zola, S. M. (1996). Structure and Yoshida, M., Naya, Y., & Miyashita, Y. (2003) function of declarative and nondeclarative Anatomical organization of forward fiber memory systems. Proceedings of the projections from area TE to perirhinal neurons National Academy of Sciences, 93, 13515– representing visual long-term memory in 13522. monkeys. Proceedings of the National AcademySquire, & Zola-Morgan. (1991). The medial of Sciences, 100, 4257–4262. temporal lobe memory system. Science, 253, Zola-Morgan, S., & Squire, L. R. (1985). Medial 1380–1386. temporal lesions in monkeys impair memorySuzuki, W. A., & Amaral, D. G. (1994). Perirhinal on a variety of tasks sensitive to human and parahippocampal cortices of the macaque amnesia. Behavioral Neuroscience, 99, 22–34. monkey: Cortical afferents. Journal of Zola-Morgan, S., & Squire, L. R. (1990). The Comparative Neurology, 350, 497–533. neuropsychology of memory. Parallel findingsTanaka, K. (1996). Inferotermporal cortex and in humans and nonhuman primates. Annals of object vision. In: W. M. Cowan, E. M. Shooter, the New York Academy of Sciences, 608, C. F. Stevens, & R. F. Thompson (Eds.), Annual 434–450.

CHAPTER 19Neurobiology of Social BehaviorDario MaestripieriThe Primate Order comprises over 300 species Regardless of the variation in social systems,and a wide range of different social systems it is clear that nonhuman primates are generally(Smuts et al., 1987). Very few species have a highly social organisms, in which successful sur-fully solitary lifestyle. Most of the others are char- vival and reproduction depend on complexacterized by permanent associations between two social interactions with other conspecifics.or more adults and their young. Pair living is Accordingly, most primate species exhibit com-relatively rare, being characteristic of only 3% to plex behavioral adaptations for communication,4% of primate species. Some of these socially affiliation, aggression, mating, and parenting.monogamous species (e.g., tamarins and marmo- Group-living monkeys and apes often use voca-sets) have a flexible social organization, in which lizations to alert others of the presence of foodone additional adult male or female may be tem- and predators, and also to keep in contact withporarily associated with the breeding pair. Stable other group members during travel. Facialgroups with one adult male and several reproduc- expressions and body postures play an impor-tively active females (i.e., harems) are shown by tant role in close-range affiliative, agonistic, andapproximately 35% of primate species. The most sexual interactions, particularly among Oldcommon type of social organization in non- World monkeys and the great apes. Olfactoryhuman primates consists of social groups with and tactile signals are also used in these andmultiple adult males and females and their other contexts, although olfactory communica-young. These multimale/multifemale social tion is relatively underdeveloped in most pri-groups represent the stable form of organization mate species relative to other mammals. Socialof about 45% primate species, with an additional bonds between family and group members are15% of species showing fluctuations between established and maintained through contact,such groups and groups with a harem structure. proximity, and grooming. Grooming is anVariation in social organization among primate altruistic behavior that can be exchanged forspecies has been explained on the basis of varia- tolerance, sex, or coalitionary support duringtion in ecological variables such as diet, food- fights. Aggression and submission, oftenrelated competition and cooperation and need expressed with facial expressions and body pos-for protection from predators or infanticide (e.g. tures, result in the establishment of dominanceDunbar, 1988). Phylogenetic history, however, relationships and hierarchies. In many species,also accounts for variation in social organization, conflict outcomes and dominance ranks areas groups of closely related species and genera determined not by the individuals’ body sizetend to have similar social systems (Rendall & and strength but by coalitionary supportDi Fiore, 1995). Finally, within-species variation received from other individuals. Fights betweenin social organization may result from variation two individuals often extend to other groupin local ecological and demographic conditions. members, whose intervention may reflect 359

360 PRIMATE NEUROETHOLOGYattempts to protect a family member or political complex social life in large groups is asso-strategies involving complex cost/benefit ana- ciated with increased cognitive capacitylyses. Both affiliative and dominance relation- (Dunbar, 1992). Brain size, however, is aships depend on individuals’ memory of their crude measure of brain function, just aspast interactions and their outcome, as well as group size is a crude measure of social com-expectations about future interactions. High plexity. To understand the relation betweendominance rank may confer survival and repro- brain evolution and the evolution of socialityductive benefits such as greater access to food in primates, one needs to have a muchand safe sites or access to more and higher- deeper knowledge of how specific socialquality mating partners. Sexual monogamy is behaviors are produced or regulated by spe-rare in primates, or may be nonexistent since cific brain structures or neurochemical sys-extrapair copulations have been reported in tems. Unfortunately, our knowledge ofsocially monogamous primates. Polyandrous brain-behavior relationships in nonhumansystems, in which one female mates with mul- primates is very preliminary. Although intiple males, are also rare. Polygynous harem recent years there have been major advancessystems are more common among species in our understanding of the neural mechan-living in small groups, while those living in isms underlying social behavior in otherlarge multimale/multifemale groups typically mammalian species, particularly rodentshave promiscuous mating systems. Successful (e.g., Young, 2002), research on the neuro-mating, especially in promiscuous species, biology of social behavior in nonhuman pri-depends not only on features that advertise fer- mates has lagged far behind. With thetility, health, or strength but also on complex availability of new research techniques suchsocial strategies to increase one’s attractiveness as brain imaging, however, the investigationand deal with competition. Successful reproduc- of the neurobiological substrates of primatetion, at least for females, also depends on par- social behavior will be a promising area ofental investment in the offspring. Male care is research in the next few decades.rare among primates, while female care involves In this chapter, I review and discuss our cur-not only lactating but also carrying and pro- rent knowledge of the neurobiological regula-tecting offspring from predators and conspeci- tion of affiliative, aggressive, sexual, andfics. Maternal care in many primate species may parental behavior in nonhuman primates.extend well beyond the period of offspring nutri- Communication is clearly an important compo-tional dependence. For example, in species with nent of primate social behavior, but this topic isfemale philopatry and male dispersal such as addressed elsewhere in this volume (seemost cercopithecine monkeys, bonds between Chapters 5 and 25). Similarly, although the per-female relatives may last throughout the ception and processing of social stimuli is clearlylifespan. a prerequisite for social behavior, brain mechan- The behavioral adaptations for social life isms underlying social cognition are addressedthat characterize many primate species must in other chapters (see Chapter 26). This chapter,be supported by underlying neurobiological instead, focuses on social behavior expressed inmechanisms so that a relation is expected the context of interactions between two or morebetween complexity of social behavior and individuals.complexity of the brain. Consistent with this Different components of social behaviorexpectation, studies have shown that there is, such as affiliation, aggression, mating, and par-across primate species, a linear relation enting may or may not share some of the samebetween the average size of the social groups neural substrates, but they probably shareand the ratio between the size of the neo- common neurochemical controls. For example,cortex and the rest of the brain. Species that endogenous opioids and oxytocin have beenlive in larger social groups tend to have a implicated in the regulation of most, if not all,larger neocortex ratio, suggesting that social behaviors in rodents (see later).

NEUROBIOLOGY OF SOCIAL BEHAVIOR 361Similarly, the activity of neurotransmitter sys- influence behavior in conjunction with mono-tems involving the monoamines dopamine, aminergic neurotransmitter systems. Fornorepinephrine, and serotonin has been example, in a recent model of the neurobiolo-shown to affect the expression or inhibition of gical regulation of affiliation in mammals,a wide range of social activities in many mam- Depue and Morrone-Strupinksy (2005) havemals and other vertebrates (see later). The study argued that dopamine plays an important roleof neurochemical control of social behavior in in incentive-reward motivation processes (seeprimates has been mainly pursued with corre- Chapter 17) associated with the appetitivelational approaches, in which measures of pep- phase of affiliation, endogenous opioids pro-tides or monoamines or their metabolites in vide the neurochemical basis for the rewardblood or cerebrospinal fluid (CSF) are analyzed processes associated with the consummatoryin relation to social behavior, and to a lesser phase of affiliation, and oxytocin and vaso-extent, with pharmacological manipulations of pressin enhance the perception and memoryneurochemical systems. Attempts to identify of affiliative stimuli. In this section of thespecific areas of the brain involved in the reg- chapter, I review our knowledge of the neuro-ulation of social behavior have mostly been chemical control of primate social behavior,made with lesion studies. Other approaches first focusing on neuropeptides, and then oninvolving, for example, the electrical stimula- the monoamine systems.tion of specific areas of the brain or the imagingof brain activation with positron emission Endogenous Opioidstomography (PET) or functional magneticresonance imaging (fMRI) have been used less Almost 30 years ago, Panksepp and colleaguesfrequently in the context of social behavior proposed that brain endogenous opioids playstudies (but see Rilling et al., 2001, 2004a; a crucial role in regulating the establishment,Snowdon et al., 2006). Single neuron recording maintenance, and disruption of social bondshas often been used to address issues of social in mammals and birds (Panksepp et al.,perception (e.g., to study the processing and 1980). This hypothesis was developed fromrecognition of faces and facial expressions) the observation that the emotional statesbut rarely for social behavior. In this chapter, accompanying the formation of social attach-first I review research on the neurochemical ments, the weaning of social bonds, and thecontrol of primate social behavior, particularly distress arising from social separation appearstudies of endogenous opioids, oxytocin and to share similarities with the characteristics ofvasopressin, and the brain monoamine systems. opiate addiction—that is, the development ofThen, I review the results of brain lesion studies dependence, tolerance, and withdrawalinvestigating the neural substrates of primate (Panksepp et al., 1980, 1999). The hypothe-social behavior. I conclude the chapter by sum- sized relationship between endogenousmarizing the main trends emerging from this opioids and social attachments was framedliterature review and discussing future research within the general theory that the emotionaldirections. substrates of attachments are an evolutionary outgrowth of more primitive brainstem and limbic circuits in the mammalian brain thatNEUROCHEMICAL CONTROL OF originally subserved basic physiological needsSOCIAL BEHAVIOR such as energy balance, thermoregulation, orResearch conducted with other mammalian pain perception.species, mostly rodents, has suggested that According to Panksepp and collaborators, aneuropeptides such as endogenous opioids, release of endogenous opioids following theoxytocin, and vasopressin are good neuro- exchange, and especially the receipt, of affiliativechemical candidates for regulating complex behavior generates the feeling of pleasure andsocial behaviors. These neuropeptides can gratification that arise from the interaction,

362 PRIMATE NEUROETHOLOGYwhereas a reduction in endogenous opioids exchange grooming behavior, was accompaniedresults in emotional distress and promotes the by a significant increase in CSF concentrationsneed to seek and maintain proximity with a of b-endorphin. There were no significant cor-social partner. Although some studies of birds relations, however, between the amount ofand mammals, including nonhuman primates grooming given or received and the concentra-(e.g., Barr et al., 2008; Kalin et al., 1988; tions of b-endorphin or their increases followingKraemer, 1992), have provided evidence for the pair housing. Nevertheless, Keverne and collea-involvement of the opioid system in distress gues (1989) suggested that the opioid releaseseparation responses, there is growing consensus may have been caused by the tactile stimulationthat the neurobiological systems regulating associated with grooming in newly formed pairs,separation distress responses are different from and that this effect may be similar to the stimu-those mediating social rewards (e.g., Insel, 1992; lation of endogenous opioid release by acupunc-Panskepp et al., 1999). Therefore, more recent ture in humans. That endogenous opioids aretheories about opioids and social behavior view sensitive to social variables had also been sug-endogenous opioids as playing a crucial role in gested by a previous study, in which CSFthe consummatory phase of affiliative interac- concentrations of b-endorphin were found totions, and therefore in the strengthening of be highest in male talapoin monkeys of lowsocial bonds, but less so in the motivation to rank and lowest in those of high rankestablish these bonds or in the response to their (Martensz et al., 1986).disruption or termination (Depue & Morrone- A relatively large number of primate studiesStrupinksy, 2005). has attempted to test the following two predic- The most systematic attempt to investigate tions of Panksepp’s hypothesis: (1) the admin-the role of endogenous opioids in the regulation istration of an exogenous opioid such asof primate social behavior has been made by morphine should create a feeling of social com-Keverne and collaborators with a series of stu- fort and reduce the motivation to seek socialdies in talapoin monkeys and rhesus macaques. contact or decrease the expression of affiliativeThey argued that of the different endogenous behavior, and (2) the blockade of endogenousopioids in the brain, b-endorphin may be the opioid receptors should increase the need forbest candidate for the regulation of social beha- social attachment and therefore the solicitationvior, and given the difficulties of measuring this of affiliative behavior from social partners.peptide directly in the brain of live primates, Research by Keverne and colleagues showedthey measured its concentration in the CSF that acute treatment of adult talapoin monkeys(Martensz et al., 1986). b-Endorphin does not with the opioid receptor blockers naloxone orgain access to CSF from the blood when CSF and naltrexone increased their grooming solicita-the cerebral extracellular fluid are in equili- tions and resulted in more grooming receivedbrium; therefore, CSF levels of b-endorphin pro- from other individuals (Fabre-Nys et al., 1982;vide a measure of its presence in the extracellular Keverne et al., 1989; Meller et al., 1980). Thesefluid of the brain, a reasonable marker of the effects were observed in both pair-housed andlevel of activity in intracerebral b-endorphin– in group-living individuals, were dose depen-containing systems (Martensz et al., 1986). dent and stronger for females than for males, Consistent with Panksepp’s hypothesized and were specific for allogrooming behavior:relationship between endogenous opioids and Self-grooming, aggressive behavior, and loco-affiliation, Keverne and colleagues (1989) pro- motor activity were not affected (but in onevided evidence for an association between study male sexual behavior was decreased;grooming behavior and opioid release in tala- Meller et al., 1980). Naltrexone administrationpoin monkeys. Specifically, they reported that was also associated with an increase in testos-moving adult talapoin monkeys from isolation terone, cortisol, and prolactin, suggesting thathousing to pair housing, and therefore pro- some of the effects of opiate receptor blockadeviding the monkeys with an opportunity to on social behavior may have been hormonally

NEUROBIOLOGY OF SOCIAL BEHAVIOR 363mediated (Meller et al., 1980). An increase in attachment share a common neurochemicalthe number of grooming solicitations and in the substrate (Nelson & Panksepp, 1998).amount of grooming received from other The role of the opioid system in mediatingindividuals following naloxone or naltrexone maternal attachment has been investigated intreatment was also reported by two studies of three primate studies producing conflictinggroup-living rhesus macaque adult females results. In one study, the opioid system was(Graves et al., 2002; Martel et al., 1995). pharmacologically manipulated after mother-Keverne and colleagues (1989) found that infant separation and reunion in rhesus maca-acute treatment of pair-housed monkeys with ques (Kalin et al., 1995). Morphine decreasednonsedative doses of morphine resulted in a clinging with the infant during the first 30 min-significant decrease in the number of grooming utes of reunion, whereas naltrexone increasedsolicitations as well as a decrease in grooming clinging. In a study of socially living rhesusperformed. A study of common marmosets, mothers and infants, however, naloxone reducedhowever, reported that morphine administra- both maternal grooming and maternaltion increased the frequency and duration of restraining of the infant, suggesting decreasedsocial play but had no effects on social contact rather than increased attachment to the infantor grooming (Guard et al., 2002). Since (Martel et al., 1993). In this study, however, thegrooming behavior in Old World monkeys effects of naloxone on affiliative interactionsmay have different functions and be regulated between mothers and other adults were alsoby different mechanisms than grooming beha- contrary to the expectations. In fact, the mothersvior in New World primates, the findings of the treated with naloxone showed reduced numberstudies reviewed previously are generally of grooming solicitations and reduced amountconsistent with the hypothesis that endogenous of grooming received from other individuals.opioids may mediate the rewarding properties Finally, in another study of rhesus macaques,of affiliative interactions between adults. naltrexone had no significant effects on any Studies manipulating the opioid system of aspects of maternal behavior, including abusiveimmature monkeys have produced results con- parenting (Graves et al., 2002). Although somesistent with the hypothesized relation between of these inconsistencies may be due to metho-opioids and attachment (Kalin et al., 1988, 1995; dological differences between studies, furtherMartel et al., 1995; Schino & Troisi, 1992). In a research is needed before any firm conclusionsstudy of group-living long-tailed macaques, can be drawn regarding the relationship betweenjuveniles receiving an acute administration of opioids and maternal attachment.naloxone increased their proximity-seeking Researchers investigating endogenous opioidbehavior toward their mothers, displayed more release following affiliative interactions madegrooming solicitations to both their mothers the assumption that the CSF concentrationsand other group members, and received more of b-endorphin mainly reflect the productiongrooming from them (Schino & Troisi, 1992). of this peptide by neurons that originate fromGrooming done by the juveniles was not affected the arcuate nucleus of the hypothalamusby naloxone, while self-grooming decreased. An (e.g., Keverne et al., 1989; Martensz et al.,increase in contact seeking with the mother was 1986). These neurons project to brain regionsalso observed in infant and juvenile rhesus that are rich in opiate receptors such as themacaques treated with naloxone (Martel et al., brainstem, basal ganglia, and areas of the1995). Miczek and colleagues (1981) reported hypothalamus, amygdala, cerebellum, andthat the acute administration of nonsedative raphe nuclei. Of the different families of opiatedoses of morphine in squirrel monkeys receptors, many of which have multiple sub-decreased the rate of affiliative behavior shown types, the m-opiate receptor family seems to beby juveniles toward their mothers. Taken the most directly implicated in the regulation oftogether, these results are consistent with the social behavior, and b-endorphin has highhypothesis that infant attachment and adult affinity for these receptors. In brain areas rich

364 PRIMATE NEUROETHOLOGYin m-receptors, b-endorphin neurons interact the formation and maintenance of social bondswith dopaminergic and serotonergic neurons, between adults (e.g., between mating partners)as well as with neurons using oxytocin and vaso- and between parents and offspring (e.g., Young,pressin (at least in rodents). It has been sug- 2002). In rodents, these peptides have also beengested that interactions between m-opiates and implicated in the regulation of sexual anddopamine neurons in the ventral tegmental area aggressive behavior as well as in stress responses.(VTA) of the hypothalamus produce the experi- Little is known, however, about the relationence of reward associated with the appetitive and between oxytocin or vasopressin and socialconsummatory phases of affiliative interactions, behavior in nonhuman primates. An earlywhile serotonergic input to the hypothalamus study by Winslow and Insel (1991) examinedvia the raphe nuclei may result in reduced the effects of intracerebroventricular adminis-arousal and facilitation of opioid-mediated feel- tration of two doses of oxytocin and vasopressin,ings of gratification following affiliation (Depue as well as of an oxytocin receptor antagonist& Morrone-Strupinksy, 2005). Finally, studies of (OTA), on aggressive, sexual, and affiliativerats have suggested that oxytocin and vaso- behavior of pair-housed male squirrel monkeyspressin may facilitate the rewarding effects of during an interaction with a familiar adultendogenous opiates, as oxytocin neurons in the female. Oxytocin increased aggressive andparaventricular nucleus of the hypothalamus sexual behavior in a dose-dependent manner inproject to the b-endorphin neurons in the arc- dominants but not in subordinates, while ituate nucleus and increase their release of opioids increased the frequency of approaches and hud-(Csiffary et al., 1992). Oxytocin, vasopressin, dles mostly in subordinates. These effects wereand the monoamines, however, can affect affilia- blocked by OTA. Vasopressin decreased aggres-tion and other forms of social behavior also sive and affiliative behaviors in both dominantsthrough mechanisms that are not dependent and subordinates. Differences in the effects ofon endogenous opiates. oxytocin in dominants and subordinates were tentatively explained in terms of different oxytocin receptor density associated with differ-Oxytocin and Vasopressin ences in testosterone between dominants andOxytocin and vasopressin are 9-amino acid pep- subordinates.tides synthesized in the supraoptic and paraven- Research on oxytocin and social behavior intricular nuclei of the hypothalamus and released nonhuman primates has been motivated by theinto systemic circulation from neurons in the interest in developing a primate model forposterior pituitary gland. Hypothalamic neu- autism. When rhesus monkey infants are sepa-rons synthesizing oxytocin and vasopressin rated from their mothers at birth and peer-also project to various areas of the brain, and reared in a small cage, they develop a widereceptors for these peptides have been found in range of behavioral abnormalities, whichthe limbic system of some mammalian species, according to some researchers share some simi-particularly rodents (Gimpl & Fahrenholz, larities with autism. Winslow and colleagues2001). Until recently, oxytocin receptors could (2003) measured CSF and plasma oxytocinnot be easily identified in the primate brain levels in mother-reared and peer-reared infants(Toloczko et al., 1997; Winslow, 2005), but the to assess whether the behavioral characteristicspresence of such receptors has recently been of peer-reared infants (e.g., low affiliation, highinferred in the hypothalamus, amygdala, aggression, and high self-directed repetitiveseptum, orbitofrontal cortex, and hippocampus behavior) were associated with alterations in(Boccia et al., 2001, 2007). oxytocin. In this study, individual differences A large body of research conducted mostly in CSF oxytocin levels at 18, 24, and 36 monthswith rodents and sheep, but more recently also of age did not correlate with differences inwith humans, has suggested that central oxy- plasma oxytocin levels. The peer-reared infantstocin and vasopressin play an important role in had lower levels of CSF oxytocin than the

NEUROBIOLOGY OF SOCIAL BEHAVIOR 365mother-reared ones, but there were no differ- beginning to be investigated. In a pilot experi-ences in CSF vasopressin or plasma oxytocin. ment conducted with two nulliparous rhesusCSF oxytocin levels were correlated with time macaque females, Holman and Goy (1995) exam-spent engaged in affiliative social behaviors ined whether an intracerebroventricular injectionsuch as allogrooming and male-male mounting of oxytocin affected responsiveness to infants.independent of rearing condition, while vaso- The two females were exposed to an unfamiliarpressin levels were negatively correlated with infant in a cage 10 minutes after the injection ofthe frequency of fear grimaces at 18 months of oxytocin or saline. The females sat near the infant,age. Taken together, the results of this study and watched, touched, and lip-smacked to theprovide evidence for an association between infant more frequently following oxytocin com-CSF oxytocin and affiliative behavior and also pared to saline administration. In no case, how-suggest that both can be affected by a traumatic ever, was more intense caregiving behaviorearly experience such as maternal deprivation. observed, perhaps because of the environmentSimilar to the Winslow et al. study (2003), in which the animals were tested. In anotherSchwandt and colleagues (2007) found no sig- study, CSF levels of oxytocin measured in 10nificant correlation between CSF and plasma multiparous rhesus females before parturition,concentrations of oxytocin and vasopressin in immediately after parturition, and 7 days post-free-ranging female rhesus macaques. Oxytocin partum were not correlated with mother-infantwas not correlated with any social behavior, behaviors such as contact or grooming (Cooke etalthough females with low levels of oxytocin al., 1997). Finally, Boccia and colleagues (2007)were classified as more fearful by human obser- reported that the administration of a humanvers. Vasopressin was correlated only with uterine oxytocin receptor blocker reduced theleaping behavior, with females with high vaso- frequency of lip-smacking, approaching, andpressin exhibiting higher frequencies of this touching a stimulus infant in one 4-year-oldbehavior. nulliparous rhesus macaque female (in a separate A possible relationship between CSF oxy- experiment, the same treatment also reducedtocin and affiliation was also inferred from a female sexual behavior, and in both experimentscomparison of closely related primate species. locomotor activity was also significantlyRosenblum and colleagues (2002) reported that reduced). This oxytocin receptor blocker hadlaboratory-born pigtail macaques had lower previously been shown to cross the blood-brainCSF concentrations of oxytocin than bonnet barrier and to accumulate in the hypothalamus,macaques. The authors of this study described orbitofrontal cortex, amygdala, hippocampus,bonnet macaques as very gregarious, affiliative, and septum, suggesting that these are brainand affectively stable, while pigtail macaques areas rich in oxytocin receptors (Boccia et al.,were described as temperamentally volatile and 2007). Although the effects of oxytocin receptorsocially distant. Therefore, the results were blockade on infant-directed behavior in one sub-interpreted as being supportive of the hypoth- ject were suggestive of a relation between oxy-esis that baseline CSF oxytocin concentrations tocin and parental responsiveness, the authorsare related to species-typical social/affective of this study acknowledged that other explana-behavior patterns. Bonnet macaques, however, tions for their results were also possible.have been described by other researchers as In a study investigating the possible neuro-highly competitive and aggressive, while pigtail biological and neurochemical substrates ofmacaques have been described as peaceful, gre- paternal responsiveness in marmoset monkeys,garious, and affiliative (Thierry et al., 2004). first-time and experienced fathers who had spentTherefore, the significance of the difference in a considerable amount of time carrying infantsoxytocin levels between these two species had a greater number of vasopressin V1a recep-remains unclear. tors in the prefrontal cortex than adult male The role of oxytocin in the regulation of nonfathers living in similar social conditionsparental responsiveness in primates is only (Kozorovitskiy et al., 2006). There were no

366 PRIMATE NEUROETHOLOGYdifferences in the abundance of vasopressin V1b Dopaminereceptors or oxytocin receptors in the prefrontal Dopaminergic neurons and their projection sitescortex, nor in the abundance of V1a receptors in (e.g., the ventral striatum, nucleus accumbens,the occipital cortex. Interestingly, fatherhood amygdala, anterior cingulate cortex, and orbito-was also associated with an increased proportion frontal cortex) constitute what is known as theof dendritic spines in the prefrontal cortex, brain reward system (see Chapter 17). Thiswhich were immunoreactive for V1a receptor, system regulates a wide range of incentive-moti-as well as increased overall density of dendritic vated behaviors, and these may also includespines on pyramidal neurons in the prefrontal social activities such as affiliation, aggression,cortex. The functional implications of this mating, and parenting. With regard to affilia-fatherhood-associated structural reorganization tion, it has been argued that dopamine plays ain the prefrontal cortex and the increased abun-dance of vasopressin V1a receptors remain crucial role in incentive-reward motivation pro-unclear. Interestingly, Hammock and Young cesses associated with the appetitive phase of(2005) have suggested that a repetitive poly- affiliation (Depue & Morrone-Strupinksy,morphic microsatellite in a regulatory region of 2005). The appetitive phase involves, at thethe vasopressin 1a receptor gene (AVPR1a) may behavioral level, a search and approach systembe responsible for both intraspecific and inter- whose function is to bring an individual in con-specific variation in social behavior in primates, tact with affiliative stimuli. Research withas in rodents. They found that this poly- rodents has shown that the incentive motivationmorphism is present in humans and in bonobos and experience of reward that underlie thebut absent in chimpanzees, and hypothesized search for and approach to affiliative stimulithat it may be responsible for some of the differ- depend on the functional properties of dopami-ences in affiliation and bonding between the nergic neurons in the VTA and nucleus accum-latter two species. bens (NAS) (Depue & Morrone-Strupinksy, Oxytocin and vasopressin have been hypo- 2005). Given the lack of relevant data, whetherthesized to promote social bonds by facilitating the same relation between dopamine and thethe perception, processing, and memorization appetitive phase of affiliation holds true also inof affiliative stimuli (Depue & Morrone- nonhuman primates remains unclear. There isStrupinksy, 2005). It has also been suggested some evidence, however, that dopamine plays athat oxytocin can reduce tension and anxiety greater role in the appetitive aspects of primateassociated with social interactions. For example, sexual behavior than in its consummatory com-human studies have suggested that oxytocin ponents. For example, a dopamine agonist, apo-released during affiliative social interactions morphine, which acts on dopamine D1 and D2reduces the hypothalamus-pituitary-adrenal receptors, enhances male sexual arousal in(HPA) axis response to stressful events (Uvnas- response to female sexual stimuli in rhesus mon-Moberg, 1998). In an attempt to test this keys (Pomerantz, 1990) but does not appear tohypothesis with primate data, Parker and collea- affect male copulatory behavior (Chambers &gues (2005) showed that chronic intranasal Phoenix, 1989). Quinelorane, another D2 ago-administration of oxytocin prior to acute social nist, also stimulates male sexual arousalisolation attenuates the adrenocorticotropic (Pomerantz, 1991). Whether these effects alsohormone (ACTH) response (but not the cortisol occur in females is unclear, since there haveresponse) to stress in squirrel monkeys (see also been no studies investigating dopaminergicHeinrichs et al., 2003, in humans). Since cortisol function and sexual behavior in female primateswas not affected and because intranasal oxytocin (Dixson, 1998).can penetrate the central nervous system (CNS), In nonhuman primates, brain dopaminergicthis suggests that oxytocin exerts its antistress function has also been investigated in relation toeffects prior to adrenal activation, either in the personality traits such novelty seeking, which arebrain or at the pituitary level. expected to influence behaviors such as

NEUROBIOLOGY OF SOCIAL BEHAVIOR 367exploration, assertiveness, aggressiveness, and macaques, adult males and females that becamedominance. In humans, genetic polymorphisms dominant within their groups had significantlyin the coding region of the dopamine D4 higher CSF HVA concentrations than those thatreceptor gene (DRD4) have been linked with a became subordinate (Kaplan et al., 2002).number of personality and behavioral disorders Greater dopaminergic activity in dominantboth in adults and in children (e.g., Sheese et females was also suggested by another study inal., 2007). A study of captive vervet monkeys the same species, in which prolactin responses toshowed that the DRD4 genetic polymorphism a challenge with the dopamine antagonist halo-accounts for a significant fraction of interindivi- peridol were greater in dominants than in sub-dual variation in novelty-seeking behavior (e.g., ordinates (Shively, 1998). In another study,latency to approach novel objects) (Bailey et al., however, dominant females showed greater2007). Correlations between CSF concentrations striatal dopamine D2 receptor binding thanof the dopamine metabolite homovanillic acid subordinates, suggesting lower dopaminergic(HVA) and measures of sexual, assertive, and activity in dominants (Grant et al., 1998).aggressive behavior have been reported by Given these discrepancies between the results ofsome studies of macaques and vervet monkeys different studies, and the fact that other studies(e.g., Fairbanks et al., 2004; Kaplan et al., 2002; have failed to report significant correlationsMehlman et al., 1994, 1997), but these correla- between CSF HVA levels and any measures oftions must be interpreted with caution. This is social behavior (e.g., Cleveland et al., 2004;because individual differences in CSF con- Maestripieri et al., 2006b), the relation betweencentrations of HVA are highly positively corre- dopaminergic activity and aggression and dom-lated with those of the serotonin metabolite inance in primates remains unclear.5-hydroxyindoleacetic acid (5-HIAA) and thenorepinephrine metabolite 3-hydroxy-4-meth- Norepinephrineoxyphenylglycol (MHPG) (see later). The CSFconcentrations of the three monoamine meta- The brain noradrenergic system has been impli-bolites also share similarities in their heritability cated in the regulation of arousal and an indivi-(Freimer et al., 2007; Higley et al., 1993; Rogers dual’s aggressive or fearful responses to novel oret al., 2004), the extent to which they are affected threatening stimuli. Aggressive and fearful beha-by early stressful experience (Higley et al., 1991; viors are associated with increased central andMaestripieri et al., 2006a,b), and age-related peripheral noradrenergic activity in primates aschanges across the primate lifespan (Higley et in other mammals, but this association is notal., 1992a). In some cases, correlations between specific to agonistic interactions. Rather, ele-CSF HVA and social behavior may be a by- vated norepinephrine is observed in all situa-product of correlations between CSF 5-HIAA tions with high arousal and is an importantor MHPG levels and behavior. For example, component of an individual’s response tothe association between low CSF HVA and stress. Central norepinephrine may also mediatehigh dominance status in male vervet monkeys sexual arousal, but there are no relevant primatewas almost entirely due to the correlation data on this topic (Dixson, 1998). Some primatebetween low 5-HIAA and high dominance and studies have reported correlations between CSFto the correlation between HVA and 5-HIAA concentrations of norepinephrine or the norepi-(Fairbanks et al., 2004). Kaplan and colleagues nephrine metabolite MHPG and aggressive(2002), however, reported a strong association behavior, but these correlations have beenbetween CSF levels of HVA and dominance, mixed. For example, Higley and colleagueswhich was independent of the other CSF mono- (1992b) reported that highly aggressive rhesusamine metabolite concentrations. This associa- monkey males had higher CSF levels of norepi-tion was in the opposite direction to that nephrine than less aggressive monkeys, whereasreported by Fairbanks and colleagues (2004). among females low CSF norepinephrine wasIn unisexual groups of captive long-tailed associated with high rates of severe aggression

368 PRIMATE NEUROETHOLOGY(Higley et al., 1996a). No correlation between SerotoninCSF MHPG and aggression was found in several Serotonin is one of the most ancient neurotrans-other studies of free-ranging rhesus monkeys mitters in mammals and has been implicated in(Higley et al., 1992b, 1996a; Howell et al., the regulation of social behavior in a number of2007). In long-tailed macaques, CSF MHPG other taxa as well (Insel & Winslow, 1998). Inwas higher in dominants than in subordinates humans and other primates, brain neurons thatamong males, but not among females (Kaplan et use serotonin as their primary neurotransmitteral., 2002), whereas in male vervet monkeys there originate in the raphe nuclei of the brainstemwas no significant correlation between CSF and project to the cerebral cortex as well as toMHPG levels and dominance rank (Fairbanks subcortical structures such as the amygdala,et al., 2004). Elevated CSF MHPG levels have septum, hypothalamus, hippocampus, tha-been reported in rhesus monkey infants rejected lamus, and basal ganglia. Studies of serotoninand abused by their mothers (Maestripieri et al., and social behavior in primates have used2006a) or in peer-reared infants (Higley et al., indirect measures of CNS serotonergic function1991). As juveniles, individuals with low CSFMHPG exhibit high avoidance of other indivi- such as the measurement of 5-HIAA in the CSFduals (Maestripieri, 2006b), while among adult or pharmacological and neuroendocrine chal-females, individuals with high CSF MHPG are lenges. The use of CSF 5-HIAA concentrationavoided more by other individuals. In wild as an indicator of brain serotonin activity hasvervet monkeys, however, low CSF MHPG been validated by various methods includinglevels were associated with higher impulsivity postmortem brain studies showing that the(Fairbanks et al., 1999). CSF content of 5-HIAA reflects the content of Given the discrepancies in these research find- this metabolite in the brain (e.g., Banki &ings, it is premature to draw any conclusions Molnar, 1981; Wester et al., 1990). Low concen-about the relation between norepinephrine and trations of CSF 5-HIAA are generally interpretedsocial behavior in nonhuman primates. Whether as representative of lower CNS serotonergicnorepinephrine-dependent arousal results in function. The relationship between CSF 5-HIAAaggressiveness or avoidance might depend on and specific serotonergic neural pathways in thethe complex relationship between arousal and brain, however, remains unclear (Insel &anxiety, fear, and impulsivity. Since different Winslow, 1998). Administration of the serotonintypes of aggression may have different emotional agonist fenfluramine, which stimulates the releasesubstrates, the relation between norepinephrine of serotonin from neurons and inhibits its reup-and aggression could be different for different take, has also been used as an indirect method oftypes of aggressive behavior. Furthermore, similar assessing CNS serotonergic function. Sinceto HVA, the relation between CSF MHPG and stimulation of serotonin receptors in theaggression may be confounded by the relation hypothalamus results in increased release of pro-between 5-HIAA and aggression and the positive lactin from the pituitary, plasma concentrationscorrelation between 5-HIAA and MHPG. of prolactin following fenfluramine administra-Although noradrenergic mechanisms can poten- tion can be used as indicators of responsivity oftially affect aggression and dominance indepen- the brain serotonergic system. Finally, the seroto-dently of serotonin, the relation between nergic system can be challenged by manipulatingaggression and serotonin seems to be stronger the availability of tryptophan, the amino acidand more specific (see later) than that between necessary for the synthesis of this monoamine,aggression and dopamine or norepinephrine. For or by using other pharmacological serotoninexample, studies of rodents have shown that the reuptake inhibitors.basal activity of the noradrenergic system, unlike Using the previously described techniques,that of the serotonergic system, does not consis- a number of human studies have shown thattently differentiate between more and less aggres- low CNS serotonin function is related tosive individuals (Miczek & Fish, 2006). impaired impulse control and to unrestrained

NEUROBIOLOGY OF SOCIAL BEHAVIOR 369aggressiveness and violence, particularly in adult Although it is possible that there is a directmales (see Manuck et al., 2006, for a recent causal relation between brain serotonergicreview). Similarly, studies of rhesus macaque function and aggressive behavior, it is moreadult males and females have shown that low likely that such a relationship is indirect andlevels of CSF 5-HIAA are associated with high mediated by impulse control. In this view,impulsivity, risk-taking behavior, and propen- reduced or dysregulated serotonergic functionsity to engage in severe forms of aggression would impair an individual’s ability to restrain(Higley et al., 1996a,b,c; Howell et al., 2007; impulses, and this would be manifested in risky,Mehlman et al., 1994; Westergaard et al., 1999, aggressive, depressed, or addictive behavior2003; see Higley, 2003, and Manuck et al., 2006, depending on the environmental circumstancesfor reviews). In vervet monkeys, individuals fed and the individual’s motivational state. Thediets high in tryptophan exhibit lower aggres- hypothesis that impulse control is an importantsion, whereas individuals placed on diets low in intervening variable is supported by evidencetryptophan become more aggressive, with the that, in rhesus macaques, low CSF 5-HIAA isincrease in aggressiveness being greater for correlated only with aggression involving phy-males than for females (Chamberlain et al., sical contact and chases, a type of aggression1987; Raleigh et al., 1985, 1986, 1991). that is more likely to result in serious injuries,Decreased aggression has also been observed and not with milder agonistic behavior invol-following short-term administration of sero- ving threats and avoidance, which is commonlytonin reuptake inhibitors (Chamberlain et al., associated with the maintenance of dominance1987; Raleigh et al., 1980, 1985, 1986, 1991), relationships (Botchin et al., 1993; Higley et al.,while administration of the serotonin synthesis 1992b, 1996a,b; Mehlman et al., 1994). Indeed,inhibitor p-chlorophenylalanine (PCPA) results the relationship between CSF 5-HIAA levelsin increased aggression (Kyes et al., 1995; and dominance is not clear, as some studiesRaleigh & McGuire, 1986; Raleigh et al., 1980, have reported that CSF 5-HIAA is higher in1983, 1986). In long-tail macaques, individuals dominants and lower in subordinates (e.g.,showing low responsivity to the fenfluramine Fairbanks et al., 2004; Westergaard et al.,challenge were more aggressive toward other 1999) while other studies have reported theindividuals and to faces with threatening expres- opposite pattern or no relation at all (Kaplansions than individuals exhibiting prolactin et al., 2002; Raleigh et al., 1991; Shively, 1998;responses of greater magnitude (Botchin et al., Shively et al., 1995; Yodyingyuad et al., 1985).1993; Kyes et al., 1995). Monkeys with low CSF 5-HIAA concentrations A relation between aggressiveness and CSF are more likely to exhibit behaviors suggestive5-HIAA concentrations has also been found in of impaired impulse control such as long leapscomparisons between different genetic strains of at high heights and repeated jumping intorhesus macaques and between closely related baited traps in which they are capturedspecies. For example, the more aggressive (Fairbanks et al., 1999; Higley et al., 1996c;Chinese-derived rhesus strain has lower CSF Mehlman et al., 1994). In the laboratory,5-HIAA concentrations than the less aggressive rhesus macaques with low CSF 5-HIAA con-Indian-derived strain (Champoux et al., 1997). centrations have a lower latency to approach aFurthermore, rhesus macaques have lower CSF novel object than do monkeys with high CSF 5-5-HIAA levels than pigtail macaques, a species HIAA concentrations (Bennett et al., 2002). Inbelieved to be generally less aggressive than vervet monkeys, individuals with low CSF 5-rhesus macaques (Westergaard et al., 1999). HIAA concentrations approached a strange andFinally, Kaplan and colleagues (1999) reported potentially dangerous adult male more quicklythat anubis baboons, who are characterized by and were more likely to act aggressively towardrelatively high levels of intermale aggression, him than monkeys with high CSF 5-HIAAhave lower CSF 5-HIAA concentrations than the (Fairbanks et al., 2001; see also Manuck et al.,less aggressive anubis-hamadryas baboon hybrids. 2003). Individuals treated with the selective

370 PRIMATE NEUROETHOLOGYserotonin reuptake inhibitor fluoxetine became & McGuire 1990; Raleigh et al., 1980, 1985).less impulsive in this strange male test than These data from vervet monkeys thus suggestcontrol individuals (Fairbanks et al., 2001). that enhancing serotonergic function facilitates Increased tendencies to exhibit risky beha- the expression of affiliative behavior, whereasviors and to engage in severe forms of aggression reducing serotonergic function inhibits affiliation.are not the only behavioral manifestations of low There is some evidence that serotonergic func-CSF 5-HIAA in monkeys. Individuals with low tion is related not only to aggressive and affiliativeCSF 5-HIAA also show reduced propensities for behavior but also to sexual and maternal beha-prosocial behaviors and affiliation. In a study of vior. Consistent with the results of studies offree-ranging adolescent rhesus macaque males, rodents, serotonin has been shown to exert inhi-individuals with low CSF 5-HIAA concentra- bitory effects on male and female sexual behaviortions exhibited reduced amounts of time spent in primates as well (e.g., Gradwell et al., 1975;in proximity and grooming other group mem- Pomerantz et al., 1991). Early studies of serotoninbers, and a lower number of social partners with and maternal behavior in primates reported thatwhom they interacted (Mehlman et al., 1995; see monkey mothers with low CSF 5-HIAA werealso Kaplan et al., 1995). Young rhesus males more protective and restrictive, and that theirwith low CSF 5-HIAA concentrations have also infants spent more time in contact with them,been reported to emigrate from their natal group than mothers with high CSF 5-HIAA (Fairbanksat an earlier age than males with higher CSF et al., 1998; Lindell et al., 1997). Cleveland and5-HIAA concentrations (Howell et al., 2007; colleagues (2004) found no relationship betweenKaplan et al., 1995; Mehlman et al., 1995, CSF 5-HIAA and maternal behavior in the first1997). In the laboratory, low CSF 5-HIAA con- few postpartum days, but on postpartum days 15centrations were associated with low rates of and 20, females with low CSF 5-HIAA brokeaffiliative interactions among rhesus juveniles contact and left their infants less frequently thanof both sexes (Higley et al., 1996a). Adult females females with high CSF 5-HIAA. A preliminarywith low CSF 5-HIAA also appear to be less study in our laboratory reported a positive corre-socially oriented, spending more time alone, lation between CSF 5-HIAA concentrations mea-grooming less, and having fewer conspecifics in sured during pregnancy and maternal rejectionclose proximity (Cleveland et al., 2004). Adult behaviors in the first postpartum month in multi-rhesus males with low CSF 5-HIAA form fewer parous females (Maestripieri et al., 2005). Ourconsorts with estrous females during the mating more recent work involving multiple measure-season, and during these consorts, they groom ments of CSF 5-HIAA during development, how-and mount the females less frequently than ever, reported a negative correlation between CSFmales with higher CSF 5-HIAA (Mehlman et 5-HIAA and maternal rejection among first-timeal., 1997). In long-tail macaques, individuals mothers (Maestripieri et al., 2007).with low responses to fenfluramine spent less Serotonin may affect maternal motivationtime in affiliative interactions with other indivi- through its actions on oxytocin or prolactinduals and more time alone (Botchin et al., 1993). release, or through its effects on emotionalIn vervet monkeys, Raleigh and colleagues found expression (Insel & Winslow, 1998; Numan &that enhancing serotonin function by admi- Insel, 2003). Emotions can be powerful elicitorsnistering tryptophan, the reuptake inhibitor of maternal behavior in nonhuman primatesfluoxetine, or the serotonin agonist quipazine and humans (Dix, 1991; Maestripieri, 1999;increased affiliative behaviors such as Pryce, 1992). For example, there are markedapproaching and grooming other monkeys individual differences in anxiety among rhesus(Raleigh et al., 1980, 1983, 1985). In contrast, mothers, and such differences translate into dif-reducing serotonin function by administering ferences in maternal style (Maestripieri,the tryptophan hydroxylase enzyme inhibitor 1993a,b). Maternal anxiety has also been impli-PCPA resulted in social withdrawal and in cated in the etiology of infant abuse in macaquesavoidance of affiliative interactions (Raleigh (Maestripieri, 1994; Troisi & D’Amato, 1991).

NEUROBIOLOGY OF SOCIAL BEHAVIOR 371Although the role of impulse control in primate prefrontal cortex, pACC, and amygdala, sug-maternal behavior is still poorly understood, it is gesting that the presence of short allele is asso-possible that impulsivity affects how primate ciated with hyperreactivity of the amygdala inmothers interact with their infants, and that response to threats (Skuse, 2006).high impulsivity is expressed as high rejection In rhesus macaques, the SERT polymorphismrates, thus explaining the association between is generally unrelated to CSF concentrations oflow CSF 5-HIAA and high rejection rates 5-HIAA, with the exception of individualsfound in first-time rhesus macaque mothers who are separated from their mothers at birth(Maestripieri et al., 2007). and reared with peers (Bennett et al., 2002; The occurrence of individual differences in Maestripieri et al., 2006a). Nevertheless, indivi-CSF concentrations of 5-HIAA and their asso- duals who carry the short allele for SERT appearciation with differences in aggressive, affiliative, to share some behavioral traits with individualsand maternal behavior has sparked interest in with low CSF 5-HIAA, including higher aggres-the origin of this variation. Studies of genotyped siveness and earlier age of male emigration fromprimate populations and studies of cross- the group (Trefilov et al., 2000). Rhesusfostered individuals have provided evidence for macaque mothers who abuse their infants aremoderate to strong heritability of CSF concen- more likely to carry the short allele of the sero-trations of 5-HIAA and other monoamine meta- tonin transporter gene than nonabusive mothersbolites (Higley et al., 1993; Rogers et al., 2004). (McCormack et., 2009). Furthermore, infantsHeritability of variation in serotonergic function with the short allele who are separated fromcould arise from any genes whose products par- their mothers at birth or physically abused byticipate in serotonin’s synthesis, release, reup- them are more likely to show anxiety and fear intake, or metabolism, or in genes that encode response to novelty and dysregulated HPA axisserotonin receptors (Manuck et al., 2006). A responses to stress and challenges than indivi-well-known case of genetic variation in seroto- duals with the same early experience who arenergic function involves the polymorphism in homozygous for the long allele (Barr et al.,the serotonin transporter (5-HTT or SERT) 2004; Bennett et al., 2002; Lesch et al., 1996;gene. In humans, rhesus macaques, and other McCormack et al., 2009).primates as well, the promoter region of this Comparative studies of functional variabilitygene (5-HTTLPR) exists in two allelic variants, of the serotonin transporter gene in seven dif-which differ in length. The short allele confers ferent species of macaques have shown that spe-lower transcriptional efficiency to the serotonin cies that are believed to more socially toleranttransporter gene (Bennett et al., 2002) and is and less despotic and nepotistic such as Barbaryassociated with reduced serotonin reuptake macaques, Tibetan macaques, and stumptailinto the presynaptic neuron and reduced sero- macaques are monomorphic for the SERTtonergic responsivity to neuroendocrine chal- gene. In contrast, species believed to be morelenges (Manuck et al., 2006). Human studies intolerant and aggressive such as rhesus, long-have shown that individuals with one or two tailed, and pigtail macaques are polymorphic forcopies of the short allele have greater amygdala the SERT gene, with rhesus macaques having theneuronal activation in response to faces with highest degree of polymorphism (Wendland etthreatening expressions (Skuse, 2006). These al., 2005). Tonkean macaques, which areindividuals also had reduced gray matter in the believed to be relatively docile and egalitarian,perigenual cingulate cortex (pACC) and in the are polymorphic as well. Although these findingsamygdala. The pACC has the greatest density of suggest that genetic variation in serotonergicserotonin terminals within the human cortex function may play an important role in deter-and it is a major target for projections from mining species differences in aggressivenessthe amygdala. fMRI studies have shown that among macaques, caution is needed in inter-people with at least one short allele had weaker preting these results for several reasons. First,functional interactions between ventromedial species differences in aggressiveness among

372 PRIMATE NEUROETHOLOGYmacaques are not well established. Second, the exhibited abusive parenting toward their first-species that are polymorphic for the SERT gene born offspring, whereas none of the femalesare all closely related to each other, and more reared by nonabusive mothers did (includingdistantly related from the species that are mono- those born to abusive mothers; Maestripieri,morphic. Although Tonkean macaques would 2005). Moreover, the abused females, bothbe expected to be monomorphic on the basis of cross-fostered and non–cross-fostered, whotheir presumed behavioral characteristics, they became abusive mothers had lower CSFare polymorphic like pigtail macaques, a closely 5-HIAA concentrations than the abused femalesrelated species from which they evolved. who did not become abusive mothers At the individual level, early experience can (Maestripieri et al., 2006a). Since abuse tendsbe an important source of variation in seroto- to co-occur with high rates of maternal rejection,nergic function in adulthood. Long-term effects our findings suggest that experience-inducedof early maternal deprivation on the develop- long-term alterations in serotonergic functionment of the brain serotonergic system have in females reared by highly rejecting and abusivebeen reported in laboratory-reared rhesus maca- mothers contribute to the manifestation ofques (Higley et al., 1991; Kraemer et al., 1989; maternal rejection and abusive parenting inShannon et al., 2005). In group-living rhesus adulthood. It is possible that experience-inducedmacaques, individuals exposed to high rates of reduction in serotonergic function results in ele-maternal rejection in infancy had significantly vated anxiety and impaired impulse control, andlower CSF concentrations of 5-HIAA across that high anxiety and impulsivity increase thetheir first 3 years of life than the individuals probability of occurrence of maternal rejectionexposed to low rates of maternal rejection and abusive parenting with one’s own offspring(Maestripieri et al., 2006a). This difference was later in life, perhaps in conjunction withfound both in individuals reared by their biolo- social learning resulting from direct experiencegical mothers and in cross-fostered juveniles, with one’s own mother or from observationsuggesting that it did not reflect genetic simila- of maternal interactions with siblingsrities between mothers and offspring. Among (Maestripieri, 2008).these juveniles, there was a significant negativecorrelation between CSF 5-HIAA and rates of THE NEURAL SUBSTRATES OFscratching (Maestripieri et al., 2006b), sug- SOCIAL BEHAVIOR: BRAINgesting that individuals with low CSF 5-HIAA LESION STUDIESwere more anxious than those with high 5-HIAA(see Maestripieri et al., 1992, and Schino et al., Studies employing brain lesions to investigate1991, for the relation between scratching and the role of different neural structures in theanxiety). When females who were reared by regulation of social behavior have focusedhigh-rejection mothers gave birth for the first mostly on the amygdala, and to a lesser extenttime, their low 5-HIAA was associated with on the hypothalamus, the hippocampus, tem-high rates of maternal rejection toward their poral lobes, and orbital frontal cortex. Theseown infants (Maestripieri et al., 2007). The structures play an important role in the proces-maternal rejection rates of daughters closely sing of environmental stimuli and the produc-resembled those of their mothers and the resem- tion of emotional responses that regulateblance was particularly strong for the cross-fos- survival-related behaviors. These brain regionstered females and their foster mothers are also rich in receptors for neuropeptides and(Maestripieri et al., 2007). monoamines as well as for other hormones and The serotonin system may also be involved in neurotransmitters that have been shown tothe intergenerational transmission of infant affect social behavior in primates and otherabuse. We reported that about half of the females animals (e.g., Way et al., 2007). In highlywho were abused by their mothers early in life, social organisms such as primates, limbic andwhether cross-fostered or non–cross-fostered, cortical responses to social stimuli can play a

NEUROBIOLOGY OF SOCIAL BEHAVIOR 373fundamental role in an individual’s ability to Lesions of the orbital frontal cortex appear toachieve successful survival and reproduction. result in avoidance of social interactions andFor example, since the amygdala is necessary alterations in aggressiveness and dominancefor the interpretation of social stimuli and the (Butter & Snyder, 1972; Raleigh & Steklis,production of emotional responses that regulate 1981). Individuals with these lesions also showavoidance and aggressive behavior, this struc- impairments in their social attachments, such asture can potentially play an important role in a weaker preference for their primary caregiverthe regulation of primate social behavior. The (Goursaud & Bachevalier, 2007).hypothalamus plays an important role in regu- An early study by Rosvold and colleagueslating the motivational aspects of sexual and (1954) reported that male rhesus monkeys whomaternal behavior, and the hippocampus may had achieved high dominance rank in artificiallybe relevant to social behavior regulation insofar created social groups fell to the bottom of theas it plays a role in processing the spatial and hierarchy and behaved submissively followingcontextual interrelations of social stimuli. The bilateral amygdalectomy and reintroduction toorbital frontal cortex is strongly connected with the group. Similarly, in a series of studies con-brain regions that process all sensory stimuli, ducted with rhesus macaques, stumptail maca-while temporal lobes have been implicated in ques and vervet monkeys, Kling and colleaguesthe processing of facial expressions and body reported that individuals with bilateral lesions ofmovements. Therefore, these structures are the amygdala who were reintroduced into theirlikely implicated in the neural control of social social groups failed to re-establish functionalcognition, and in the acquisition and processing social relationships with other individuals andof information that motivates and controls were either attacked or ignored by others (Dickssocial interactions. et al., 1969; Kling & Cornell, 1971; Kling & Studies of brain lesions and social behavior in Steklis, 1976; Kling et al., 1970). Changes inprimates have generally taken two different maternal and sexual behavior in amygdala-approaches. Some of them have investigated lesioned animals were reported as well (e.g.,the effects of lesions on the expression of social Kling & Brothers 1992). Spies and colleaguesbehavior in adults, while others have investi- (1976) reported that female rhesus monkeysgated the effects of lesions on the development with bilateral lesions of the amygdala showedof social behavior in infants. In general, the impaired sexual proceptive behavior butresults of studies in which temporal lobes, normal receptivity and copulatory behaviororbital frontal cortex, and hippocampus were when paired with a male in a cage (similar resultslesioned in adults have been consistent with have also been obtained with lesions of thethose of studies in which these brain areas were hypothalamus; see later).lesioned in infants. In the case of amygdala Early studies involving amygdala lesions inlesions, however, the results of studies involving monkeys have been criticized by Amaral (2002;adults and infants have produced somewhat see also Amaral et al., 2003) because these lesionsinconsistent results (see later). were not selective enough and the behavioral Young and adult monkeys with lesions of the observations were not accurate enough to war-temporal lobes are generally socially withdrawn rant strong conclusions regarding the relationor inactive and hyporeactive to fear-inducing between amygdala and social behavior. Amaralstimuli, and in the case of adults, they also and colleagues conducted a series of studiesshow inappropriate sexual behavior such as involving amygdala lesions in rhesus monkeys,mounting inanimate objects (Bachevalier et al., in which the specificity of such lesions was2001; Brown & Schaefer, 1888; Kluver & Bucy, greatly enhanced by the use of ibotenic acid, a1937, 1939). Individuals with these lesions neurotoxin that is injected stereotaxically intoappear to have impaired ability to discriminate the brain and selectively destroys the amygdalabetween conspecifics and objects, and therefore without affecting adjacent areas. One set of stu-to respond properly to socially relevant stimuli. dies examined the effects of amygdala lesions on

374 PRIMATE NEUROETHOLOGYthe behavior of adult males in a variety of labora- (Bauman et al., 2004). This finding was inter-tory testing conditions. In dyadic social interac- preted as indicative of an impairment in thetions in which the amygdala-lesioned males and perception of potential danger rather than as atheir unoperated controls were tested with the specific deficit in the bond with the mother. Thesame stimulus individuals, the lesioned males amygdala-lesioned infants, however, did notshowed reduced latency to engage in social inter- differ significantly from controls in theiractions with their partners and greater affilia- plasma cortisol response to separation fromtion, particularly during the early encounters, their mothers or in their cortisol responses tosuggesting that they had lower social anxiety dexamethasone suppression and ACTH chal-and more social disinhibition than controls lenge (Goursaud et al., 2006). At 6 to 12(Emery et al., 2001). The lesioned animals also months of age, the amygdala-lesioned infantsexhibited lower elevations in plasma cortisol showed reduced fear of novel objects such aslevels in response to the social encounters than rubber snakes but more fearful behavior thanthe controls did, suggesting that the amygdala both hippocampus-lesioned and sham-operatedlesion reduced the extent to which the indivi- controls during dyadic encounters with bothduals interpreted the interaction with an unfa- familiar and unfamiliar conspecifics (Baumanmiliar individual as stressful (Amaral, 2002). et al., 2004; Prather et al., 2001). The behaviorPerhaps as a result of the behavior of the lesioned of both amygdala- and hippocampus-lesionedindividuals, the stimulus partners directed more infants in both dyadic and group interactions,affiliative behavior toward them than toward the however, was generally normal and age appro-controls (Emery et al., 2001). The amygdala- priate. If anything, the amygdala-lesionedlesioned adult males showed reduced behavioral infants showed more affiliative and submissiveinhibition also in response to people and novel behavior than the infants in the other groupsobjects. For example, they had a lower latency to (Bauman et al., 2004). At approximately 18retrieve a grape and to approach and handle a months of age, dominance tests were conductedrubber snake than controls (Mason et al., 2006). in which the juveniles were given the opportu-Thus, adult males with bilateral lesions of the nity to retrieve preferred food items in a compe-amygdala show behavioral characteristics similar titive situation involving other individualsto those of individuals who experienced much (Bauman et al., 2006). In these tests, the amyg-larger lesions of the temporal lobes (Kluver & dala-lesioned individuals showed longer laten-Bucy, 1937, 1939). cies to retrieve the food, reduced aggressive Amaral and collaborators also investigated behaviors, and more frequent fear and submis-the long-term effects of brain lesions on infant sive behaviors than hippocampus-lesioned indi-behavioral development. Bilateral lesions of the viduals and sham-lesioned controls. Theamygdala or the hippocampus were performed behavioral effects of amygdala lesions conductedin 2-week-old infants, who were returned to in infancy, therefore, appeared to be opposite totheir mothers and reared by them in small those of similar lesions performed in adultcages with or without other mother-infant males. Finally, Goursaud and Bachevalierpairs. Mother-infant behavioral interactions (2007) reported that rhesus monkeys receivingwere not significantly altered by amygdala or bilateral ibotenic acid lesions of amygdala andhippocampus lesions, with the exception of hippocampus at 1 to 2 weeks of age and whoincreased mother-infant contact in the amyg- were subsequently reared by human caregiversdala-lesioned group (Bauman et al., 2004). did not differ from controls in their preferenceWhen infants were permanently separated from for their primary caregiver versus anothertheir mothers at 6 months of age, the amygdala- familiar human when tested in a social prefer-lesioned infants did not preferentially seek ence task at 11 months of age.proximity to their mother in a social preference Taken together, the results of these studiestest in which they could choose between their suggest that an intact amygdala is not necessarymother and another familiar adult female for the expression of normal social behavior in

NEUROBIOLOGY OF SOCIAL BEHAVIOR 375adult macaques or for normal social develop- primates, and since female primates are uniquement in infants (Amaral et al., 2003). Although in their ability to engage in sexual behaviorevidence from single neuron studies suggests outside of the fertile phase of their cycle (Wallen,that neurons in the amygdala fire at different 1990), it is possible that the neural control ofrates following exposure to different facial sexual behavior in primates overlaps with theexpressions of emotion (Gothard et al., 2007), neural control of affiliation and bonding to aAmaral and colleagues (2003) have recently larger extent than in other mammals. Althoughquestioned the hypothesis that the amygdala the hypothalamus plays an important role inplays an important role in social cognition (see regulating maternal motivation in rodents andBrothers, 1996). Rather, their view is that amyg- other mammals, there have been no studies inves-dala serves the function of a protection device: It tigating the effects of hypothalamic lesions onallows an individual to evaluate the extent to parental motivation and behavior in nonhumanwhich novel objects in the environment or social primates.situations pose a threat or danger and helps theindividual to produce an appropriate response,through projections to other areas of the brain CONCLUSIONSsuch as the cortex and the hippocampus. Withoutan intact amygdala, monkeys fail to properly Studies of the neural substrates or neurochem-evaluate and recognize the riskiness of a particular ical mechanisms underlying social behavior instimulus. As a result, amygdala-lesioned monkeys nonhuman primates are clearly limited whenshow a lack of fear responses to threatening compared to those conducted with otherobjects and appear to be uninhibited in poten- animals, particularly laboratory rodents.tially dangerous social situations. Moreover, since most research on the neuro- Primate brain lesion studies investigating biology of primate social behavior has been con-hypothalamic influences on social behavior have ducted with the few primate species that aremostly focused on sexual behavior. Research con- readily available in captivity, such as rhesusducted by Dixson and colleagues showed that macaques, marmosets, or squirrel monkeys, thelesions of the anterior and medial hypothalamus conclusions of these studies may not be general-in female marmosets impair female active initia- izable to other primates, let alone to othertion of sexual activity (proceptivity) but not animals. Nevertheless, the research findingsresponses to male sexual advances (receptivity) reviewed in this chapter have made a significant(Dixson, 1990; Dixson & Hastings, 1992; contribution to our understanding of the neu-Kendrick & Dixson, 1986). Consistent with these robiological regulation of primate socialresults, studies of macaques have shown that elec- behavior.trical stimulation of the ventromedial or preoptic Research on social cognition aside, most ofarea of the hypothalamus enhances female procep- the work investigating the neural and neuro-tive behavior toward males (Koyama et al., 1988). chemical control of social behavior has focusedMoreover, neurons in the ventromedial hypotha- on the limbic system and its relation to emo-lamus increase their firing rate while female maca- tional and motivational substrates of behavior.ques are engaged in proceptive behavior or The best experimental evidence linking specificcopulation, while those of the preoptic area brain regions or neurochemical systems to emo-decrease their firing rate during these activities tional substrates of social behavior has been(Aou et al., 1988). Taken together, the results of obtained for ‘‘negative’’ emotions such asthese studies suggest that the hypothalamic anxiety, fear, and impulsivity and for agonisticmechanisms regulating sexual behavior in pri- behaviors such as aggression and avoidance.mates may differ from nonprimate mammals in With the exception of work on endogenoussome important ways (Dixson, 1998). Different opioids and affiliation, ‘‘positive’’ emotionsareas of the hypothalamus may control different and their relations to affiliation and socialcomponents of sexual behavior in female bonding have proven more difficult to study.

376 PRIMATE NEUROETHOLOGYResearch in this area has been driven by findings research in primate social neuroethology. Oneobtained with rodents, but whether conceptua- constraint of brain imaging studies of non-lizations of affiliation and social bonding in human primates is that they must be conductedrodents can be directly extrapolated to primates under controlled laboratory conditions. Aremains unclear. Similarly, although there is a similar constraint exists also for human studies,wealth of evidence linking social bonding but despite this constraint, thousands of brainto oxytocin and vasopressin in rodents imaging studies with humans have been con-(e.g., Carter, 1998; Young, 2002), empirical evi- ducted in the past few decades, many of whichdence that these peptides affect social bonding in focused on social cognition and social behaviorprimates is very preliminary or equivocal. (e.g., Rilling et al.,). Brain imaging—both struc-Studies of complex affiliative behavior, and to tural and functional—is arguably also thesome extent also of aggressive, sexual, and par- experimental technique with the greatest poten-ental behavior, in primates will need greater tial for answering evolutionary questionsconceptual and experimental sophistication about brain-behavior relationships in primatesthan studies conducted with laboratory rodents. (e.g., Rilling & Insel, 1999). By systematically Because of ethical and logistical constraints in documenting similarities and differences in thethe study of brain-behavior relationships in non- structure of different brain regions acrosshuman primates, most studies to date have primate species and in how these regions areattempted to measure, often very indirectly, the activated during complex social interactions,activity of brain regions or neurochemical systems we could potentially acquire a great deal of newand then to correlate these measures with aspects information about the evolution of social andof behavior. Although the neuropharmacological cognitive complexity in the Primate Order andmanipulation of behavior in complex social set- the brain mechanisms that support it.tings could be an effective approach for testingneuroethological hypotheses, this approach hasgenerally been underutilized in primate research. REFERENCESThe effects of various psychotropic drugs on thesocial behavior of nonhuman primates have been Amaral, D. G. (2002). The primate amygdala andinvestigated in a number of studies (see Smith & the neurobiology of social behavior: Implications for understanding social anxiety.Byrd, 1983). In many of these studies, however, the Biological Psychiatry, 51, 11–17.relation between drugs and behavior was investi- Amaral, D. G., Capitanio, J. P., Jourdain, M.,gated without a clear understanding of the drug’s Mason, W. A., Mendoza, S. P., & Prather, Mmechanisms of action in the brain, and without (2003). The amygdala: Is it an essentialattempting to test specific hypotheses concerning component of the neural network for socialthe neurobiological regulation of behavior. Since cognition? Neuropsychologia, 41, 235–240.the physiological and molecular mechanisms of Aou, S., Oomura, Y., & Yoshimatsu, H. (1988).action of many neuropharmacological agents are Neuron activity of the ventromedialnow well understood, hypothesis-driven neuro- hypothalamus and the medial preoptic area ofpharmacological manipulations of social behavior the female monkey during sexual behavior.could play an important role in primate Brain Research, 455, 65–71. Bachevalier, J., Malkova, L., & Mishkin, M. (2001).neuroethological research. Effects of selective neonatal temporal lobe Brain lesions have proven useful in investi- lesions on socioemotional behavior in infantgating the role of particular brain regions in the rhesus monkeys (Macaca mulatta). Behavioralexpression of primate social behavior. Brain Neuroscience, 115, 545–559.lesion studies, however, have limitations in that Bailey, J. N., Breidenthal, S. E., Jorgensen, M. J.,lesions are not always specific and cause per- McCracken, J. T., & Fairbanks, L. A. (2007). Themanent and irreversible brain damage. Brain association of DRD4 and novelty seeking isimaging techniques are far less invasive than found in a nonhuman primate model.lesions and hold great promise for future Psychiatric Genetics, 17, 23–27.

NEUROBIOLOGY OF SOCIAL BEHAVIOR 377Banki, C. M., & Molnar, G. (1981). Cerebrospinal Brown, S., & Schaefer, E. A. (1888). An fluid 5-HIAA as an index of central serotonergic investigation into the functions of the occipital processes. Psychiatry Research, 5, 23–32. and temporal lobes of the monkey’s brain.Barr, C. S., Newman, T. K., Shannon, C., Parker, C., Philosophical Transactions of the Royal Society Dvoskin, R. L., Becker, M. L., et al. (2004b). of London Series B, 179, 303–327. Rearing condition and rh5-HTTLPR interact Butter, C. M., & Snyder, D. R. (1972). Alterations to influence limbic-hypothalamic-pituitary- in aversive and aggressive behaviors adrenal axis response to stress in infant following orbital frontal lesions in rhesus macaques. Biological Psychiatry, 55, 733–738. monkeys. Acta Neurobiologica Experimentalis,Barr, C. S., Schwandt, M. L., Lindell, S. G., Higley, J. 32, 525–565. D., Maestripieri, D., Goldman, D., et al. (2008). Carter, C. S. (1998). Neuroendocrine perspectives Variation at the mu-opioid receptor gene on social attachment and love. Psychoneuro- (OPRM1) influences attachment behavior in endocrinology, 23, 779–818. infant primates. Proceedings of the National Chamberlain, B., Ervin, F. R., Pihl, R. O., & Young, Academy of Sciences USA, 105, 5277–5281. S. N. (1987). The effect of raising of loweringBauman, M. D., Lavenex, P., Mason, W. A., tryptophan levels on aggression in vervet Capitanio, J. P., & Amaral, D. G. (2004). The monkeys. Pharmacology, Biochemistry, and development of social behavior following Behavior, 28, 503–510. neonatal amygdala lesions in rhesus monkeys. Chambers, K. C., & Phoenix, C. H. (1989). Journal of Cognitive Neuroscience, 16, Apomorphine, (-)-deprenyl, and yohimbine 1388–1411. fail to increase sexual behavior in rhesusBauman, M. D., Toscano, J. E., Mason, W. A., males. Behavioral Neuroscience, 103, 816–823. Lavenex, P., Amaral, D. G. (2006). The Champoux, M., Higley, J. D., & Suomi, S. J. (1997). expression of social dominance following Behavioral and physiological characteristics of neonatal lesions of the amygdala or Indian and Chinese-Indian hybrid rhesus hippocampus in rhesus monkeys (Macaca macaque infants. Developmental Psychobiology, mulatta). Behavioral Neuroscience, 120, 31, 49–63. 749–760. Cleveland, A., Westergaard, G. C., Trenkle, M. K.,Bennett, A. J., Lesch, K. P., Heils, A., Long, J. C., & Higley, J. D. (2004). Physiological predictors Lorenz, J. G., Shoaf, S. E., et al. (2002). Early of reproductive outcome and mother-infant experience and serotonin transporter gene behaviors in captive rhesus macaque females variation interact to influence primate CNS (Macaca mulatta). Neuropsychopharmacology, function. Molecular Psychiatry, 7, 118–122. 29, 901–910.Boccia, M. L., Goursaud, A. S., Bachevalier, J., Cooke, B., Higley, J. D., Shannon, C., Lindell, S. G., Anderson, K. D., & Pedersen, C. A. (2007). Higley, H. M., Suomi, S. J., et al. (1997). Rearing Peripherally administered non-peptide oxytocin history and CSF oxytocin as predictors of antagonist, L368,899, accumulates in limbic maternal competency in rhesus macaques. brain areas: A new pharmacological tool for the American Journal of Primatology, 42, 102. study of social motivation in non-human Csiffary, A., Ruttner, Z., Toth, Z., & Palkovits, M. primates. Hormones and Behavior, 52, 344–351. (1992). Oxytocin nerve fibers innervateBoccia, M. L., Panicker, A. K., Pedersen, C., & b-endorphin neurons in the arcuate nucleus of Petrusz, P. (2001). Oxytocin receptors in the rat hypothalamus. Neuroendocrinology, 56, nonhuman primate brain visualized with 429–435. monoclonal antibody. Neuroreport, 12, Depue, R. A., & Morrone-Strupinsky, J. V. (2005). 1723–1726. A neurobehavioral model of affiliative bonding:Botchin, M. B., Kaplan, J. R., Manuck, S. B., & Implications for conceptualizing a human trait Mann, J. J. (1993). Low versus high prolactin of affiliation. Behavioral and Brain Sciences, 28, responders to fenfluramine challenge: Marker of 313–395. behavioral differences in adult male cynomolgus Dicks, D., Myers, R. E., & Kling, A. (1969). Uncus macaques. Neuropsychopharmacology, 9, 93–99. and amygdala lesions: Effects on social behaviorBrothers, L. (1996). Brain mechanisms of social in the free ranging rhesus monkey. Science, 165, cognition. Journal of Psychopharmacology, 10, 2–8. 69–71.

378 PRIMATE NEUROETHOLOGYDix, T. (1991). The affective organization of in dopamine metabolism mapped in a primate parenting: Adaptive and maladaptive model using rference sequences from related processes. Psychological Bulletin, 110, 3–25. species. Proceedings of the National Academy ofDixson, A. F. (1990). Medial hypothalamic lesions Sciences USA, 104, 15811–15816. and sexual receptivity in the female common Gimpl, G., & Fahrenholz, F. (2001). The oxytocin marmoset (Callithrix jacchus). Folia receptor system: Structure, function, and Primatologica, 54, 46–56. regulation. Physiological Reviews, 81, 629–683.Dixson, A. F. (1998). Primate sexuality. Oxford: Gothard, K. M., Battaglia, F. P., Erickson, C. A., Oxford University Press. Spitler, K. M., & Amaral, D. G. (2007). NeuralDixson, A. F., & Hastings, M. H. (1992). Effects of responses to facial expression and facial identity ibotenic acid-induced neuronal degeneration in in the monkey amygdala. Journal of the hypothalamus upon proceptivity and sexual Neurophysiology, 97, 1671–1683. receptivity in the female marmoset. Journal of Goursaud, A. S., & Bachevalier, J. (2007). Social Neuroendocrinology, 4, 719–726. attachment in juvenile monkeys with neonatalDunbar, R. I. M. (1988). Primate social systems. lesion of the hippocampus, amygdala and orbital London: Croom Helm. frontal cortex. Behavioral Brain Research, 176,Dunbar, R. I. M. (1992). Neocortex size as a 75–93. constraint on group size in primates. Journal Goursaud, A. S., Mendoza, S. P., & Capitanio, J. P. of Human Evolution, 20, 469–493. (2006). Do neonatal bilateral ibotenic acidEmery, N. J., Capitanio, J. P., Mason, W. A., lesions of the hippocampal formation or of the Machado, C. J., Mendoza, S. P., & Amaral, D. amygdala impair HPA axis responsiveness and G. (2001). The effects of bilateral lesions of the regulation in infant rhesus macaques (Macaca amygdala on dyadic social interactions in rhesus mulatta)? Brain Research, 1071, 97–104. monkeys (Macaca mulatta). Behavioral Gradwell, P. B., Everitt, B. J., & Herbert, J. (1975). Neuroscience, 115, 515–544. 5-hydroxytryptamine in the central nervousFabre-Nys, C., Meller, R. E., & Keverne, E. B. system and sexual receptivity of female rhesus (1982). Opiate antagonists stimulate affiliative monkeys. Brain Research, 88, 281–293. behaviour in monkeys. Pharmacology, Grant, K. A., Shively, C. A., Nader, M. A., Biochemistry, and Behavior, 16, 653–659. Ehrenkaufer, R. L., Line, S. W., Morton, T. E.,Fairbanks, L. A., Fontenot, M. B., Phillips-Conroy, J. et al. (1998). Effect of social status on striatal E., Jolly, C. J., Kaplan, J. R., & Mann, J. J. (1999). dopamine D2 receptor binding characteristics CSF monoamines, age and impulsivity in wild in cynomolgus monkeys assessed with positron grivet monkeys (Cercopithecus aethiops aethiops). emission tomography. Synapse, 29, 80–83. Brain, Behavior, and Evolution, 53, 305–312. Graves, F. C., Wallen, K., & Maestripieri, D. (2002).Fairbanks, L. A., Jorgensen, M. J., Huff, A., Blau, K., Opioids and attachment in rhesus macaque Hung, Y., & Mann, J. J. (2004). Adolescent abusive mothers. Behavioral Neuroscience, 116, impulsivity predicts adult dominance 489–493. attainment in male vervet monkeys. American Guard, H. J., Newman, J. D., & Roberts, R. L. Journal of Primatology, 64, 1–17. (2002). Morphine administration selectivelyFairbanks, L. A., Melega, W. P., Jorgensen, M. J., facilitates social play in common marmosets. Kaplan, J. R., & McGuire, M. T. (2001). Social Developmental Psychobiology, 41, 37–49. impulsivity inversely associated with CSF 5-HIAA Hammock, E. A. D., & Young, L. J. (2005). Micro- and fluoxetine exposure in vervet monkeys. satellite instability generates diversity in brain and Neuropsychopharmacology, 24, 370–378. sociobehavioral traits. Science, 308, 1630–1634.Fairbanks, L. A., Melega, W. P., & McGuire, M. T. Heinrichs, M., Baumgartner, T., Kirschbaum, C., & (1998). CSF 5-HIAA is associated with individual Ehlert, U. (2003). Social support and oxytocin differences in maternal protectiveness in vervet interact to suppress cortisol and subjective monkeys. American Journal of Primatology, 45, responses to psychosocial stress. Biological 179–180. Psychiatry, 54, 1389–1398.Freimer, N. B., Service, S. K., Ophoff, R. A., Higley, J. D. (2003). Aggression. In: D. Maestripieri Jasinska, A. J., Villeneuve, A., Belisle, A., et al. (Ed.), Primate psychology (pp. 17–40). (2007). A quantitative trait locus for variation Cambridge, MA: Harvard University Press.

NEUROBIOLOGY OF SOCIAL BEHAVIOR 379Higley, J. D., King, S. T., Hasert, M. F., Champoux, Insel, T. R. (1992). Oxytocin and the neurobiology M., Suomi, S. J., & Linnoila, M. (1996a). of attachment. Behavioral and Brain Sciences, Stability of interindividual differences in 15, 515–516. serotonin function and its relationship to Insel, T. R., & Winslow, J. T. (1998). Serotonin and aggressive wounding and competent social neuropeptides in affiliative behaviors. Biological behavior in rhesus macaque females. Psychiatry, 44, 207–219. Neuropsychopharmacology, 14, 67–76. Kalin, N. H., Shelton, S. E., & Barksdale, C. M.Higley, J. D., Mehlman, P. T., Higley, S. B., Fernald, (1988). Opiate modulation of separation- B., Vickers, S., Lindell, S. G., et al. (1996b). induced distress in non-human primates. Excessive mortality in young male nonhuman Brain Research, 440, 285–292. primates with low CSF 5-HIAA concentrations. Kalin, N. H., Shelton, S. E., & Lynn, D. E. (1995). Archives of General Psychiatry, 53, 537–543. Opiate systems in mother and infant primatesHigley, J. D., Mehlman, P. T., Poland, R. E., Taub, coordinate intimate contact during reunion. D. M., Vickers, J., Suomi, S. J., et al. (1996c). Psychoneuroendocrinology, 20, 735–742. CSF testosterone and CSF 5-HIAA correlate Kaplan, J. R., Fontenot, M. B., Berard, J. D., with different types of aggressive behaviors. Manuck, S. B., & Mann, J. J. (1995). Delayed Biological Psychiatry, 40, 1067–1082. dispersal and elevated monoaminergic activityHigley, J. D., Mehlman, P. T., Taub, D. M., Higley, in free-ranging rhesus monkeys. American S. B., Suomi, S. J., Linnoila, M., et al. (1992b). Journal of Primatology, 35, 229–234. Cerebrospinal fluid monoamine and adrenal Kaplan, J. R., Phillips-Conroy, J., Fontenot, M. B., correlates of aggression in free-ranging rhesus Jolly, C. J., Fairbanks, L. A., & Mann, J. J. monkeys. Archives of General Psychiatry, 49, (1999). Cerebrospinal fluid monoaminergic 436–441. metabolites differ in wild anubis and hybridHigley, J. D., Suomi, S. J., & Linnoila, M. (1991). (Anubis hamadryas) baboons: Possible CSF monoamine metabolite concentrations relationships to life history and behavior. vary according to age, rearing, and sex, and are Neuropsychopharmacology, 20, 517–524. influenced by the stressor of social separation in Kaplan, J. R., Manuck, S. B., Fontenot, B., & rhesus monkeys. Psychopharmacology, 103, Mann, J. J. (2002). Central nervous system 551–556. monoamine correlates of social dominanceHigley, J. D., Suomi, S. J., & Linnoila, M. (1992a). A in cynomolgus monkeys (Macaca longitudinal assessment of CSF monoamine fascicularis). Neuropsychopharmacology, 26, metabolite and plasma cortisol concentrations 431–443. in young rhesus monkeys. Biological Psychiatry, Kendrick, K. M., & Dixson, A. F. (1986). 32, 127–145. Anteromedial hypothalamic lesions blockHigley, J. D., Thompson, W. W., Champoux, M., proceptivity but not receptivity in the female Goldman, D., Hasert, M. F., Kraemer, G. W., et common marmoset (Callithrix jacchus). Brain al. (1993). Paternal and maternal genetic and Research, 375, 221–229. environmental contributions to cerebrospinal Keverne, E. B., Martensz, N. D., & Tuite, B. (1989). fluid monoamine metabolites in rhesus b-endorphin concentrations in cerebrospinal monkeys (Macaca mulatta). Archives of fluid of monkeys are influenced by grooming General Psychiatry, 50, 615–623. relationships. Psychoneuroendocrinology, 14,Holman, S. D., & Goy, R. W. (1995). Experiential 155–161. and hormonal correlates of care-giving in Kling, A., & Brothers, L. A. (1992). The amygdala rhesus macaques. In: C. R. Pryce, R. D. and social behavior. In: J. Aggleton (Ed.), The Martin, & D. Skuse (Eds.), Motherhood in amygdala: Neurobiological aspects of emotion, Human and Nonhuman Primates. Biosocial memory, and mental dysfunction (pp. 353–377). Determinants (pp. 87–93). Basel: Karger. New York: Wiley-Liss.Howell, S., Westergaard, G., Hoos, B., Chavanne, Kling, A., & Cornell, R. (1971). Amygdalectomy T. J., Shoaf, S. E., Cleveland, A., et al. (2007). and social behavior in the caged stump-tailed Serotonergic influences on life-history macaque (Macaca speciosa). Folia outcomes in free-ranging male rhesus Primatologica, 14, 190–208. macaques. American Journal of Primatology, Kling, A., Lancaster, J., & Benitone, J. (1970). 69, 1–15. Amygdalectomy in the free-ranging vervet

380 PRIMATE NEUROETHOLOGY (Cercopithecus aethiops). Journal of Psychiatric Measurement of anxiety and identification of Research, 7, 191–199. anxiety-eliciting situations. Ethology, 95, 19–31.Kling, A., & Steklis, H. D. (1976). A neural Maestripieri, D. (1993b). Maternal anxiety in substrate for affiliative behavior in nonhuman rhesus macaques (Macaca mulatta). II. primates. Brain, Behavior, and Evolution, 13, Emotional bases of individual differences in 216–238. mothering style. Ethology, 95, 32–42.Kluver, H., & Bucy, P. C. (1937). ‘‘Psychic Maestripieri, D. (1994). Infant abuse associated blindness’’ and other symptoms following with psychosocial stress in a group-living bilateral temporal lobectomy in rhesus pigtail macaque (Macaca nemestrina) mother. monkeys. American Journal of Physiology, 119, American Journal of Primatology, 32, 41–49. 352–353. Maestripieri, D. (1999). The biology of humanKluver, H., & Bucy, P. C. (1939). Preliminary parenting: Insights from nonhuman primates. analysis of functions of the temporal lobes in Neuroscience and Biobehavioral Reviews, 23, monkeys. Archives of Neurology and Psychiatry, 411–422. 42, 979–1000. Maestripieri, D. (2005). Early experience affectsKoyama, Y., Fujita, I., Aou, S., & Oomura, Y. the intergenerational transmission of infant (1988). Proceptive presenting elicited by abuse in rhesus monkeys. Proceedings of the electrical stimulation of the female monkey National Academy of Sciences USA, 102, hypothalamus. Brain Research, 446, 199–203. 9726–9729.Kozorovitskiy, Y., Hughes, M., Lee, K., & Gould, E. Maestripieri, D. (2008). The role of the brain (2006). Fatherhood affects dendritic spines and serotonergic system in the origin and vasopressin V1a receptors in the primate transmission of adaptive and maladaptive prefrontal cortex. Nature Neuroscience, 9, variations in maternal behavior in rhesus 1094–1095. macaques. In: R. Bridges (Ed.), Neurobiology ofKraemer, G. W. (1992). A psychobiological theory the parental brain (pp. 163–173). Amsterdam: of attachment. Behavioral and Brain Sciences, Elsevier. 15, 493–451. Maestripieri, D., Higley, J. D., Lindell, S. G.,Kraemer, G. W., Ebert, M. H., Schmidt, D. E., & Newman, T. K., McCormack, K. M., & McKinney, W. T. (1989). A longitudinal study Sanchez, M. M. (2006a). Early maternal of the effect of different social rearing rejection affects the development of conditions on cerebrospinal fluid norepine- monoaminergic systems and adult abusive phrine and biogenic amine metabolites in parenting in rhesus macaques. Behavioral rhesus monkeys. Neuropsychopharmacology, 2, Neuroscience, 120, 1017–1024. 175–189. Maestripieri, D., Lindell, S. G., Ayala, A., Gold, P.Kyes, R. C., Botchin, M. B., Kaplan, J. R., Manuck, W., & Higley, J. D. (2005). Neurobiological S. B., & Mann, J. J. (1995). Aggression and brain characteristics of rhesus macaque abusive serotonergic responsivity: Response to slides in mothers and their relation to social and male macaques. Physiology and Behavior, 57, maternal behavior. Neuroscience and 205–208. Biobehavioral Reviews, 29, 51–57.Lesch, K. P., Bengel, D., Heils, A., Sabol, S. Z., Maestripieri, D., Lindell, S. G., & Higley, J. D. Greenberg, B. D., Petri, S., et al. (1996). (2007). Intergenerational transmission of Association of anxiety-related traits with a maternal behavior in rhesus monkeys and its polymorphism in the serotonin transporter underlying mechanisms. Developmental gene regulatory region. Science, 274, 1527– Psychobiology, 49, 165–171. 1531. Maestripieri, D., McCormack, K. M., Lindell, S. G.,Lindell, S. G., Higley, J. D., Shannon, C., & Linnoila, Higley, J. D., & Sanchez, M. M. (2006b). M. (1997). Low levels of CSF 5-HIAA in female Influence of parenting style on the offspring’s rhesus macaques predict mother-infant behavior and CSF monoamine metabolite levels interaction patterns and mother’s CSF 5-HIAA in crossfostered and noncrossfostered female correlates with infant’s CSF 5-HIAA. American rhesus macaques. Behavioral Brain Research, Journal of Primatology, 2, 129. 175, 90–95.Maestripieri, D. (1993a). Maternal anxiety in Maestripieri, D., Schino, G., Aureli, F., & Troisi, A. rhesus macaques (Macaca mulatta). I. (1992). A modest proposal: Displacement

NEUROBIOLOGY OF SOCIAL BEHAVIOR 381 activities as an indicator of emotions in free-ranging primates. American Journal of primates. Animal Behaviour, 44, 967–979. Psychiatry, 152, 907–913.Manuck, S. B., Kaplan, J. R., & Lotrich, F. E. Mehlman, P. T., Higley, J. D., Fernald, B. J., (2006). Brain serotonin and aggressive Sallee, F. R., Suomi, S. J., & Linnoila, M. (1997). disposition in humans and nonhuman CSF 5-HIAA, testosterone, and sociosexual primates. In: R. J. Nelson (Ed.), Biology of behaviors in free-ranging male rhesus macaques aggression (pp. 65–113). Oxford: Oxford in the mating season. Psychiatry Research, 72, University Press. 89–102.Manuck, S. B., Kaplan, J. R., Rymeski, B. A., Meller, R. E., Keverne, E. B., & Herbert, J. (1980). Fairbanks, L. A., & Wilson, M. E. (2003). Behavioural and endocrine effects of naltrexone Approach to a social stranger is associated in male talapoin monkeys. Pharmacology, with low central nervous system serotonergic Biochemistry, and Behavior, 13, 663–672. responsivity in female cynomolgus monkeys Miczek, K. A., & Fish, E. W. (2006). Monoamines, (Macaca fascicularis). American Journal of GABA, glutamate, and aggression. In: R. J. Nelson Primatology, 61, 187–194. (Ed.), Biology of aggression (pp. 114–149). Oxford:Martel, F. L., Nevison, C. M., Rayment, F. D., Oxford University Press. Simpson, M. J. A., & Keverne, E. B. (1993). Miczek, K. A., Woolley, J., Schlisserman, S., & Opioid receptor blockade reduces maternal Yoshimura, H. (1981). Analysis of affect and social grooming in rhesus monkeys. amphetamine effects on agonistic and Psychoneuroendocrinology, 18, 307–321. affiliative behavior in squirrel monkeysMartel, F. L., Nevison, C. M., Simpson, M. J. A., & (Saimiri sciureus). Pharmacology, Biochemistry, Keverne, E. B. (1995). Effects of opioid receptor and Behavior, 14, 103–107. blockade on the social behavior of rhesus Nelson, E., & Panksepp, J. (1998). Brain substrates monkeys living in large family groups. of infant-mother attachment: Contributions of Developmental Psychobiology, 28, 71–84. opioids, oxytocin, and norepinephrine.Martensz, N. D., Vellucci, S. V., Keverne, E. B., & Neuroscience and Biobehavioral Reviews, 22, Herbert, J. (1986). b-endorphins levels in the 437–452. cerebrospinal fluid of male talapoin monkeys in Numan, M., & Insel, T. R. (2003). The neurobiology social groups related to dominance status and of parental behavior. Berlin: Springer. the luteinizing hormone response to naloxone. Panksepp, J., Herman, B. H., Vilberg, T., Bishop, Neuroscience, 18, 651–658. P., & DeEskinazi, F. G. (1980). EndogenousMason, W. A., Capitanio, J. P., Machado, C. J., opioids and social behavior. Neuroscience and Mendoza, S. P., & Amaral, D. G. (2006). Biobehavioral Reviews, 4, 473–487. Amygdalectomy and responsiveness to novelty Panksepp, J., Nelson, E., & Bekkedal, M. (1999). in rhesus monkeys (Macaca mulatta): Brain systems for the mediation of social Generality and individual consistency of separation-distress and social-reward: Evolutio- effects. Emotion, 6, 73–81. nary antecedents and neuropeptide intermedi-McCormack, K., Newman, T. K., Higley, J. D., aries. In: C. S. Carter, I. I. Lederhendler, & B. Maestripieri, D., & Sanchez, M. M.’(2009). Kirkpatrick (Eds.), The integrative neurobiology Serotonin transporter gene variation, infant of affiliation (pp. 221–243). Cambridge, MA: abuse, and responsiveness to stress in rhesus MIT Press. acaque mothers and infants. Hormones and Parker, K. J., Buckmaster, C. L., Schatzberg, A. F., & Behavior, 55, 538-547. Lyons, D. M. (2005). Intranasal oxytocinMehlman, P. T., Higley, J. D., Faucher, I., Lilly, A. administration attenuates the ACTH stress A., Taub, D. M., Suomi, S. J., et al. (1994). Low response in monkeys. Psychoneuroendocrinology, CSF 5-HIAA concentrations and severe 30, 924–929. aggression and impaired impulse control in Pomerantz, S. M. (1990). Apomorphine facilitates nonhuman primates. American Journal of male sexual behavior of rhesus monkeys. Psychiatry, 151, 1485–1491. Pharmacology, Biochemistry, and Behavior, 35,Mehlman, P. T., Higley, J. D., Faucher, I., Lilly, A. 659–664. A., Taub, D. M., Vickers, J. H., et al. (1995). Pomerantz, S. M. (1991). Quinelorane (LY163502), Correlation of CSF 5-HIAA concentrations a D2 dopamine receptor agonist, acts centrally with sociality and the timing of emigration in to facilitate penile erections of male rhesus

382 PRIMATE NEUROETHOLOGY monkeys. Pharmacology, Biochemistry, and Raleigh, M. J., & Steklis, H. D. (1981). Effects of Behavior, 39, 123–128. orbital frontal and temporal neocortical lesionsPomerantz, S. M., Hepner, B. C., & Wertz, J. M. on affiliative behavior of vervet monkeys (1991). 5-HT1A and 5-HT1C/1D receptor (Cercopithecus aethiops sabaeus). Experimental agonists produce reciprocal effects on male Neurology, 73, 378–389. sexual behavior of rhesus monkeys. European Rendall, D., & Di Fiore, A. (1995). The road less Journal of Pharmacology, 243, 227–234. traveled: Phylogenetic perspectives inPrather, M. D., Lavenex, P., Mauldin-Jordan, M. L., primatology. Evolutionary Anthropology, 4, Mason, W. A., Capitanio, J. P., Mendoza, S. P., 43–52. et al. (2001). Increased social fear and decreased Rilling, J. K., & Insel, T. R. (1999). The primate fear of objects in monkeys with neonatal neocortex in comparative perspective using amygdala lesions. Neuroscience, 106, 653–658. magnetic resonance imaging. Journal ofPryce, C. R. (1992). A comparative systems model Human Evolution, 37, 191–223. of the regulation of maternal motivation in Rilling, J. K., Sanfey, A. G., Nystrom, L. E., Cohen, mammals. Animal Behaviour, 43, 417–441. J. D., Gutman, D. A., Zeh, T. R., et al. (2004b).Raleigh, M. J., Brammer, G. L., Yuwiler, A., Imaging the social brain with fMRI and Flannery, J. W., & McGuire, M. T. (1980). interactive games. International Journal of Serotonergic influences on the social behavior Neuropsychopharmacology, 7, S477–S478. of vervet monkeys (Cercopithecus aethiops Rilling, J. K., Winslow, J. T., & Kilts, C. D. (2004a). sabaeus). Experimental Neurology, 68, 322–334. The neural correlates of mate competition inRaleigh, M. J., Brammer, G. L., & McGuire, M. T. dominant male rhesus macaques. Biological (1983). Male dominance, serotonergic systems, Psychiatry, 56, 364–375. and the behavioral and physiological effects of Rilling, J. K., Winslow, J. T., O’Brien, D., Gutman, drugs in vervet monkeys (Cercopithecus aethiops D. A., Hoffman, J. M., & Kilts, C. D. (2001). sabaeus). In: K. A. Miczek (Ed.), Neural correlates of maternal separation in Ethopharmacology: Primate models of rhesus monkeys. Biological Psychiatry, 49, neuropsychiatric disorders (pp. 185–197). New 146–157. York: Alan R. Liss. Rogers, J., Martin, L. J., Comuzzie, A. G., Mann, J.Raleigh, M. J., Brammer, G. L., McGuire, M. T., & J., Manuck, S. B., Leland, M., et al. (2004). Yuwiler, A. (1985). Dominant social status Genetics of monoamine metabolites in facilitates behavioral effects of serotonergic baboons: Overlapping sets of genes influence agonists. Brain Research, 348, 274–282. levels of 5-hydroxyindolacetic acid, 3-hydroxy-Raleigh, M. J., Brammer, G. L., Ritvo, E. R., Geller, 4-methoxyphenylglycol, and homovanillic E., McGuire, M. T., & Yuwiler, A. (1986). acid. Biological Psychiatry, 55, 739–744. Effects of chronic fenfluramine on blood Rosenblum, L. A., Smith, E. L. P., Altemus, M., serotonin, cerebrospinal fluid metabolites, and Scharf, B. A., Owens, M. J., Nemeroff, C. B., et behavior in monkeys. Psychopharmacology, 90, al. (2002). Differing concentrations of 503–508. corticotropin-releasing factor and oxytocin inRaleigh, M. J., & McGuire, M. T. (1986). Animal the cerebrospinal fluid of bonnet and pigtail analogues of ostracism: Biological mechanisms macaques. Psychoneuroendocrinology, 27, and social consequences. Ethology and 651–660. Sociobiology, 7, 53–66. Rosvold, H. E., Mirsky, A. F., & Pribram, K. H.Raleigh, M. J., & McGuire, M. T. (1990). Social (1954). Influence of amygdalectomy on social influences on endocrine function in male behavior in monkeys. Journal of Comparative vervet monkeys. In: T. E. Ziegler & F. B. Physiology and Psychology, 47, 173–178. Bercovitch (Eds.), Socioendocrinology of Schino, G., & Troisi, A. (1992). Opiate receptor primate reproduction (pp. 95–111). New York: blockade in juvenile macaques: Effect on Wiley-Liss. affiliative interactions with their mothers andRaleigh, M. J., McGuire, M. T., Brammer, G. L., group companions. Brain Research, 576, Pollack, D. B., & Yuwiler, A. (1991). 125–130. Serotonergic mechanisms promote dominance Schino, G., Troisi, A., Perretta, G., & Monaco, V. acquisition in adult male vervet monkeys. Brain (1991). Measuring anxiety in nonhuman Research, 559, 181–190. primates: Effect of lorazepam on macaque

NEUROBIOLOGY OF SOCIAL BEHAVIOR 383 scratching. Pharmacology, Biochemistry, and behavior in female rhesus monkeys. Physiology Behavior, 38, 889–891. and Behavior, 17, 985–992.Schwandt, M. L., Howell, S., Bales, K., Jaffe, B. D., Thierry, B., Singh, M., & Kaumanns, W. (Eds.). Westergaard, G. C., & Higley, J. D. (2007). (2004). Macaque societies: A model for the Associations between the neuropeptides evolution of social organization. Cambridge: oxytocin and vasopressin and the behavior of Cambridge University Press. free-ranging female rhesus macaques (Macaca Toloczko, D. M., Young, L., & Insel, T. R. (1997). mulatta). American Journal of Physical Are there oxytocin receptors in the primate Anthropology Supplement, 44, 210–211. brain? Annals of the New York Academy ofShannon, C., Schwandt, M. L., Champoux, M., Sciences, 807, 506–509. Shoaf, S. E., Suomi, S. J., Linnoila, M., et al. Trefilov, A., Berard, J., Krawczak, M., & Schmidtke, (2005). Maternal absence and stability of J. (2000). Natal dispersal in rhesus macaques is individual differences in CSF 5-HIAA related to serotonin transporter gene promoter concentrations in rhesus monkey infants. variation. Behaviour Genetics, 30, 295–301. American Journal of Psychiatry, 162, Troisi, A., & D’Amato, F. R. (1991). Anxiety in the 1658–1664. pathogenesis of primate infant abuse: ASheese, B. E., Voelker, P. M., Rothbart, M. K., & pharmacological study. Psychopharmacology, Posner, M. I. (2007). Parenting quality interacts 103, 571–572. with genetic variation in dopamine receptor D4 Uvnas-Moberg, K. (1998). Oxytocin may mediate the to influence temperament in early childhood. benefits of positive social interaction and Development and Psychopathology, 19, emotions. Psychoneuroendocrinology, 23, 819–835. 1039–1046. Wallen, K. (1990). Desire and ability: HormonesShively, C. A. (1998). Social subordination stress, and the regulation of female sexual behavior. behavior, and central monoaminergic function Neuroscience and Biobehavioral Reviews, 14, in female cynomolgus monkeys. Biological 233–241. Psychiatry, 44, 882–891. Way, B. M., Lacan, G., Fairbanks, L. A., & Melega,Shively, C. A., Fontenot, B., & Kaplan, J. R. (1995). W. P. (2007). Architectonic distribution of the Social status, behavior, and central serotonergic serotonin transporter within the orbitofrontal responsivity in female cynomolgus monkeys. cortex of the vervet monkey. Neuroscience, 148, American Journal of Primatology, 37, 333–350. 937–948.Skuse, D. (2006). Genetic influences on the neural Wendland, J. R., Lesch, K. P., Newman, T. K., basis on social cognition. Philosophical Timme, A., Gachot-Neveu, H., Thierry, B., et Transactions of the Royal Society of London al. (2005). Differential functional variability of Series B, 361, 2129–2141. serotonin transporter and monoamine oxidaseSmith, E. O., & Byrd, L. D. (1983). Studying the A genes in macaque species displaying behavioral effects of drugs in group-living contrasting levels of aggression-related nonhuman primates. In: K. A. Miczek (Ed.), behavior. Behaviour Genetics, 36, 163–172. Ethopharmacology: Primate models of Wester, P., Bergstrom, U., Eriksson, A., Gezelius, neuropsychiatric disorders (pp. 1–31). New C., Hardy, J., & Winblad, B. (1990). Ventricular York: Alan Liss. cerebrospinal fluid monoamine transmitter andSmuts, B. B., Cheney, D. L., Seyfarth, R. M., metabolic concentrations reflect human brain Wrangham, R. W., & Struhsaker, T. T. (Eds.). neurochemistry in autopsy cases. Journal of (1987). Primate societies. Chicago: University of Neurochemistry, 54, 1148–1156. Chicago Press. Westergaard, G. C., Mehlman, P. T., Shoaf, S. E.,Snowdon, C. T., Ziegler, T. E., Schultz-Darken, N. Suomi, S. J., & Higley, J. D. (1999). CSF 5-HIAA J., & Ferris, C. F. (2006). Social odours, sexual and aggression in female macaque monkeys: arousal and pairbonding in primates. Species and individual differences. Philosophical Transactions of the Royal Society Psychopharmacology, 145, 440–446. of London Series B, 361, 2079–2089. Westergaard, G. C., Suomi, S. J., Chavanne, T. J.,Spies, H. G., Norman, R. L., Clifton, D. K., Houser, L., Hurley, A., Cleveland, A., et al. Ochsner, A. J., Jensen, J. N., & Phoenix, C. H. (2003). Physiological correlates of aggression (1976). Effects of bilateral amygdaloid lesions and impulsivity in free-ranging female primates. on gonadal hormones in serum and on sexual Neuropsychopharmacology, 28, 1045–1055.

384 PRIMATE NEUROETHOLOGYWinslow, J. T. (2005). Neuropeptides and nonsocial buffering in rhesus monkeys. human primate social deficits associated with Neuropsychopharmacology, 28, 910–918. pathogenic rearing experience. International Yodyingyuad, U., de La Riva, C., Abbott, D. H., Journal of Developmental Neuroscience, 23, Herbert, J., & Keverne, E. B. (1985). 245–251. Relationship between dominance hierarchy,Winslow, J. T., & Insel, T. R. (1991). Social status cerebrospinal fluid levels of amine transmitter in pairs of male squirrel monkeys determines the metabolites (5-hydroxy-indoleacetic acid and behavioral response to central oxytocin adminis- homovanillic acid) and plasma cortisol levels tration. Journal of Neuroscience, 11, 2032–2038. in monkeys. Neuroscience, 16, 851–858.Winslow, J. T., Noble, P. L., Lyons, C. K., Sterk, S. Young, L. J. (2002). The neurobiology of social M., & Insel, T. R. (2003). Rearing effects on recognition, approach, and avoidance. cerebrospinal fluid oxytocin concentration on Biological Psychiatry, 51, 18–26.

CHAPTER 20Neural Bases of Numerical CognitionAndreas NiederINTRODUCTION NUMERICAL QUANTITY (CARDINALITY)Dealing with numbers had long been thought tobe an exclusive faculty of language-endowed Neurons Encoding Numerical Quantityhumans. Symbolic number representations, suchas numerals and number words, and the infinite Recordings in monkeys trained to discriminatemathematical manipulations they enable are numerosity demonstrated the capacity of singleuniquely human cultural achievements and neurons to encode cardinality (Nieder & Miller,shape our technologically advanced and scientific 2004a; Nieder et al., 2002, 2006). In the basicculture. However, basic numerical competence layout of the task, monkeys viewed a sequence ofdoes not depend on language; it is rooted in bio- two displays separated by a memory delay andlogical primitives that can already be found in were required to judge whether the displays con-animals. As illustrated by Elizabeth Brannon and tained the same number of items (Fig. 20.1A).colleagues (Chapter 8), animals possess impressive To ensure that the monkeys solved the task bynumerical capabilities and are able to nonverbally judging number per se rather than simply mem-and approximately grasp the numerical properties orizing sequences of visual patterns or exploitingof objects and events. Such a numerical estimation low-level visual features that correlate withsystem for representing number as language-inde- number, sensory cues (such as position, shape,pendent mental magnitudes (analog magnitude overall area, circumference, and density) weresystem) is a precursor on which verbal numerical varied considerably and controlled for (Niederrepresentations build, and their neural founda- et al., 2002).tions can be studied in animal models. Numerosity-selective neurons were tuned to This review charts the progress made in our the number of items in a visual display; that is,understanding of the neuronal substrates and they showed maximum activity to one of themechanisms of nonverbal numerical compe- presented quantities—a neuron’s ‘‘preferredtence in nonhuman primates. It is structured numerosity’’—and a progressive dropoff as theaccording to the two major concepts numerical quantity became more remote from the pre-cognition encompasses: numerical quantity and ferred number (Nieder & Merten, 2007; Niedernumerical rank (Wiese, 2003). Numerical quan- et al., 2002). Importantly, changes in the physicaltity refers to the empirical property cardinality appearance of the displays had no effect on the(‘‘numerosity,’’ the size of a set) of objects and activity of numerosity-selective neurons (Niederevents. It pertains to the question ‘‘How many?’’ & Miller, 2004a; Nieder et al., 2002, 2006).for numerable quantity, and ‘‘How much?’’ for A high proportion of numerosity detectorsinnumerable quantity. Numerical rank refers to (Fig. 20.1B) was found in the lateral prefrontalthe empirical property serial order and is sought cortex (PFC) (Nieder et al., 2002). In the pos-after by the question ‘‘Which position?’’ terior parietal cortex (PPC), numerosity-selective 385

Figure 20.1 Representation of visual cardinality in rhesus monkeys. (A) Delayed match-to-sample taskwith visually presented numerosity as the stimulus dimension of interest. A trial started when the monkeygrasped a lever and fixated at a central target. After 500 ms of pure fixation, the sample stimulus (800 ms)cued the monkey for a certain numerosity it had to memorize during a 1,000-ms delay period. Then, the test1 stimulus was presented, which in 50% of cases was a match showing the same number of dots as cuedduring the sample period. In the other 50% of cases the test 1 display was a nonmatch, which showed adifferent numerosity as the sample display. After a nonmatch test stimulus, a second test stimulus (test 2)appeared that was always a match. To receive a fluid reward, monkeys were required to release the lever assoon as a match appeared. Trials were pseudo-randomized and balanced across all relevant features.Monkeys were required to maintain fixation throughout the sample and delay period. (B) Lateral view ofa monkey brain showing the recording sites in lateral prefrontal cortex (LPFC), posterior parietal cortex(PPC), and anterior inferior temporal cortex (aITC). The proportion of numerosity-selective neurons ineach area is color coded according to the color scale. The intraparietal sulcus (IPS) is unfolded to show thedifferent areas in the lateral and medial walls. Numbers on prefrontal cortex (PFC) indicate anatomicalareas. As, arcuate sulcus; Cs, central sulcus; LF, lateral fissure; LS, lunate sulcus; Ps, principal sulcus; Sts,Superior temporal sulcus. After Nieder, A., & Miller, E. K. (2004a). A parieto-frontal network for visualnumerical information in the monkey. Proceedings of the National Academy of Sciences of the USA, 101,7457–7462. (C,D) Responses of single neurons that were recorded from the PFC (C) and the IPS (D). Bothneurons show graded discharge during sample presentation (interval shaded in gray, 500 to 1,300 ms) as afunction of numerosities 1 to 5 (color-coded averaged discharge functions). The insets in the upper rightcorner show the tuning of both neurons and their responses to different control stimuli. The preferrednumerosity was 4 for the PFC neuron (B) and 1 for the IPS neuron (C). After Nieder, A., Freedman, D. J., &Miller, E. K. (2002). Representation of the quantity of visual items in the primate prefrontal cortex. Science,297, 1708–1711. 386

NEURAL BASES OF NUMERICAL COGNITION 387neurons were sparsely distributed in several areas, In addition to the previously described neu-but relatively abundant in the fundus of the intra- rons selective to numerosity in multiple-dotparietal sulcus (IPS), termed the ventral patterns, roughly 25% of the neurons in theintraparietal area (VIP) (Colby et al., 1993). fundus of the IPS also encoded sequentiallyThere were few such cells in the anterior inferior presented numerical quantity (Fig. 20.2B).temporal cortex (aITC) (Nieder & Miller 2004a) However, numerical quantity was represented(Fig. 20.1B). by distinct populations of neurons during the Item numbers can be determined in two fun- ongoing spatial or temporal enumerationdamentally different spatio-temporal presenta- process (i.e., in the sample phase); cellstion formats. When presented simultaneously as encoding the number of sequential itemsin multiple-item patterns, numerosity can be were not tuned to numerosity in multiple-estimated at a single glance in a direct, percep- item displays, and vice versa. Once the enu-tual-like way from a spatial arrangement. On a meration process was completed, though, andbehavioral level, constant reaction times and the monkeys had to store information inequal numbers of scanning eye movements to mind, a third population of neurons codedindividual items (Nieder & Miller, 2004b) indi- numerosity both in the sequential and simul-cate parallel processing mechanisms for quantity taneous protocol; about 20% of the cells wereassessments from multiple-dot patterns. tuned to numerosity irrespective of whetherMoreover, the response latencies of single neu- is was cued simultaneously or in sequencerons are indifferent across numerosities (Nieder (Fig. 20.2C–E). This argues for segregatedet al., 2002). In contrast to a simultaneous pre- processing of numerosity during the actualsentation, the elements of a set can be presented encoding stage in the parietal lobe, but alsoone by one and, thus, need to be enumerated for a final convergence of the segregatedsuccessively across time (Cordes et al., 2001; information to form most abstract quantityMeck & Church, 1983; Whalen et al., 1999). representations. The intermediate numerositySequential enumeration is cognitively more of an ongoing quantification process and thedemanding; it incorporates multiple encoding, storage of the final cardinality are accom-memory, and updating stages; it may even be plished by different neuronal populations.regarded as a form of addition of one. Sequential In another domain, cells in the superior parietalenumeration is particularly interesting in that it lobule (SPL) have been reported to keep track ofconstitutes a nonverbal precursor of real the number of movements (Sawamura et al.,counting; after all, verbal counting is a sequential 2002). The authors trained monkeys to alternateenumeration process using number symbols between five arm movements of one type (‘‘push’’(i.e., 1-2-3, etc.). and ‘‘turn’’) and five of another. They found neu- To address the neuronal representation of rons in a somatosensory-responsive region (part ofan abstract counting-like accumulation of area 5) of the SPL that maintained the number ofsensory events and to compare it to the movements. Relatively few such neurons wereencoding of numerosity in simultaneous dis- found in the same lateral PFC regions whereplays, Nieder and colleagues (2006) recorded other perceptual categories were investigated.single-cell activity in the fundus of the IPS One possibility for the difference between thesewhile monkeys performed a delayed match- studies may be modality (touch vs. vision), butto-sample task in which sample numerosity another may be the level of abstraction. Mostwas specified either by single dots appearing movement-number representations found byone by one to indicate the number of items Sawamura and colleagues (85%) were notin sequence (‘‘sequential protocol,’’ Fig. abstract; number-selective activity depended20.2A) or by multiple-dot patterns (‘‘simul- on whether the monkey’s movement wastaneous protocol,’’ Fig. 20.1A). It was ‘‘push’’ or ‘‘turn.’’ By contrast, the visualensured that temporal or spatial cues could numerosity representations found in the PFCnot be used by the animals to solve the task. and fundus of the IPS were abstract and

388 PRIMATE NEUROETHOLOGYgeneralized (Nieder & Miller, 2004a; Nieder et selective than PFC neurons (Nieder & Miller,al., 2002, 2006). 2004a). As PPC and PFC are functionally inter- The parietal cortex, and the IPS in particular, connected (Chafee & Goldman-Rakic 2000;might be the first cortical stage that extracts visual Quintana & Fuster, 1999), that informationnumerical information because its neurons might be conveyed directly or indirectly to therequire shorter latencies to become numerosity PFC, where it is amplified and maintained to gainFigure 20.2 Coding of sequentially presented numerosity. (A) Sequential delayed match-to-numerositytask (here for numerosity 3). The sample numerosity was cued by sequentially presented items temporallyseparated by pauses containing no items. The temporal succession and duration of individual items werevaried within and across quantities. (B) Responses of an example neuron selective to the sequentialquantity 2 (only one condition shown for clarity). Top panel illustrates the temporal succession ofindividual items (square pulses represent single items). The corresponding latency-corrected dischargesfor many repetitions of the protocol are plotted as averaged spike density functions. The first 500 msrepresent the fixation period. Corresponding colors were used for the stimulation illustration and theplotting of the neural data. Gray shaded areas denote item presentation. The inset shows the tuningfunctions of the neuron to the standard and a control protocol (error bars represent SEM) for foursequential dots. In both protocols, the neuron was tuned to numerosity 2. (C–E) Spike-densityhistograms of a single intraparietal sulcus (IPS) neuron showing remarkably similar delay activity in thesequential (C) versus simultaneous (D) presentation protocol, with 3 as the preferred numerosity. Thisneuron was not numerosity selective in the sample period. (E) Tuning functions of the neuron in C and Dcalculated for the delay period. After Nieder, A., Diester, I., & Tudusciuc, O. (2006). Temporal and spatialenumeration processes in the primate parietal cortex. Science, 313, 1431–1435.

NEURAL BASES OF NUMERICAL COGNITION 389Figure 20.2 (Continued)control over behavior. As classical association neurons’ sequentially arranged overlappingcortices, the prefrontal and posterior parietal cor- tuning curves preserved an inherent order oftices are ideal brain structures for an abstract cardinalities. This is important because numeros-encoding of quantity. They receive highly pro- ities are not isolated categories, but exist in rela-cessed multimodal input (Bremmer et al., 2001; tion to one another (e.g., 3 is greater than 2 andDuhamel et al., 1998; Lewis & Van Essen, 2000; less than 4); they need to be sequentially orderedMiller & Cohen, 2001)—a prerequisite for to allow meaningful quantity assignments.numerical competence because the number con- The response properties of numerosity-selec-cept applies equally well to all sensory modalities. tive cortical cells can explain basic psychophy-Both are cardinal processing stages for executive sical phenomena in monkeys, such as thefunctions (e.g., categorization, working memory, numerical distance and size effect (Fig. 20.3).decision making, goal-directed behavior, etc.) The numerical distance effect states that it isand play an important role in maintaining infor- easier to discriminate quantities that aremation ‘‘online’’ (Freedman et al., 2001; Stoet & numerically remote from each other (say, 2 vs.Snyder, 2004; Wallis et al., 2001), and the PPC in 6 is easier than 5 vs. 6), while the numerical sizeparticular hosts neural circuitry dedicated to the effect captures the finding that pairs of numer-representation of abstract spatial information osities of a constant numerical distance are easier(Colby & Goldberg, 1999). to discriminate if the quantities are small (e.g., 2 vs. 3 is easier than 5 vs. 6). The numerical distance effect results from the fact that the neuralBehavioral Significance of Numerosity- filter functions that are engaged in the discrimi-Selective Neurons nation of adjacent numerosities heavily overlapThe activity of all numerosity-selective neurons, (Nieder & Miller, 2003). As a consequence, theeach tuned to a specific preferred numerosity, signal-to-noise ratio of the neural signal detec-formed a bank of overlapping numerosity filters tion process is low, and the monkeys make many(Fig. 20.3B), mirroring the animals’ behavioral errors. On the other hand, the filter functions ofperformance (Fig. 20.3A). Interestingly, the neurons that are tuned to remote numerosities

390 PRIMATE NEUROETHOLOGYFigure 20.3 Relation between monkey behavior and numerosity-selective neurons. (A) Behavioralnumerosity discrimination functions of two monkeys. The curves indicate whether they judged the firsttest stimulus as containing the same number of items as the sample display. The function peaks (and thecolor legend) indicate the sample numerosity for which each curve was derived. Behavioral filter functionsare skewed on a linear scale (left), but symmetric on a logarithmic scale (right). (B) The averaged single-cellnumerosity-tuning functions (from prefrontal cortex) are also asymmetric on a linear scale, but symmetricafter logarithmic transformation. After Nieder, A., & Miller, E. K. (2003). Coding of cognitive magnitude:Compressed scaling of numerical information in the primate prefrontal cortex. Neuron, 37, 149–157.barely overlap, which results in a high signal-to- An important piece of evidence for the con-noise ratio and, therefore, good performance in tribution of numerosity-selective neurons tocases where the animal has to discriminate sets of behavioral performance came from the examina-a larger numerical distance. The behavioral con- tion of error trials. When the monkeys madesequences of the numerical size effect are there- judgment errors, the neural activity for the pre-fore in accordance with Weber’s Law. ferred quantity was significantly reduced as com- The numerical size effect is directly related to pared to correct trials (Nieder & Merten, 2007;the precision of the neuronal numerosity filters: Nieder & Miller, 2004a; Nieder et al., 2002, 2006).The widths of the tuning curves (or neuronal As a result of this (and the ordered representationnumerical representations) increase linearly with of quantity), the activity to a given preferredpreferred numerosities (i.e., on average, tuning numerosity on error trials was more similar toprecision deteriorates as the preferred quantity that elicited by adjacent nonpreferred quantitiesincreases). Hence, more selective neural filters on correct trials. In other words, if the neuronsthat do not overlap extensively are engaged if a did not encode the numerosity properly, themonkey has to discriminate small numerosities monkeys were prone to mistakes.(say, 1 and 2), which results in high signal-to-noise ratios and few errors in the discrimination. Implementing Numerosity DetectorsConversely, if a monkey has to discriminate large How may numerosity-selective neurons tunednumerosities (such as 4 and 5), the filter functions to preferred numerosities arise in the course ofwould overlap considerably. Therefore, the dis- cortical processing? Purely sensory, nonnume-crimination has a low signal-to-noise ratio, which rical properties (such as binocular disparity,leads to poor performance. wavelength, and contrast in the visual system)

NEURAL BASES OF NUMERICAL COGNITION 391are encoded already at the earliest processing architecture was proposed by Verguts and Fiasstages of the sensory epithelia (Van Essen & (2004) using a backpropagation network.DeYoe, 1993). Number, on the other hand, is a Interestingly, summation units developed spon-most abstract category devoid of specific sensory taneously in the second processing stage (thefeatures; two cats and two calls have nothing in ‘‘hidden units’’) after tuned numerosity detectorscommon, except that the size of their sets is were determined at the output stage.‘‘two.’’ How, then, may the cardinality of objects Even though numerosity representationsor events, the pure number of entities, be derived derived with both models are noisy (approxi-in terms of neuronal information processing? mate) and obey Weber’s Law, the two models Two main models have been proposed to differ in important aspects. The mode-controlexplain the implementation of quantity infor- model by Meck and Church (1983) operatesmation. The mode-control model by Meck and serially and assumes representation of cardin-Church (1983) works in series and suggests that ality on a linear scale, whereas in the neuraleach item is encoded by an impulse from a pace- filtering model by Dehaene and Changeuxmaker, which is added to an accumulator (1993), numerosity is encoded in parallel and(Fig. 20.4A). The magnitude in the accumulator represented on a logarithmic scale (the sameat the end of the count is then read into memory, holds for the backpropagation model byforming a representation of the number of a set. Verguts and Fias [2004]). Both models, however,Thus, it is assumed that quantity is encoded by have summation units implemented that accu-‘‘summation coding’’ (i.e., the monotonically mulate number in a graded fashion prior toincreasing and decreasing response functions of feeding into numerosity detectors at thethe neurons; see also network model by Zorzi & output. As a putative physiological reflection ofButterworth, 1999; Zorzi et al., 2005). this computational stage in models of number Another model, the neural filtering model by processing, Roitman and colleagues (2007)Dehaene and Changeux (1993), implements recorded neurons in the parietal lobe whosenumerosity in parallel (Fig. 20.4B). First, each responses resembled the output of accumulator(visual) stimulus is coded as a local Gaussian neurons with response functions that systemati-distribution of activation by topographically cally increased or decreased with an increase oforganized input clusters (simulating the retina). stimulus set size (Fig. 20.4C,D).Next, items of different sizes are normalized to a In this study, the activity of single neurons wassize-independent code. At that stage, item size, recorded in the lateral intraparietal (LIP) area ofwhich was initially coded by the number of active monkeys performing a delayed saccade taskclusters on the retina (quantity code), is now (Roitman et al., 2007). On each trial, monkeysencoded by the position of active clusters on a maintained fixation on a central point while alocation map (position code). Clusters in the saccade target was placed at a random locationlocation map project to every unit of downstream lateral to the fixation spot. When the fixation‘‘summation clusters,’’ whose thresholds increase point was extinguished, the monkey shifted itswith increasing number and pool the total activity gaze to the saccade target to receive a fluidof the location map. The summation clusters reward. Prior to the monkey performing the sac-finally project to ‘‘numerosity clusters.’’ cade toward the target, a set of items (2, 4, 8, 16, orNumerosity clusters are characterized by central 32 dots) at the location of the recorded cell’s recep-excitation and lateral inhibition so that each tive field informed the monkey about the amountnumerosity cluster responds only to a selected of reward it would receive after a correct saccade.range of values of the total normalized activity One of the five possible numerosities was selected(i.e., their preferred numerosity). Since the as the standard, which was then presented in halfnumerosity of a stimulus is encoded by peaked of the trials. In such standard trials, the animaltuning functions with a preferred numerosity received a fixed amount of fluid as reward. In each(causing maximum discharge), this mechanism of the remaining trials, one of the four deviantis termed ‘‘labeled-line code.’’ A similar numerosities was shown, resulting in a slightly

392 PRIMATE NEUROETHOLOGYFigure 20.4 Implementation of numerosity detectors. (A) Mode-control model. After Meck, W. H., &Church, R. M. (1983). A mode control model of counting and timing processes. Journal of ExperimentalPsychology: Animal Behavioral Processes, 9, 320–334. (B) Neural filtering model. After Dehaene, S., &Changeux, J. P. (1993). Development of elementary numerical abilities: A neural model. Journal of CognitiveNeuroscience, 5, 390—407. (C, D) Neurons in lateral intraparietal (LIP) area discharge monotonically as afunction of set size during an implicit numerosity task. Two single cells are depicted that show an increase (C)or decrease (D) of discharge rate, respectively, with increasing numerosity. Each neuron was tested withdifferent standard (color code) and deviant numerosities (see text for explanation of the task). From Roitman,J. D., Brannon, E. M., & Platt, M. L. (2007). Monotonic coding of numerosity in macaque lateral intraparietalarea. PLoS Biology, 8, e208. (E–H) Prefrontal cortex neurons tuned to preferred numerosities in monkeysperforming a delayed match-to-numerosity task. Preferred numerosity was 2 (E), 4 (F), 6 (G), and 20 (H).From Nieder, A., & Merten, K. (2007). A labeled-line code for small and large numerosities in the monkeyprefrontal cortex. Journal of Neuroscience, 27, 5986–5993.larger amount of fluid as reward. This encouraged stimulus features or cognitive demands).the monkey to pay attention to the numerosities, Thus, a population of neurons in the LIP areaeven though they were not to be discriminated or encoded the number of elements in a visualotherwise used in the task. array in a roughly monotonic manner, similar Roitman and coworkers (2007) found that to neurons representing sensory magnitudethe activity of most LIP neurons increased or (Brody et al., 2003; Romo et al., 1999). Thedecreased systematically with increasing authors suggested that these two classes ofnumber of elements during stimulus presenta- number-selective neurons may be the physio-tion (Fig. 20.4C,D) (irrespective of other logical instantiation of the summation units

NEURAL BASES OF NUMERICAL COGNITION 393and numerosity units proposed in neural net- stimulus intensity (I) multiplied by a modality-work models of numerical representation; and dimension-specific constant (k). If the tuningmonotonic magnitude coding of LIP neurons functions for behavioral discrimination andmay provide input to neurons in the PPC and single units are regarded as the monkeys’ beha-PFC that compute cardinal numerical represen- vioral and neural numerical representationstations via tuning to preferred numerosities (Fig. 20.3), the crucial question then concerns(Fig. 20.4E–H) (Nieder & Merten, 2007; which scaling scheme provides symmetric (i.e.,Nieder & Miller, 2004a; Nieder et al., 2002, Gaussian) probability density distributions. Both2006; Piazza et al., 2004). the performance and the single-unit data for In agreement with this hypothesis, monotonic numerosity judgments are better described by aneurons described in the LIP area would operate compressed, as opposed to a linear, scale (Niederon an intermediate level of numerosity detection. & Miller, 2003). Therefore, single-neuron repre-The numerosity stimuli used by Roitman and sentations of numerical quantity in monkeyscoworkers (2007) were carefully placed over the obey Fechner’s Law.spatially confined response fields of LIP neuronsand thus allowed to address local information Coding of Continuous and Discreteprocessing. Final processing stages of abstract Quantitynumerical information, however, are required tointegrate across time, space, and modality. Area Quantitative knowledge, such as understandingVIP and the PFC are ideal candidate structures for how much drinking water is available (Vana global representation of numerosity; both areas Marle et al., 2006; Woodruff et al., 1978) orintegrate multimodal input (Duhamel et al., how many individuals belong to an opponent1998), and neurons in PFC in particular exhibit party (McComb et al., 1994; Wilson et al., 2001),global cognitive processing properties that are no guide vital decisions in the life of animals andlonger spatially restricted (Everling et al., 2002). humans alike. Functional imaging studies inBehavioral relevance is another aspect that could humans suggest that anatomical vicinityhave a substantial effect on the coding scheme. All (Castelli et al., 2006; Fias et al., 2003; Pinel etstudies that required the monkeys to use cardinal al., 2004) or even a common magnitude systemnumerical information explicitly to solve a task (Walsh, 2003) for the representation of numer-found a labeled-line code, irrespective of stimulus ical (discrete) and spatial (continuous) quantitymodality, presentation format, and recording site in the parietal cortex might be responsible for(Nieder et al., 2002, 2006; Sawamura et al., 2002), behavioral interference phenomena betweenwhereas numerosity was implicitly informative in numerical and spatial quantity. In a numberthe delayed saccade task (Roitman et al., 2007) but comparison task, for example, choosing thehad no impact on the monkeys’ task performance. numerically larger number takes significantlyThus, the neuronal representation may change if longer if the numeral is physically smaller inquantity is encoded as an explicit category. size compared to the numerically smaller number (e.g., in the comparison 2 versus 7) (Henik & Tzelgov, 1982; Pinel et al., 2004).The Scaling of Numerical Representations To investigate how continuous quantity isAs mentioned previously, the neurons’ overlap- encoded by single nerve cells and how it relatesping tuning curves are ordered along a ‘‘number to numerosity representations, Tudusciuc andline’’ (Fig. 20.3B). But what is the scaling scheme Nieder (2007) trained two rhesus monkeys in aof such a ‘‘number line’’? Are neuronal numerical delayed match-to-sample task to discriminate dif-representations best described on a linear or a ferent types of quantity randomly alternatingnonlinear, possibly logarithmically compressed within each session. In the ‘‘length protocol,’’ thescale? The latter would be predicted if Fechner’s length of a line (out of four different lengths)Law holds. Fechner’s Law states that the perceived needed to be discriminated (continuous-spatialmagnitude (S) is a logarithmic function of quantity). In the ‘‘numerosity protocol’’

394 PRIMATE NEUROETHOLOGY(Fig. 20.1A), the number of (one to four) items in After the monkeys solved more than 81% ofmultiple-dot displays (discrete-numerical the trials correctly for both the length and thequantity) was the relevant stimulus dimension. numerosity protocols, single-unit activity fromTo ensure that the monkeys solved the task based the depth of the IPS was analyzed while theon the relevant quantitative information, other animals performed the task. About 20 % of ana-covarying features of the stimuli were again con- tomically intermingled single neurons in thetrolled, and the positions of the dots and lines were monkey IPS each encoded discrete-numericalgreatly varied. (Fig. 20.5A), continuous-spatial (Fig. 20.5B), orFigure 20.5 Neuronal coding of continuous and discrete quantity. (A–C) Three example neurons exhibitingselectivity for quantity. Top panels in A illustrate the four different numerosities (left) and four different linelengths (right) used as stimuli. Left and right graphs illustrate the discharge rates (displayed as smoothed spike-density histograms) of the same neuron in the numerosity and length protocol, respectively. The first 500 msrepresent the fixation period. The area between the two black vertical bars represents the sample presentation;the following 1,000 ms indicate the delay phase. Colors correspond to the quantity dimensions. The insetsbetween two histograms depict the tuning functions of each of the three neurons to numerosity and length. (A)Neuron tuned to numerosity 3, but not to length. (B) Neuron tuned to the third longest line, but not to anytested numerosity. (C) Neuron encoding both discrete and continuous quantity. (D, E) Classificationperformance in the sample period across the neuronal population. Confusion matrices describing the patternof quantity classification performed on 72 quantity-selective neurons (D) and 72 nonselective (untuned)neurons (E). The rows in each confusion matrix represent the true classes the monkey had seen; the columnscorrespond to the output of the classifier. Color codes the classification probability. The eight classes correspondto the eight stimulus quantities: numerosity 1 to 4 and line length 1 to line length 4, where length 1 is the shortestline. Thus, the main diagonal shows how often the classifier correctly assigned quantity stimuli to their realcategory (i.e., a measure of accuracy). Averaging the classification probabilities over each diagonal parallel to themain diagonal results in the average performance of the classifier as a function of distance from the actualquantity, which is plotted as a tuning function at the top left (discrete) and bottom right (continuous) of eachmatrix. From Tudusciuc, O., & Nieder, A. (2007). Neuronal population coding of continuous and discretequantity in the primate posterior parietal cortex. Proceedings of the National Academy of Sciences of the USA, 104,14513–14518.

NEURAL BASES OF NUMERICAL COGNITION 395Figure 20.5 (Continued)both types of quantities (Fig. 20.5C), suggesting the same pool of neurons (i.e., from data it hadthat two partly overlapping populations of neu- not used for learning).rons within this area may give rise to a general- The quantitative results based on the statisticalized magnitude system (Walsh, 2003). classifier demonstrated that the small population Can functionally overlapping groups of par- of quantity-selective neurons carried most of theietal neurons provide sufficient information for categorical information (Fig. 20.5D); bythe monkey to make correct quantity judgments? exploiting the classical spike-rate measure thatTo assess the discriminative power of this small contributes to the monkeys’ quantity discrimina-population of neurons, Tudusciuc and Nieder tion performance (Nieder & Miller, 2004a; Nieder(2007) applied a population decoding technique et al., 2002, 2006), the classifier was able to accu-(Averbeck et al., 2006; Hung et al., 2005; Laubach, rately and robustly discriminate both continuous2004; Laubach et al., 2008) based on an artificial and discrete quantity classes. Interestingly, evenneuronal network (Kohonen, 1997). The classifier the population of untuned neurons had a remark-was trained with neuronal responses (i.e., prepro- able and significant quantity coding ability, albeitcessed spike trains) of a set of neurons recorded to a lesser extent than the tuned neurons (Fig.while the monkeys judged each of the eight quan- 20.5E). This suggests that the classifier extracted,tity categories; at this stage, the classifier was beyond the averaged spike rate, additional infor-informed about the stimulus configuration and mation from the temporal structure of the neu-learned the neuronal features that were best ronal responses. Moreover, the comparisonsuited for identifying a given category. In the between the monkeys’ neuronal and behavioralsubsequent test phase, the classifier predicted responses showed that the brain indeed utilizesthe categories from novel neuronal responses of this information for decision making; neuronal

396 PRIMATE NEUROETHOLOGYresponses recorded whenever the monkeys failed childhood. During this learning process, and asto discriminate the quantity categories prevented a prerequisite for the utilization of signs asthe classifier from predicting the correct quantity numerical symbols, long-term associationscategory. Future studies need to clarify whether between initially meaningless shapes (that latercomplex quantity judgments require an interplay become numerals) and inherent semantic numer-with, or readout by, structures with more execu- ical categories must be established. This necessarytive functions, such as the prefrontal cortex but by no means sufficient step toward the utili-(Miller & Cohen, 2001). zation of number symbols in humans can also be mastered by different animal species (Boysen & Berntson, 1989; Matsuzawa, 1985; Washburn &Toward Symbolic Number Representations Rumbaugh, 1991; Xia et al., 2001).As shown previously, humans and animals share To investigate the single-neuron mechanismsan evolutionarily old quantity representation of such semantic associations, Diester and Niedersystem that allows the estimation of set sizes. (2007) mimicked a semantic mapping process byNonverbal numerical cognition, however, training two monkeys to associate the a prioriis limited to approximate quantity representa- meaningless visual shapes of Arabic numeralstions and rudimentary arithmetic operations. (that became ‘‘signs’’ or, more precisely,Language-endowed humans, on the other ‘‘indices’’; Wiese, 2003) with the inherentlyhand, invented number symbols (numerals and meaningful numerosity of multiple-dot displays.number words) during cultural evolution. These After this long-term learning process was com-mental tools enable us to create precise quantity pleted, a relatively large proportion of PFC neu-representations and perform exact calculation rons (24%) encoded plain numerical values,that is beyond the reach of any animal species. irrespective of whether they had been presented Even though number symbols are of para- as a specific number of dots or as a visual signmount importance in today’s scientifically and (Fig. 20.6A,B). Such ‘‘association neurons’’technologically advanced culture, their inven- showed similar tuning during the course of thetion dates back only a couple thousand years trial to both the direct numerosity in dot stimuli(Ifrah, 2000). Given the time scale of brain evo- and the associated numerical values of signs (Fig.lution, a de novo development of brain areas 20.6C). Interestingly, the tuning functions ofwith distinct, culturally dependent number association neurons showed a distance effect forsymbol functions is more than unlikely both protocols (i.e., a dropoff of activity with(Dehaene, 2005). Rather, it is conceivable that increasing numerical distance from the preferredbrain structures that originally evolved for other numerical value; Fig. 20.6D). This distance effectpurposes are built upon in the course of conti- found in the shape protocol indicates that asso-nuing evolutionary development (Gould, 1982). ciation neurons responded as a function ofAccording to the ‘‘redeployment hypothesis’’ numerical value rather than visual shape per se.(Anderson, 2007) or ‘‘recycling hypothesis’’ Most cells coded the (direct and associated)(Dehaene, 2005; Dehaene & Cohen, 2007), numerical values during specific time phases inalready existing simpler cell assemblies are largely the trial (e.g., only at sample onset or toward thepreserved, extended, and combined as networks end of the delay period). The neuronal popula-become more complex (Sporns & Kotter, 2004). tion as a whole, however, represented the numer- In the number domain, existing neuronal ical association throughout the entire trial andcomponents in PFC and IPS subserving non- thus provided crucial associative informationverbal quantity representations could be used over time. The activity of association neuronsfor the new purpose of number symbol encoding, predicted the monkeys’ judgement performance;without disrupting their participation in existing if the monkeys failed to match the correctcognitive processes (Piazza et al., 2007). Guided number of dots to the learned signs, theby the faculty of language, children learn to use tuning behavior of a given neuron to numer-number symbols as mental tools during osities and their associated visual shapes were

NEURAL BASES OF NUMERICAL COGNITION 397Figure 20.6 Semantic associations between signs and numerical categories by single neurons. (A, B) Theresponses of the same single prefrontal cortex (PFC) neuron to both direct numerosities and associatednumeral shapes (top panels in A and B illustrate sample stimuli). Neuronal responses in A and B areshown as smoothed spike-density histograms (colors denote discharge to the corresponding samplenumerical value 1 to 4). The first 500 ms indicate the fixation period. Black vertical lines mark sampleonset (500 ms) and offset (1,300 ms). This neuron’s preferred numerical value in the sample and delayperiod was 4. Note the similarity in the association neuron’s temporal discharge profiles in response to themultiple-dot displays and the shape of Arabic numerals. (C) Time course of original cross-correlationcoefficients (red) and chance cross-correlation coefficients (shuffle predictor, blue). The intervalbordered by vertical dotted lines indicates the time phase of significant cross-correlation (asdetermined by measures from signal detection theory) between tuning to numerical values in themultiple-dot displays and Arabic numerals; in this period, the neuron associated numerical values inthe two protocols. (D) Tuning functions to numerical values in the multiple-dot displays and Arabicnumerals. (E) Lateral view of a monkey brain indicating the recording sites in the PFC and intraparietalsulcus (IPS). Proportions of association neurons in the PFC and IPS are displayed as pie charts. AfterDiester, I., & Nieder, A. (2007). Semantic associations between signs and numerical categories in theprefrontal cortex. PLoS Biology, 5, e294.severely disrupted. These findings argue for of all recorded IPS neurons associated signs withassociation neurons as a neuronal substrate numerosities (Fig. 20.6E). Moreover, the qualityfor the semantic mapping processes between of neuronal association in the IPS was weak andsigns and categories. occurred much later during the trial. In the same study (Diester & Nieder, 2007), The conclusion drawn from these results isthe activity of neurons in the fundus of the IPS that even though monkeys use the PFC andwas also recorded. In contrast to PFC, only 2% IPS for nonsymbolic quantity representations,

398 PRIMATE NEUROETHOLOGYonly the prefrontal part of this network is NUMERICAL RANKengaged in semantic shape-number associations. (ORDINALITY)Interest-ingly, this neural pattern seems to bepreserved in children (Ansari et al., 2005; The frontal lobe is an ideal region in the brain toKaufmann et al., 2006; Rivera et al., 2005). In encode both sensory object properties and rank-contrast to adults, preschoolars lacking ample order information because PFC receives massiveexposure to number symbols show elevated sensory input from the temporal and parietalPFC activity when dealing with ymbolic cardin- lobes and projects to premotor and motoralities. With age and proficiency, however, the areas of the frontal lobe (Miller & Cohen,activation seems to shift to parietal areas. The 2001). Thus, neurons that encode the ordinalPFC could thus be ontogenetically and phylo- position of task-related hand or eye movementsgenetically the first cortical area establishing have been found frequently in prefrontalsemantic associations, which might be relocated (Funahashi et al., 1997) and a subset of motor-to the parietal cortex in human adolescents in related cortical areas in trained monkeys. Josephparallel with maturing language capabilities that and coworkers (Barone & Joseph, 1989; Kermadiendow our species with a sophisticated symbolic & Joseph, 1995; Procyk & Joseph, 2001; Procyksystem (Deacon, 1997). et al., 2000) identified order-selective neurons in The prefrontal region is strategically situated the frontal eye field (FEF), caudate nucleus, andto establish semantic associations (Miller & anterior cingulate cortex of monkeys that hadCohen, 2001); it receives input from both the been trained to sequentially order spatiallyanterior inferotemporal cortex encoding shape arranged items. These neurons were only activeinformation (Tanaka, 1996) as well as the when the monkeys reached for the first, second,posterior parietal cortex that contains numer- or third target, irrespective of the targets’ loca-osity-selective neurons (Nieder & Miller, 2004a; tion and the precise type of hand movement.Nieder et al., 2006). Previous studies showed Clower and Alexander (1998) trained a monkeythat neurons in the PFC encode learned associa- to position a cursor on a video display bytions between two purely sensory stimuli moving a joystick clockwise or counterclockwisewithout intrinsic meaning (e.g., the association along a spatially arranged four-item path. In theof a ertain color with a specific sound, or pairs of presupplementary motor areas (pre-SMAs),pictures) (Fuster et al., 2000; Miller et al., 1996; more than two thirds of the recorded neuronsRainer et al., 1999). In the anterior inferotem- showed significant effects of numerical order,poral cortex, Miyashita and coworkers found but only about one third of the neurons dis-‘‘pair-coding neurons’’ that responded to played an effect of rank order in the supplemen-arbitrary pairs of images monkeys learned to tary motor area (SMA). Rank-order selectivitymatch in a pair-association task (Sakai & was also identified in the pre-SMA of monkeysMiyashita, 1991); the same group found evi- that had been trained to sequentially performdence that the PFC is important for active three different hand movements (‘‘push,’’retrieval of these associative representations ‘‘pull,’’ or ‘‘turn’’) in four to six different(Tomita et al., 1999). Our findings (Diester & orders separated by waiting times (Shima &Nieder, 2007) demonstrate that neurons in the Tanji, 2000; Tanji & Shima, 1994). In the pre-PFC represent semantic long-term associations SMA, the activity differed selectively in the pro-not only between pairs of pictures but also cess of preparing the first, second, or thirdbetween arbitrary shapes and systematically movements in individual trials. The SMA, onarranged categories with inherent meaning the other hand, was more involved in linking(i.e., the ordered cardinalities of sets). In that the occurrence of two different movements and,respect, the PFC of primates may not only con- therefore, in determining the order of the com-trol the retrieval of long-term associations but ponent movements in the sequence (relationalalso may in fact onstitute a crucial processing order). Ordinal position of movements instage for abstract semantic associations. the pre-SMA seems to be encoded in an

NEURAL BASES OF NUMERICAL COGNITION 399effector-independent manner as in the case of eye recall of the temporal order of events and stimulimovement sequences (Isoda & Tanji, 2004). In (Petrides, 1995). In two recent elegant studiestwo motor areas that are specialized in processing (Ninokura et al., 2003, 2004), the single-neuroneye movements, activity that reflected saccade correlate of temporal rank order information insequence or the numerical position of a saccade visual lists was addressed. Monkeys werewithin a sequence (rank) was more common in trained to observe and remember the order inthe supplementary eye field (SEF), whereas which three visual objects appeared.activity that reflected saccade direction was Subsequently, the animals planned and exe-more dominant in the FEF (Isoda & Tanji, 2003). cuted a triple-reaching movement in the same Interestingly, encoding of numerical order order as previously seen (Fig. 20.7). Neurons inhas not only been observed in these premotor the ventrolateral PFC selectively encoded visualand supplementary motor areas but also in the object properties (26% of the total sample),primary motor cortex. Carpenter and collea- whereas neurons in the dorsolateral PFCgues (1999) showed monkeys five spatially (44%) were selectively tuned to the rank orderarranged visual targets that appeared succes- of the objects irrespective of the sensory prop-sively on a screen. After the target sequence erties of objects. For example, a rank-order–was complete, one of the items changed its selective neuron would be active whenever thecolor. The monkeys needed to memorize the second item of a shuffled lists appears. A thirdorder in which the targets appeared and point class of neurons (30%), found in the ventrolat-to the item that appeared just after the one that eral PFC, showed the most complex responses,had changed its color at the end of the list integrating the objects’ sensory and order infor-presentation. In approximately one third of mation. Such neurons would only dischargethe neurons recorded from the arm region of whenever a certain object appeared at a giventhe topographically organized primary motor position in the sequence. Similar results havecortex (M1), the ordinal position of the targets also been reported by Inoue and Mikamiwas the only factor that covaried with neuronal (2006).activity. Therefore, the motor cortex—an area The representational formats of nonverbalthat is traditionally regarded as purely motor serial-order information are still poorlyexecutive—also participates in the processing understood. However, the behavioral andof cognitive information about serial order neuronal data indicate an imprecisewithin the context of a motor task. The representation of discrete numerical rank,authors pointed out, however, that the motor which is reminiscent of the analog-magnitudecortex most likely is just one component in a mechanism that has been proposed for car-distributed network that encodes, stores, and dinality. To elaborate a computational modelrecalls a sequence. Motor-related areas such as of working memory for serial order, BotvinickM1, SMA, pre-SMA, and FEF may receive and Watanabe (2007) recently wove item,numerical information that has already been numerosity, and rank information together.computed in earlier stages of the cortical hier- Their network combined graded neuronalarchy to perform appropriate serial-order responses to different items (not yet verifiedactions. experimentally) and tuning functions for Ordinal categorization of items requires both sequential enumeration processes (Nieder etinformation about the rank of an item (e.g., al., 2006) with the data showing that neuronsbased on temporal order) and its identity. in the PFC code the rank of items within aNeuropsychological studies emphasize the sequence (Ninokura et al., 2004; Fig 20.7). Theimportance of the lateral prefrontal cortex in model’s output, a recalled multi-itemmaintaining temporal order information sequence, replicated many behavioral charac-(McAndrews & Milner, 1991; Milner, 1971). In teristics of working memory such as the pri-monkeys, lesioning the dorsolateral frontal macy effect (a recall advantage for initialcortex causes impairments in tasks that require items) and the recency effect (advantage for

400 PRIMATE NEUROETHOLOGYFigure 20.7 Temporal ordering task and single cell responses from the PFC. (A) Monkeys were requiredto observe and remember the order in which three visual objects appeared, so that the animals could plan asubsequent triple-reaching movement in the same order. (B) Two single neurons encoding the first (cell 1)and the second rank (cell 2), irrespective of the order in which the three items (symbolized by letters ABC)appeared. Neural responses are shown in a dot-raster histogram (top panels, each dot represents an actionpotential) and averaged as peri-stimulus time histograms (bottom panels). From Ninokura, Y., Mushiake,H., & Tanji, J. (2004). Integration of temporal order and object information in the monkey lateral prefrontalcortex. Journal of Neurophysiology, 91, 555–560.the last one or two items). Furthermore, chan- cognitive functions may exploit both quantityging the width of the model rank tuning curve and rank (Jacob & Nieder, 2008). It posits thatsimulated the developmental finding of working memory of ordered sequences at leastimproved recall accuracy with age. Thus, this in part relies on neuronal assemblies in themodel integrates across several neurophysio- parietal and prefrontal cortices that also con-logical studies to demonstrate how higher tribute to numerosity representations.

NEURAL BASES OF NUMERICAL COGNITION 401CONCLUSION Bremmer, F., Schlack, A., Shah, N. J., Zafiris, O., Kubischik, M., Hoffmann, K., et al. (2001).This chapter illustrates that nonverbal numerical Polymodal motion processing in posteriorrepresentations can engage a wide cortical net- parietal and premotor cortex: A humanwork, with the PFC and the IPS, in particular, as fMRI study strongly implies equivalencieskey structures. Neurons in these areas are char- between humans and monkeys. Neuron, 29,acterized by response properties that underlie 287–296.basic psychophysical phenomena during cardin- Brody, C. D., Hernandez, A., Zainos, A., & Romo,ality or serial-order judgments. So far, single-cell R. (2003). Timing and neural encoding ofstudies in monkeys have concentrated on either somatosensory parametric working memory in macaque prefrontal cortex. Cerebral Cortex, 13,cardinal or ordinal aspects, but never both. In 1196–1207.the human primate, a recent study showed that Carpenter, A. F., Georgopoulos, A. P., & Pellizzer, G.the cortical network for rank coincides with the (1999). Motor cortical encoding of serial order inareas processing numerical quantity (Fias et al., a context-recall task. Science, 283, 1752–1757.2007), even though this does not necessarily Castelli, F., Glaser, D. E., & Butterworth, B. (2006).mean that single neurons may encode both car- Discrete and analogue quantity processing indinality and ordinality. Based on the findings that the parietal lobe: A functional MRI study.discrete and continuous quantities and even Proceedings of the National Academy of Sciencessimultaneously and sequentially presented of the USA, 103, 4693–4698.numerosities are encoded by largely distinct neu- Chafee, M. V., & Goldman-Rakic, P. S. (2000).ronal populations in the IPS, a cellular segrega- Inactivation of parietal and prefrontal cortex reveals interdependence of neural activitytion also for cardinal and ordinal stimulus aspects during memory-guided saccades. Journal ofis likely. Single-cell electrophysiology is well Neurophysiology, 83, 1550–1566.poised to answer this and other important ques- Clower, W. T., & Alexander, G. E. (1998). Movementtions about the neural basis of numerical cogni- sequence-related activity reflecting numericaltion in the years to come. order of components in supplementary and presupplementary motor areas. Journal of Neurophysiology, 80, 1562–1566.REFERENCES Colby, C. L., Duhamel, J. R., & Goldberg, M. E. (1993). Ventral intraparietal area of theAnderson, M. L. (2007). Evolution of cognitive macaque – anatomical location and visual function via redeployment of brain areas. response properties. Journal of Neurophysiology, Neuroscientist, 13, 13–21. 69, 902–914.Ansari, D., Garcia, N., Lucas, E., Hamon, K., & Colby, C. L., & Goldberg, M. E. (1999). Space and Dhital, B. (2005). Neural correlates of attention in parietal cortex. Annual Review of symbolic number processing in children and Neuroscience, 22, 319–349. adults. Neuroreport, 16, 1769–1773. Cordes, S., Gelman, R., Gallistel, C. R., & Whalen, J.Averbeck, B. B., Sohn, J. W., & Lee, D. (2006). (2001). Variability signatures distinguish verbal Activity in prefrontal cortex during dynamic from nonverbal counting for both large and selection of action sequences. Nature small numbers. Psychonomic Bulletin and Neuroscience, 9, 276–282. Review, 8, 698–707.Barone, P., & Joseph, J. P. (1989). Prefrontal cortex Deacon, T. (1997). The symbolic species: The co- and spatial sequencing in macaque monkey. evolution of language and the human brain. Experimental Brain Research, 78, 447–464. London: Norton.Botvinick, M., & Watanabe, T. (2007). From Dehaene, S. (2005). Evolution of human numerosity to ordinal rank: A gain-field model cortical circuits for reading and arithmetic: of serial order representation in cortical working The ‘‘neuronal recycling’’ hypothesis. In: memory. Journal of Neuroscience, 27, 8636–8642. S. Dehaene, J. R. Duhamel, M. D. Hauser, &Boysen, S. T., & Berntson, G. G. (1989). Numerical G. Rizzolatti (Eds.), From monkey brain to competence in a chimpanzee (Pan troglodytes). human brain (pp. 133–157). Cambridge, Journal of Comparative Psychology, 103, 23–31. MA: MIT Press.

402 PRIMATE NEUROETHOLOGYDehaene, S., & Changeux, J. P. (1993). Ifrah, G. (2000). The universal history of numbers: Development of elementary numerical From prehistory to the invention of the computer abilities: A neural model. Journal of Cognitive (p. 633). New York: Wiley. Neuroscience, 5, 390–407. Inoue, M., & Mikami, A. (2006). Prefrontal activityDehaene, S., & Cohen, L. (2007). Cultural recycling during serial probe reproduction task: of cortical maps. Neuron, 56, 384–98. Encoding, mnemonic, and retrieval processes.Diester, I., & Nieder, A. (2007). Semantic Journal of Neurophysiology, 95, 1008–1041. associations between signs and numerical Isoda, M., & Tanji, J. (2003). Contrasting neuronal categories in the prefrontal cortex. PLoS activity in the supplementary and frontal eye Biology, 5, e294. fields during temporal organization ofDuhamel, J. R., Colby, C. L., & Goldberg, M. E. multiple saccades. Journal of Neurophysiology, (1998). Ventral intraparietal area of the 90, 3054–3065. macaque: Congruent visual and somatic Isoda, M., & Tanji, J. (2004). Participation of the response properties. Journal of primate presupplementary motor are in Neurophysiology, 79, 126–136. sequencing multiple saccades. Journal ofEverling, S., Tinsley, C. J., Gaffan, D., & Duncan, J. Neurophysiology, 92, 653–659. (2002). Filtering of neural signals by focused Jacob, S. N., & Nieder, A. (2008). The ABC of attention in the monkey prefrontal cortex. cardinal and ordinal number representations. Nature Neuroscience, 5, 671–676. Trends in Cognitive Science, 29,4652-4657.Fias, W., Lammertyn, J., Caessens, B., & Orban, G. Kaufmann, L., Koppelstaetter, F., Siedentopf, C., A. (2007). Processing of abstract ordinal Haala, I., Haberlandt, E., Zimmerhackl, L.B. et knowledge in the horizontal segment of the al. (2006). Neural correlates of the number-size intraparietal sulcus. Journal of Neuroscience, interference task in children. Neuroreport, 17, 27, 8952–8956. 587–591.Fias, W., Lammertyn, J., Reynvoet, B., Dupont, Kermadi, I., & Joseph, J. P. (1995). Activity in the P., & Orban, G. A. (2003). Parietal caudate nucleus of monkey during spatial representation of symbolic and nonsymbolic sequencing. Journal of Neurophysiology, 74, magnitude. Journal of Cognitive Neuroscience, 911–933. 15, 47–56. Kohonen, T. (1997). Self-organizing maps (2ndFreedman, D. J., Riesenhuber, M., Poggio, T., & ed.). Berlin: Springer-Verlag. Miller, E. K. (2001). Categorical representation Laubach, M. (2004). Wavelet-based processing of of visual stimuli in the primate prefrontal neuronal spike trains prior to discriminant cortex. Science, 291, 312–316. analysis. Journal of Neuroscience Methods, 134,Funahashi, S., Inoue, M., & Kubota, K. (1997). 159–168. Delay-period activity in the primate prefrontal Laubach, M., Narayanan, N. S., & Kimchi, E. Y. cortex encoding multiple spatial positions and (2008). Single-neuron and ensemble their order of presentation. Behavioral Brain contributions to decoding simultaneously Research, 84, 203–223. recorded spike trains. In: C. Holscher & M.Fuster, J. M., Bodner, M., & Kroger, J. K. (2000). Munk (Eds.), Information processing by Cross-modal and cross-temporal association neuronal populations. New York: Cambridge in neurons of frontal cortex. Nature, 405, University Press. 347–351. Lewis, J. W., & Van Essen, D. C. (2000).Gould, S. J. V. E. (1982). Exaptation: A missing Corticocortical connections of visual, term in the science of form. Paleobiology, 8, sensorimotor, and multimodal processing areas 4–15. in the parietal lobe of the macaque monkey.Henik, A., & Tzelgov, J. (1982). Is three greater than Journal of Comparative Neurology, 428, 112–37. five: The relation between physical and Matsuzawa, T. (1985). Use of numbers by a semantic size in comparison tasks. Memory chimpanzee. Nature, 315, 57–59. and Cognition, 10, 389–395. McAndrews, M. P., & Milner, B. (1991). TheHung, C. P., Kreiman, G., Poggio, T., & DiCarlo, J. J. frontal cortex and memory for temporal order. (2005). Fast readout of object identity from Neuropsychologia, 29, 849–859. macaque inferior temporal cortex. Science, 310, McComb, K., Packer, C., & Pusey, A. (1994). 863–866. Roaring and numerical assessment in contests

NEURAL BASES OF NUMERICAL COGNITION 403 between groups of female lions, Panthera leo. Petrides, M. (1995). Impairments on nonspatial Animal Behaviour, 47, 379–387. self-ordered and externally ordered workingMeck, W. H., & Church, R. M. (1983). A mode memory tasks after lesions of the mid-dorsal control model of counting and timing part of the lateral frontal cortex in the processes. Journal of Experimental Psychology: monkey. Journal of Neuroscience, 15, 359–375. Animal Behavioral Processes, 9, 320–334. Piazza, M., Izard, V., Pinel, P., Le Bihan, D., &Miller, E. K., & Cohen, J. D. (2001). An integrative Dehaene, S. (2004). Tuning curves for theory of prefrontal cortex function. Annual approximate numerosity in the human Review of Neuroscience, 24, 167–202. intraparietal sulcus. Neuron, 44, 547–555.Miller, E. K., Erickson, C. A., & Desimone, R. Piazza, M., Pinel, P., Le Bihan, D., & Dehaene, S. (1996). Neural mechanisms of visual (2007). A magnitude code common to working memory in prefrontal cortex of the numerosities and number symbols in human macaque. Journal of Neuroscience, 16, intraparietal cortex. Neuron, 53, 293–305. 5154–5167. Pinel, P., Piazza, M., Le Bihan, D., & Dehaene, S.Milner, B (1971). Interhemispheric differences (2004). Distributed and overlapping cerebral in the localization of psychological representations of number, size, and processes in man. British Medical Bulletin, luminance during comparative judgments. 27, 272–277. Neuron, 41, 983–993.Nieder, A., Diester, I., & Tudusciuc, O. (2006). Procyk, E., & Joseph, J. P. (2001). Characterization Temporal and spatial enumeration processes of serial order encoding in the monkey anterior in the primate parietal cortex. Science, 313, cingulate sulcus. European Journal of 1431–1435. Neuroscience, 14, 1041–1046.Nieder, A., Freedman, D. J., & Miller, E. K. (2002). Procyk, E., Tanaka, Y. L., & Joseph, J. P. (2000). Representation of the quantity of visual items in Anterior cingulate activity during routine and the primate prefrontal cortex. Science, 297, non-routine sequential behaviors in macaques 1708–1711. Nature Neuroscience, 3, 502–508.Nieder, A., & Merten, K. (2007). A labeled-line Quintana, J., Fuster, J. M., & Yajeya, J. (1989). Effects code for small and large numerosities in the of cooling parietal cortex on prefrontal units in monkey prefrontal cortex. Journal of delay tasks. Brain Research, 503, 100–110. Neuroscience, 27, 5986–5993. Rainer, G., Rao, S. C., & Miller, E. K. (1999).Nieder, A., & Miller, E. K. (2003). Coding of Prospective coding for objects in primate cognitive magnitude: Compressed scaling of prefrontal cortex. Journal of Neuroscience, 19, numerical information in the primate 5493–5505. prefrontal cortex. Neuron, 37, 149–157. Rivera, S. M., Reiss, A. L., Eckert, M. A., & Menon,Nieder, A., & Miller, E. K. (2004a). A parieto- V. (2005). Developmental changes in mental frontal network for visual numerical arithmetic: Evidence for increased functional information in the monkey. Proceedings of specialization in the left inferior parietal the National Academy of Sciences of the USA, cortex. Cerebral Cortex, 15, 1779–1790. 101, 7457–7462. Roitman, J. D., Brannon, E. M., & Platt, M. L.Nieder, A., & Miller, E. K. (2004b). Analog (2007). Monotonic coding of numerosity in numerical representations in rhesus monkeys: macaque lateral intraparietal area. PLoS Evidence for parallel processing. Journal of Biology, 8, e208. Cognitive Neuroscience, 16, 889–901. Romo, R., Brody, C. D., Hernandez, A., & Lemus,Ninokura, Y., Mushiake, H., & Tanji, J. (2003). L. (1999). Neuronal correlates of parametric Representation of the temporal order of working memory in the prefrontal cortex. visual objects in the primate lateral Nature, 399, 470–473. prefrontal cortex. Journal of Neurophysiology, Sakai, K., & Miyashita, Y. (1991). Neural 89, 2868–2873. organization for the long-term memory ofNinokura, Y., Mushiake, H., & Tanji, J. (2004). paired associates. Nature, 354, 152–155. Integration of temporal order and object Sawamura, H., Shima, K., & Tanji, J. (2002). information in the monkey lateral prefrontal Numerical representation for action in the cortex. Journal of Neurophysiology, 91, parietal cortex of the monkey. Nature, 415, 555–560. 918–922.

404 PRIMATE NEUROETHOLOGYShima, K., & Tanji, J. (2000). Neuronal activity in Wallis, J. D., Anderson, K. C., & Miller, E. K. the supplementary and presupplementary motor (2001). Single neurons in prefrontal cortex areas for temporal organization of multiple encode abstract rules. Nature, 411, 953–956. movements. Journal of Neurophysiology, 84, Walsh, V. (2003). A theory of magnitude: Common 2148–2160. cortical metrics of time, space and quantity.Sporns, O., & Kotter, R. (2004). Motifs in brain Trends in Cognitive Science, 7, 483–488. networks. PLoS Biology, 2, e369. Washburn, D. A., & Rumbaugh, D. M. (1991).Stoet, G., & Snyder, L. H. (2004). Single neurons in Ordinal judgments of numerical symbols by posterior parietal cortex of monkeys encode macaques (Macaca mulatta). Psychological cognitive set. Neuron, 42, 1003–1012. Science, 2, 190–193.Tanaka, K. (1996). Inferotemporal cortex and Whalen, J., Gallistel, C. R., & Gelman, R. (1999). object vision. Annual Review of Neuroscience, Nonverbal counting in humans: The 19, 109–139. psychophysics of number representation.Tanji, J., & Shima, K. (1994). Role for Psychological Science, 10, 130–137. supplementary motor area cells in planning Wiese, H. (2003). Numbers, language and the human several movements ahead. Nature, 371, 413–416. mind. New York: Cambridge University Press.Tomita, H., Ohbayashi, M., Nakahara, K., Wilson, M. L., Hauser, M. D., & Wrangham, R. W. Hasegawa, I., & Miyashita, Y. (1999). Top- (2001). Does participation in intergroup down signal from prefrontal cortex in conflict depend on numerical assessment, executive control of memory retrieval. Nature, range location, or rank for wild chimpanzees? 401, 699–703. Animal Behaviour, 61, 1203–1216.Tudusciuc, O., & Nieder, A. (2007). Neuronal Woodruff, G., Premack, D., & Kennel, K. (1978). population coding of continuous and discrete Conservation of liquid and solid quantity by the quantity in the primate posterior parietal chimpanzee. Science, 202, 991–994. cortex. Proceedings of the National Academy of Xia, L., Emmerton, J., Siemann, M., & Delius, J. D. Sciences of the USA, 104, 14513–14518. (2001). Pigeons (Columba livia) learn to linkVan Essen, D. C., & DeYoe, E. A. (1993). numerosities with symbols. Journal of Concurrent processing in the primate visual Comparative Psychology, 115, 83–91. cortex. In: M. S. Gazzaniga (Ed.), The cognitive Zorzi, M., & Butterworth, B. (1999). A neurosciences (pp. 383–400). Cambridge, MA: computational model of number comparison. MIT Press. In: M. Hahn & S. C. Stoness (Eds.), ProceedingsVan Marle, K., Aw, J., McCrink, K., & Santos, L. R. of the Twenty First Annual Conference of the (2006). How capuchin monkeys (Cebus apella) Cognitive Science Society (pp. 778–783). quantify objects and substances. Journal of Mahwah, NJ: Erlbaum. Comparative Psychology, 120, 416–26. ` Zorzi, M., Stoianov, I., & Umilta, C. (2005).Verguts, T., & Fias, W. (2004). Representation of Computational modeling of numerical number in animals and humans: A neural cognition. In: J. Campbell (Ed.), Handbook of model. Journal of Cognitive Neuroscience, 16, mathematical cognition (pp. 67–84). New York: 1493–1504. Psychology Press.

CHAPTER 21Executive Control CircuitsJonathan D. WallisPrimate behavior is complex, and as behavioral evolved to accomplish this. For example, condi-complexity increases so too must the complexity tioning enables an organism to appreciate itsof the systems responsible for controlling the sensory environment in terms of whether thebehavior. In this chapter, we examine the neuronal situation enhances or decreases the organism’ssystems that are responsible for this control. We survival prospects. This enables the organism tobegin by examining one of the essential features of approach and seek out beneficial situations andsuch systems. High-level or executive control avoid potentially harmful ones.operates on highly processed information that is Over the course of mammalian evolution,increasingly abstracted from the concrete world of however, behavioral repertoires became increas-specific sensory stimuli and motor responses. We ingly intricate. Animals began to learn morewill examine the evidence that the prefrontal about their environments than simply whethercortex, an area of the brain that has increased they were good or bad for survival, and thedramatically in size over the course of mammalian relationships that they formed became moreevolution, enables this abstraction. We will look at complex. For example, mammals can learn arbi-the various psychological constructs that rely on trary relationships between sensory stimuli andabstraction, including high-level rule use, strate- motor responses. A rat can learn to push a levergies, and task sets. We will then examine how this up when a chamber is brightly lit and press ithigh-level control is organized. We will see that down when the chamber is dimly lit (Passinghamstudies investigating how humans implement et al., 1988). Humans can learn to brake when theyhigh-level information suggest that there is a hier- see a red traffic light and continue driving whenarchical organization of such information within they see a green light. Although such learning isthe prefrontal cortex. We will then examine clearly useful for our morning commute, it sufferswhether there is evidence for such an organization from a number of disadvantages relative to morein the monkey prefrontal cortex. Finally, we will complex types of learning. First, it does not gen-examine how high-level information in the pre- eralize very well: Learning about red and greenfrontal cortex interacts with other regions of the traffic lights, for example, tells you nothing aboutbrain to control behavior. the red and green lights on your stereo. A second problem is that we need to learn these stimulus- response associations through trial and error. SuchCOMPLEX BEHAVIOR AND learning necessarily involves errors, which canABSTRACTION be costly. Errors may result in the lost opportu-The ultimate goal of behavior is simple: Ensure nity for reward, or even physical harm. A thirdthe continued existence of the organism long problem is that by dealing with the world in aenough to permit reproduction and survival of literal fashion, we are potentially encoding it inone’s offspring. Relatively simple behaviors an inefficient manner. There are so many potential 405

406 PRIMATE NEUROETHOLOGYcombinations between stimuli and responses that (Herman & Gordon, 1974), and sea lions (Kastakwe would be unable to remember all the possible & Schusterman, 1994). The ability to use abstractcombinations and their meanings. rules is not limited to mammals. Corvids (the bird One solution to these problems resides in the family that includes crows, rooks, jays, and jack-ability to abstract information across experi- daws) and parrots all show the ability to performences. We learn to attend to the commonalities the matching-to-sample task with novel stimuliof a situation and ignore trivial differences. For (Pepperberg, 1987; Wilson et al., 1985). In con-example, after dining in a few restaurants, we trast, although pigeons can learn the matching-to-learn to abstract the general rules that underlie sample task for small sets of stimuli, if novelordering a meal, such as ‘‘wait to be seated,’’ stimuli are used on every trial, they perform at‘‘order from the menu,’’ and ‘‘pay the bill.’’ chance, suggesting that they are unable to abstractSuch rules are easy to generalize, and we can the rule (Wilson et al., 1985).apply them to any restaurant that we subse- In terms of comparative psychology, otherquently visit. No two restaurants are physically tasks that involve abstract rules yield similaridentical, but because rules operate at a concep- results. One example is the formation of atual level, they are relevant to physically different ‘‘learning set.’’ If primates learn a series of stan-situations. Furthermore, although we initially dard visual discriminations, where they see twoacquire the rules through trial-and-error pictures and must learn to select one of them tolearning (as we blunder our way through our get a reward, their rate of learning gets progres-first restaurant experience), once these rules sively better with each discrimination they solvehave been established, we can use them to (Harlow, 1949). Eventually, the monkey canorder a meal in a new restaurant. Finally, by learn the problem in a single trial: Performanceabstracting the ‘‘gist’’ of a restaurant experience, on the first trial is necessarily at chance, butwe can substantially reduce the amount of infor- performance is virtually 100% correct on themation we need to store about restaurants. second trial. The monkey has learned to extract The ability to abstract information is a capacity the abstract rule ‘‘win-stay, lose-shift,’’that seems to be limited to only certain species. which dramatically speeds performance (Restle,One abstract rule that is relatively easy to 1958). So, too, do corvids, but pigeons mustdemonstrate in animals is that of sameness: The solve each discrimination individually (Huntermatching-to-sample task tests this ability. A & Kamil, 1971; Wilson et al., 1985).subject sees a sample stimulus, and then a short Interestingly, corvid brains differ from those ofwhile later sees two test stimuli, one of which is the other birds, in that they have an enlarged meso-same as the sample. To get a reward, the subject pallium and nidopallium, areas that are analo-must select the test stimulus that matches the gous to prefrontal cortex in mammalssample stimulus. If an animal can successfully (Rehkamper & Zilles, 1991).perform this task, it suggests that the animal can Finally, in humans, we find the crowningappreciate that the sample stimulus and the test achievement of the capacity to use abstractstimulus are the same, and that this relationship information. Language is arguably the abilitycontrols the choice of the test stimulus. A particu- that most clearly distinguishes us from otherlarly stringent test is whether the animal can solve animals (Wynne, 2004). It involves the abilitythe task when the experimenter uses new stimuli to learn thousands of arbitrary associationson each trial since this eliminates any explanations (between words and their meanings) as well asin terms of trial-and-error learning. the capacity to follow a system of abstract rules There is substantial evidence that some species for the meaningful combination of those asso-understand the ‘‘same’’ rule and can solve the ciations (grammar). Humans also have the lar-matching-to-sample task, even when every trial gest and most well-developed frontal lobes, anduses new stimuli. These species include chimpan- as we shall see, there is substantial evidence thatzees (Nissen et al., 1948; Oden et al., 1988), this region of the brain may underpin therhesus monkeys (Mishkin et al., 1962), dolphins capacity to use abstract information.

EXECUTIVE CONTROL CIRCUITS 407PREFRONTAL CORTEX AND THE memory, a limited capacity system responsibleREPRESENTATION OF ABSTRACT for the temporary storage and manipulation ofINFORMATION information. The spatial delayed response task was the first behavioral task to uncover a deficitOne of the most prominent gross anatomical that was specifically associated with damageddifferences in mammalian brain phylogeny is LPFC. In this task, the monkey sees a rewardthe dramatic expansion in the size of the frontal hidden at one of two locations, and then after alobe. This development reaches its zenith in the brief delay is allowed to retrieve it. Monkeys withgreat apes, where frontal cortex accounts for large LPFC lesions act as if they forgot where theapproximately 38% of total cortical surface experimenter hid the reward, even after shortarea (Bush & Allman, 2004; Semendeferi et al., delays of just a few seconds (Jacobsen, 1935).2002). For comparison, frontal cortex accounts Neurophysiologists subsequently discoveredfor 32% of total cortical surface area in the that a high proportion of neurons in LPFCrhesus monkey, 18% in the coyote, and 16% in increased their firing rate when a subject isthe lion. Indeed, comparing across 43 mammals using working memory to bridge a task delayreveals that primate frontal cortex increases dis- (Fuster & Alexander, 1971; Kubota & Niki,proportionately relative to the rest of the neo- 1971). Later studies showed that these neurons’cortex as brain size increases (Bush & Allman, firing properties were selective to the specific2004). Consequently, an attractive hypothesis is memorized cue (Constantinidis et al., 2001;that the expansion of the frontal lobe underlies Funahashi et al., 1989). These firing patternsthe increase in the complexity of the behavioral became synonymous with the notion of a ‘‘neu-repertoire exhibited by primates relative to other ronal representation’’ of the cue, since the neu-mammals. In anatomical terms, the frontal lobe ronal activity appeared to represent the cue, evenis ideally suited to represent abstract, high-level when the cue was no longer present in therules. The anterior region of the frontal lobe is environment.called the prefrontal cortex, and it receives input Despite this focus, working memory alonefrom all sensory modalities (Barbas & Pandya, provided an unsatisfactory account of some of1991; Pandya & Yeterian, 1990). Neurons in the deficits observed in humans and monkeysprefrontal cortex are also responsive to a wide following LPFC damage. Monkeys with lesionsrange of sensory modalities (Rao et al., 1997; of LPFC were impaired on certain cognitiveRolls & Baylis, 1994; Romanski et al., 1999; tasks, even when those tasks did not have aRomo et al., 1999). This ability to respond to a working memory component (Rushworth et al.,wide variety of sensory inputs is critical for 1997). Many of the tasks sensitive to LPFCorganizing sensory information into supra- damage in humans also did not have an obviousmodal concepts and rules. Prefrontal cortex working memory component. For example, thealso sends projections to a variety of secondary Wisconsin Card-Sorting Task (WCST), one ofmotor areas, including premotor cortex (PMC); the first neuropsychological tests sensitive tothese areas in turn project to more primary LPFC damage in humans, requires patients tomotor structures (Barbas & Pandya, 1991; sort a deck of cards based on different abstractPandya & Yeterian, 1990). Thus, prefrontal dimensions (e.g., ‘‘sort according to color’’ orcortex appears to reside at the apex of the per- ‘‘sort according to shape’’). Patients had parti-ception-action cycle, receiving highly processed cular problems switching between these differentsensory information and projecting to high-level dimensions (Milner, 1963). Deficits in rulemotor areas (Fuster, 2002). implementation also occur when a subject must Research on the neural substrates of the override a strongly prepotent response tendencyrepresentation of abstract information focused in favor of a recently learned rule. For example,primarily on the lateral region of prefrontal patients with LPFC damage are impaired on thecortex (LPFC). Many of the first studies of the Stroop task, where one must inhibit the tendencyfunction of LPFC examined its role in working to name a word, and instead name the color of

408 PRIMATE NEUROETHOLOGYthe ink in which the word is written (Perret, the other (Asaad et al., 2000). Thus, LPFC neu-1974). rons were not simply encoding sensory informa- This disparity between the findings from tion, but rather they were encoding sensoryneurophysiology and neuropsychology endured information as it pertained to the demands ofuntil the last decade or so of research, when the task.neurophysiologists began to find that the delay Subsequently, we looked at whether LPFCselectivity in LPFC neurons encoded more than neurons could encode high-level abstract rulesjust memorized cues. Several studies examined (Wallis et al., 2001a). We presented a picture onthe neuronal response to the same sensory cue a computer screen, and we instructed the mon-under different task conditions (Asaad et al., keys to follow either the ‘‘same’’ rule or the ‘‘dif-2000; Hoshi et al., 2000; White & Wise, 1999). ferent’’ rule (Fig. 21.2A). After a brief delay, weFor example, at the sample phase of a task, the presented a second picture. The monkey had toanimal might see a picture of an apple on the make a given response depending on which rulecomputer screen. At a subsequent test phase, if was in effect and whether the second picturethe animal was performing an object task, it matched or did not match the first picture. Anwould see two pictures and would have to sac- advantage of this design is that until the secondcade to the picture that matched the sample, in picture appears the monkey cannot plan its beha-this case the picture of the apple. In contrast, if vioral response, because it has no way of knowingthe animal was performing a conditional visuo- whether the second picture will match the first.motor task, it would have to make a specific Thus, neuronal activity during the delay periodarbitrary motor response, for example, a right- between the two pictures can only encode eitherward saccade (Fig. 21.1). Remarkably, many the abstract rule or the identity of the picture.LPFC neurons responded to the initial presenta- Figure 21.2B shows the activity of a single LPFCtion of the picture of an apple in one task but not neuron during performance of this task. This A Object Associative task task B Fix Cue Delay Firing rate (Hz) 10 Object 5 task 0 10 Associative 5 task 0 0 500 1000 1500 Time from cue onset (ms)Figure 21.1 (A) In the object task, the subject has to remember the cue, and at the test phase, saccade tothe picture that matches the cue. In the associative task, at the test phase the subject has to make a saccade ina specific direction, dependent on the identity of the cue. Both tasks used the same set of cues. (B) Anexample of a lateral prefrontal cortex (LPFC) neuron that responds to a specific cue in the associative taskbut not in the object task, even though the subject is viewing the identical stimulus in both cases. (FromAsaad et al., 2000, with permission).

EXECUTIVE CONTROL CIRCUITS 409Figure 21.2 (A) The subject grasps a lever and a cue picture appears. After a brief delay, a second pictureappears. If the ‘‘same’’ rule is in effect, the monkey must release a lever if the second picture matches the first,but hold the lever if it is different. If the ‘‘different’’ rule is in effect, the monkey must release a lever if thesecond picture does not match the first, but hold the lever if it is the same. Thus, to solve the task correctly,the subject must remember the identity of the cue picture and the rule that is currently in effect. (B) A lateralprefrontal cortex (LPFC) neuron that encodes the rule: It shows a higher firing rate when the ‘‘same’’ rule isin effect compared to when the ‘‘different’’ rule is in effect, irrespective of the picture that the subject isremembering.neuron shows a higher firing rate whenever the LPFC when the subject encoded and maintained‘‘same’’ rule is in effect, but does not discriminate the rule (Bunge et al., 2003). These results suggestbetween the different pictures that the monkey is that encoding of abstract rules is an importantremembering. Many LPFC neurons, particularly function of LPFC, indeed more so than thein the dorsal region of LPFC, encoded the abstract encoding of sensory information. In addition,rule during the delay period (58 of 197 or 29% of the results help explain why patients with LPFCneurons) rather than the picture (7 of 197 or 4% damage have difficulties with tasks such as theof neurons). The task was subsequently adapted WCST.for use in humans where functional magnetic In conclusion, early studies of LPFC functionresonance imaging (fMRI) revealed activation in focused on its role in working memory.

410 PRIMATE NEUROETHOLOGYAlthough this is undoubtedly an important role cognitive resources necessary to solve a task.of LPFC, it may be one of several functions that For example, in a spatial self-ordered searchcollectively enable the abstraction of informa- task, monkeys have to search through ation, by allowing the subject to disengage from number of possible locations to find a hiddenthe processing of the immediate sensory envir- reward, but are penalized for returning to aonment. A recent study, however, has shown location that they have already searched.that while LPFC appears to instantiate abstract Monkeys gradually develop a strategy to helprule representation in mammals, it is not neces- with the task such as searching through locationssary to have the complex neuronal machinery of in a clockwise order. Doing so reduces thea frontal lobe in order to use abstract informa- memory demands of the task, since the animaltion in the control of behavior. Investigators no longer has to keep track of which locations ittrained honeybees on a Y-maze (Giurfa, 2001). has previously visited. Lesions of LPFC severelyAt the entrance to the maze was the sample disrupted the ability of marmosets to implementstimulus, and at the entrance to the two forks this strategy effectively (Collins et al., 1998). Onein the Y-maze were two test stimuli (Fig. 21.3a). sees a similar effect in marmosets that learn toBees received a reward for choosing the arm with search for a hidden door into a box that containsthe matching test stimulus. Not only could the food. Animals adopt a strategy of searching eachbees learn this task, but they also were able to side of the box in turn, but marmosets withapply the rule to novel stimuli (Fig. 21.3b). This lesions of LPFC fail to learn this strategystudy raises interesting questions. For example, (Wallis et al., 2001b). Recent studies havewhy should the capacity to use an abstract rule emphasized the interaction of the frontal andbe useful to bees, but not to pigeons (Wilson temporal cortices, at least in the implementationet al., 1985)? This capacity is not simply the of strategies pertaining to visual discriminations.ability to know that one flower is the ‘‘same’’ as Animals that had acquired a complex alternationanother, a very simple (and useful) behavioral strategy to solve a visual discrimination taskadaptation that can be solved through stimulus were impaired following crossed lesions ofgeneralization and conditioning. Rather, it is frontal and temporal cortex (Gaffan et al.,using the relationship between two stimuli to 2002). However, they could still solve a simplegovern behavior in an arbitrary fashion. Quite visual discrimination that did not require awhat use the bee finds for this ability is a mys- higher-order strategy.tery, but it does demonstrate that a remarkably Taken together, these results strongly impli-simple nervous system, consisting of a brain of cate the primate LPFC in the implementation of1 mm3 and fewer than 1 million neurons strategies. Likewise, neurophysiological studies ¨(Witthoft, 1967), is capable of using abstract have found neurons in LPFC that represent stra-information. It remains an open question tegic information. Wise and colleagues trainedwhether it can learn a variety of abstract infor- monkeys to learn a three-choice conditionalmation, as does the mammalian brain, or visuomotor task (Murray et al., 1996). In thiswhether its abilities are more constrained. task, the monkey learns to make one of three possible motor responses to each of three unique visual stimuli. For example, visual stimulusSTRATEGIES AND SETS A may indicate that the monkey should move aIn addition to enabling explicit, abstract rules to joystick left, stimulus B right, and stimulus Ccontrol information, the LPFC also appears to be down. The monkey learns which response goesinvolved in implementing other forms of high- with which stimulus by trial and error. However,level information, such as strategies and cogni- the experimenters soon realized that the animalstive sets. The acquisition and adoption of a were incorporating high-level information intostrategy involves the control and organization their choices that enabled them to performof cognitive processes in a manner that speeds better than would be expected by chance.learning or otherwise reduces the overall Specifically, the animals kept track of the

EXECUTIVE CONTROL CIRCUITS 411 A BFigure 21.3 (A) The behavioral apparatus used to train bees on a delayed matching-to-sample paradigm.The bee flew into a Y-maze, at the entrance of which was one of two sample stimuli. Half the bees learned thetask using a vertical or horizontal grating, while the other half learned the task using the colors blue andyellow. The entrance led to a decision chamber, where the bee could choose one of two arms. Each armcarried one of the two sample stimuli. If the bee chose the arm with the stimulus that was identical to thesample stimulus, it received a sucrose solution reward. After several days of training, the bees could performthe task with greater than 70% accuracy. (B) Critically, the bees were not solving the task using simple rules(e.g., ‘‘Choose blue and not yellow if blue was at the entrance’’ and ‘‘Choose yellow and not blue if yellowwas at the entrance), but had abstracted the general concept of sameness. This was evident by the bees’performance on transfer tests where they were confronted with new stimuli (colors for the bees trained ongratings, and gratings for those trained on colors). The bees immediately performed the task with greaterthan 70% accuracy, indicating that they had abstracted a rule that they could apply to novel stimuli. (FromGiurfa et al., 2001, with permission).stimulus-response mappings that they knew in A on trial one, and correctly guessed that itorder to rule out potential alternatives when should make a leftward joystick movement. Onencountering a novel stimulus by adopting a the next trial, if stimulus A appeared it would‘‘repeat-stay, change-shift’’ strategy. For exa- always select the leftward movement (repeat-mple, suppose an animal encountered stimulus stay). If, however, one of the other two pictures

412 PRIMATE NEUROETHOLOGYappeared, it would never select the leftward shape is now relevant, it takes the animal a longmovement, but would instead choose one of time to learn, since it mistakenly focuses on itsthe two remaining alternative movements color. Lesions of LPFC dramatically exacerbate(change-shift). This simple strategy enabled the the difficulty that monkeys have in shifting theiranimal to perform at 67% correct on this second attentional set (Dias et al., 1996). Subsequenttrial, rather than the 33% expected by chance. studies showed that dopaminergic innervationSubsequent neurophysiological recording of LPFC was critical in mediating attentionalrevealed that 41% of LPFC neurons encoded shifts. Dopamine depletions in LPFC impairedthis high-level strategy, showing differences in attentional set shifting, but dopamine depletionsactivity on repeat-stay versus change-shift trials in the neighboring orbitofrontal cortex or seroto-(Genovesio et al., 2005). nergic depletions of LPFC did not affect set A cognitive construct related to strategies is shifting (Clarke et al., 2004, 2005).the notion of ‘‘task set.’’ A task set consists of The aforementioned task is very similar to theinformation about which stimulus attributes to WCST. When subjects learns to sort a deck ofattend to, important conceptual criterion, goal cards according to color, they are in effect devel-states, and condition-action rules (Monsell, oping an attentional set to color. The WCST was2003). It reflects not just which items a subject originally developed as a test of cognitive flex-is preparing to process, but also how the subject ibility (Berg, 1948) and was subsequently foundplans to process the items and the rules of the to- to detect PFC damage in humans (Milner, 1963),be-performed task. One type of task set is an Recent fMRI studies using primate analogs ofattentional set, which consists of an expectancy the task show that the same neuronal substratesthat certain abstract properties of one’s sensory underlie the ability to perform the task acrossenvironment will be necessary to solve a task, species (Nakahara et al., 2002). The same regionwhich in turn leads to a focusing of attention on of LPFC (the ventral portion consisting ofthose properties. Consider the animal that learns Brodmann’s areas 44 and 45) is activatedto solve a series of discriminations between mul- during the performance of the WCST in bothtidimensional visual stimuli. For example, a set humans and rhesus monkeys (Fig. 21.4).of stimuli might consist of a red heart, a red Subsequent neurophysiological studies showeddiamond, a black heart, and a black diamond. that approximately a third of LPFC neuronsOn each trial two of the stimuli appear, one of were engaged in encoding the attentional shifteach color and shape. The correct answer is to (Mansouri et al., 2006).choose the red stimulus regardless of whether it The formation of attentional sets speedsis a heart or a diamond. Once the animal has learning. However, task sets are also evidentlearned to do this consistently, the experimenter even when a cue indicates the currently relevantgives it a second problem. For example, the new task, thereby removing the learning componentset might consist of a green triangle, green (Monsell, 2003; Monsell et al., 2003). Forsquare, yellow triangle, and yellow square, and example, a subject might have to indicatethe correct response is to choose the green sti- whether a number is either odd or less thanmulus irrespective of its shape. The animal seven depending on a task instruction cue.learns successive discriminations progressively This experimental design introduces an incon-quicker as it realizes which dimension of the gruity cost (also referred to as ‘‘conflict’’):stimulus is relevant (color in this particular Subjects are typically faster to respond when aexample). The animal has learned what is stimulus instructs different responses on thetermed an ‘‘attentional set.’’ In effect, the animal tasks (e.g., the number nine in the previoushas developed an expectancy that one particular task, which is odd, but greater than seven)abstract property of the stimulus will be necessary than if they instruct the same response (theto solve the discrimination, and so it focuses its number three, which is both odd and less thanattention on that property. When the animal seven). In addition, there is a switch cost:subsequently encounters a discrimination where Performance is faster when the task remains

EXECUTIVE CONTROL CIRCUITS 413 A BFigure 21.4 Functional magnetic resonance imaging scans reveal how performance of WCST activatesposterior ventrolateral prefrontal cortex in both (A) monkeys and (B) humans. Green arrowhead indicatesthe principal sulcus, blue arrowhead the inferior arcuate sulcus, and yellow arrowhead the inferior frontalsulcus. (From Nakahara et al., 2002, with permission).the same (nonswitch trials), but worse when the that happens to correlate with incongruity pro-task changes (switch trials). A study of the cessing (di Pellegrino et al., 2007; Fellows &comparative psychology of task switching in Farah, 2005; Rushworth et al., 2004). In contrast,monkeys and humans suggests that these two a recent study shows that LPFC does have acosts may inversely correlate, and reflect the singular role in the implementation of incon-degree to which the two species can effectively gruity resolution (Mansouri et al., 2007). Whenimplement task sets (Stoet & Snyder, 2003). subjects experience an incongruity, they typicallyHumans typically show large switch costs but make a behavioral adjustment such that theirsmall incongruity costs, while the opposite pat- performance on the next trial is slower, buttern occurs in monkeys. This may reflect the also more accurate (Rabbitt, 1966). Lesions ofbetter implementation of the task set by LPFC in monkeys abolished this behavioralhumans. Thus, humans are better able to main- adjustment induced by incongruity, whereastain focused attention on a particular task for lesions of ACC had no effect (Fig. 21.5A,B).long periods and ignore irrelevant aspects of the The modulation of behavior in the current trialstimulus, leading to smaller incongruity costs by the degree of incongruity experienced in thethan monkeys. However, this ability comes previous trial requires a system to hold thewith a cost, namely, that it is more difficult for memory of experienced incongruity acrossthe human to switch to a different task. trials. The activity of LPFC neurons was consis- Regarding the neuronal underpinnings of tent with such a system since they encoded bothtask switching, activation in LPFC in humans the degree of incongruity on the current trialis associated with switch costs (Badre & (Fig. 21.5C) as well as the degree of incongruityWagner, 2006; Brass & von Cramon, 2004; on the previous trial (Fig. 21.5D).Braver et al., 2003) and incongruity costs(Dreher & Berman, 2002; Kerns et al., 2004;MacDonald et al., 2000). This is not to say that FUNCTIONAL ORGANIZATIONLPFC is the only area involved in task switching. OF HIGH-LEVEL INFORMATIONCorrelates of incongruity costs are evident in theactivity of single neurons in posterior parietal To recap, we have seen that LPFC has undergonecortex of monkeys (Stoet & Snyder, 2007) and a dramatic expansion in primate evolution andactivations of anterior cingulate cortex (ACC) in that it is responsible for the encoding of a widehumans (Dreher & Berman, 2002; Kerns et al., variety of information that would be useful for2004; MacDonald et al., 2000). However, it has high-level cognitive behavior, including abstractproven controversial whether these areas are rules, behavioral strategies, and task sets. Recentnecessary for the processing of incongruity and work has attempted a more fine-grain analysis ofresolution of such conflict, or whether their the relationship between LPFC and the encodingactivity reflects a different cognitive process of high-level information. A number of models

414 PRIMATE NEUROETHOLOGYFigure 21.5 (A) Lateral (top) and medial (bottom) views of the macaque brain illustrating the location ofthe lateral prefrontal cortex (LPFC) and anterior cingulate cortex (ACC) lesions. (B) Control animals showlonger reaction times on high-conflict trials when the trial was preceded by a high-conflict trial (HH) asopposed to when it was preceded by a low-conflict trial (LH). Lesions of LPFC, but not ACC, abolished thisconflict-induced behavioral adjustment. (C) An LPFC neuron that shows a higher firing rate when thesample stimulus elicits low conflict. (D) An LPFC neuron that shows a higher firing rate when the subjectexperienced low conflict on the previous trial. (From Mansouri et al., 2007, with permission).have been developed that propose a hierarchical that the implementation of simple conditionalorganization of rules within the frontal lobe, rules depends on the premotor cortex.with progressively higher-level information Conditional rules take the form ‘‘If X, then Y,’’represented in progressively more anterior frontal and typically take the form of stimulus-responseregions (Badre & D’Esposito, 2007; Bunge & associations, for example, ‘‘If red light, then brake.’’Zelazo, 2006; Christoff & Keramatian, 2008; Lesions of premotor cortex in monkeys impair theKoechlin et al., 2003; Sakai & Passingham, ability to perform such tasks (Halsband &2003). These findings are consistent with models Passingham, 1982; Petrides, 1982) and premotorthat propose that hierarchical organization is a cortex is activated in humans when they performgeneral feature of brain organization (Fuster, tasks requiring conditional rules (Amiez et al.,2002; Hawkins & Blakeslee, 2007). 2006). However, rules that are more complex Although there are differences between the than conditional rules typically produce activationmodels regarding the manner of the hierarchical more anterior to the premotor cortex. Fororganization, nearly all begin with the observation example, compound rules that require subjects to

EXECUTIVE CONTROL CIRCUITS 415integrate two pieces of information to determine manipulating not only the abstractness of thethe correct response (‘‘If X and if Y, then Z’’) rules (as in the previous tasks) but also the overallactivate LPFC more than simple conditional rules complexity of the task via the number of contin-(Bunge et al., 2003). In a similar vein, simple con- gencies that the subjects had to follow (Badre &ditional rules activate PMC, while rules that require D’Esposito, 2007). Although increasing theconsideration of both the stimulus and the context number of contingencies, and thus the overallin which the stimulus occurs activate LPFC complexity of the task, led to larger activations, it(Koechlin et al., 2003). In addition, rules that had no effect on the location of those activations.require consideration of the stimulus, the context, Only the abstractness of the rule altered the activa-and the temporal episode in which the task occurs tion locus, with increasingly abstract rules acti-activate cortex anterior to LPFC in the frontal pole vating increasingly anterior regions of the frontal(Koechlin et al., 2003). However, task complexity lobe (Fig. 21.6).per se does not recruit more anterior regions of So far, we have considered the human frontalcortex. Investigators studied the effects of lobe. Is there evidence that such a hierarchy of A Response task B Feature task 1 2 Feature1 Feature2 Feature2 Feature1 1 2 1 2 C Dimension task A B Shape1 Shape2 Size1 Size2 C D Shape1 Shape2 Shape1 Shape2 Size1 Size2 Size1 Size2 1 2 2 1 1 2 2 1 D Context task Block 1 Block 2 Shape1 Shape2 Size1 Size2 Shape1 Shape2 Size1 Size2 Shape1 Shape2 Shape1 Shape2 Size1 Size2 Size1 Size2 Shape1 Shape2 Shape1 Shape2 Size1 Size2 Size1 Size2 1 2 2 1 1 2 2 1 1 2 2 1 1 2 2 1Figure 21.6 Subjects performed four different tasks that required them to follow rules at varying degreesof abstraction. The experimenters could alter the complexity independent of its level of abstraction byincreasing the number of contingencies presented to the subject. (A) The subject learns a stimulus-response contingency. The color of a presented square instructs which of two responses to make. (B) Thesubject learns a stimulus-feature contingency. The color of the square tells the subject to look for one oftwo features present in a sample stimulus, and make one of two responses depending on the presence orabsence of that feature. (C) The subject learns a stimulus-dimension contingency. The color of the squaretells the subject to attend to one of two stimulus dimensions (and ignore the other dimension), and makeone of two responses depending on whether two stimuli share a specific feature within that stimulusdimension. (D) The subject learns context-based stimulus-dimension contingencies. The colored squareinstructs the subject to attend to a specific stimulus dimension, but the color-dimension contingencychanges between blocks of trials. The subject must then identify whether two stimuli share a specificfeature within that stimulus dimension and make the appropriate response. Brain activation patternsreveal that as the task becomes increasingly abstract, the locus of activation shifts anteriorly within lateralprefrontal cortex. (From Badre & D’Esposito, 2007, with permission).

416 PRIMATE NEUROETHOLOGYfunction is also present in the monkey? A series of Having them learn a number of abstract rulesstudies investigating the neuronal basis of that depend on some other task context, as inreaching found evidence of hierarchical organi- the tasks used in humans (Badre & D’Esposito,zation stretching from primary motor cortex into 2007; Koechlin et al., 2003), is likely to beLPFC in terms of the control of motor behavior. impossible.LPFC neurons sat atop the hierarchy and The obvious implication of these findings is thatencoded the abstract behavioral rule that indi- the most anterior part of the frontal lobe (termedcated whether to select a stimulus based on either the ‘‘frontal pole’’) should also be responsible forthe location or identity of a previous stimulus the highest-level control. A recent review of the(Hoshi et al., 2000). PMC neurons encoded neuroimaging literature concluded that, ininformation about the target location, as well as humans, this was indeed the case, arguing that thewhich arm the subject would use to reach to the frontal pole was particularly important when twotarget, consistent with a role in planning the or more cognitive operations need integrating inaction (Hoshi & Tanji, 2000). Finally, neurons the pursuit of a higher behavioral goal (Ramnani &in primary motor cortex were active during the Owen, 2004). For example, frontal pole activationexecution of the movement, consistent with their occurs when subjects must maintain informationrole in motor control (Hoshi & Tanji, 2000). in working memory while performing a secondaryHowever, hierarchical organization of cognitive task during the delay interval (Braver &information is not apparent within LPFC itself. Bongiolatti, 2002; Koechlin et al., 1999).We recorded from a large expanse of LPFC However, the frontal pole is also the region of thewhile monkeys were implementing ‘‘same’’ and primate frontal lobe that is most diverse across‘‘different’’ rules, and there was no evidence of an different primate species (Fig. 21.7), seeming toanterior-posterior gradient within LPFC in terms occupy a disproportionately greater area inof the prevalence of neurons encoding these humans than in other primates (Semendeferi etabstract rules (Wallis & Miller, 2003; Wallis et al., 2001). The anatomical connections of theal., 2001a). Thus, the question remains as to why frontal pole in the monkey are only partially con-we do not see a hierarchical organization of rules sistent with its role as the apex of high-level control.in monkey LPFC. In part, it may be because we It connects most strongly with cingulate cortex,have not investigated multiple levels of abstract orbitofrontal cortex, and multimodal regions ofrules, as we have done in humans. The reason for the temporal lobe (Petrides & Pandya, 2007).this is that we seem to be approaching the limit of Thus, it is unusual within the frontal lobe in thatthe cognitive capabilities of monkeys. Teaching it does not connect with any motor or sensorythem two relatively simple abstract rules, such as areas, but rather connects with areas responsible‘‘same’’ and ‘‘different,’’ takes %200,000 trials. for multisensory processing and internal emotionalFigure 21.7 The right prefrontal cortex of (A) a human and (B) a monkey brain, represented as flattenedsurfaces, illustrating the considerably larger proportion of the surface area occupied by area 10 (shaded inred) in humans relative to monkeys. (From Ramnani & Owen, 2007, with permission).

EXECUTIVE CONTROL CIRCUITS 417states. While this is consistent with the role of the than in LPFC. Despite this early neural correlatefrontal pole in processing the most abstract, high- of learning in CN, the monkeys’ behavior didlevel information (such as multimodal sensory not change until LPFC encoded the associations.information), its relatively weak connections with These results present us with a challenge: Whymuch of LPFC seems inconsistent with the idea would the monkeys continue to make errors,that it sits atop the functional hierarchy in the despite the fact that CN was encoding the correctlateral frontal lobe. A challenge for future studies picture-response associations? The apparentis to reconcile these findings from monkeys and implication of these findings is that overt beha-humans. However, given the disparity in the size of vior is under the control of LPFC more so thanthe frontal pole between the two species under that of CN. Furthermore, they suggest(Semendeferi et al., 2001), it is possible that our that LPFC will not necessarily use all of theuse of the monkey to model human cognitive available information to control behaviorprocesses is not appropriate to the frontal pole. (since it apparently ‘‘ignores’’ CN). One possibility is that LPFC is integrating information from many low-level learning systems, notIMPLEMENTATION OF just CN, and that some of these systems may notHIGH-LEVEL INFORMATION necessarily agree with CN as to the correct response. For example, consider the brain systems thatRepresenting information relating to high-level acquire stimulus-reward associations or action-control is one step in using this information to reward associations. It is impossible to learn sti-control behavior. However, the information mulus-response associations using such stimulus-must then interact with systems responsible for reward or action-reward associations because eachmotor control. We have examined how this action and each stimulus are rewarded equallytakes place by simultaneously recording from often. However, this does not necessarily meanLPFC, as well as two of the major output struc- that these systems will be silent during the perfor-tures of the LPFC, PMC and the caudate nucleus mance of a task dependent on stimulus-response(CN), while subjects were applying the ‘‘same’’ associations. For example, perhaps after a rein-or ‘‘different’’ rule to successively presented forced leftward saccade, the action-reward systempictures (Muhammad et al., 2006; Wallis et al., instructs LPFC to make another leftward response,2001a; Wallis & Miller, 2003). We found that the oblivious to the fact that on the next trial, thestrongest rule selectivity was in the frontal lobe stimulus instructs a rightward response. LPFC(LPFC and PMC) with only weak rule selectivity would need to learn that such information is notin CN. During the delay between the two pic- useful to solve the task and ignore this system.tures, rule-selective neurons were significantly Lesion studies support the idea that these differentmore prevalent in LPFC (58 of 197 or 29%) low-level learning systems can compete with oneand PMC (59 of 258 or 23%) than they were in another. For example, lesions of anterior cingulateCN (38 of 341 or 11%). cortex impair the learning of stimulus-reward asso- These results stand in marked contrast to ciations (Bussey et al., 1997; Gabriel et al., 1991), butstudies examining the role of CN in simpler facilitate the learning of stimulus-response associa-levels of behavioral control, such as the imple- tions (Bussey et al., 1996). These findings suggestmentation of conditional rules involving con- that in the healthy animal, anterior cingulate con-crete stimulus-response associations. Pasupathy tributes to the learning of stimulus-reward associa-and Miller (2005) recorded data simultaneously tions, and that removing the capacity to learn suchfrom LPFC and CN while monkeys learned associations can improve the ability to learn sti-which of two behavioral responses (saccade left mulus-response associations.or right) to make to which of two pictures (A A puzzling feature of our results is that infor-and B). Both structures encoded the associations mation about the abstract rule appeared approxi-between the pictures and the responses, but mately 90 ms earlier in PMC than in LPFC (CNselectivity appeared earlier in learning in CN neurons lay somewhere in between, but the low

418 PRIMATE NEUROETHOLOGYnumbers of rule-selective neurons in CN make problem. The current evidence favors such an orga-this difficult to interpret). This seems to contra- nization in the human frontal lobe and a rudimen-dict the notion of a hierarchical organization, tary analog in the frontal lobe of the monkey. In thissince PMC is anatomically lower on the hierarchy way, primate behavior has evolved to be the mostthan LPFC, yet encodes high-level information complex behavior in the animal kingdom.quicker. One possible explanation lies in the factthat the rules were highly familiar to the animals;they had performed this task for more than a year. REFERENCESEvidence suggests that LPFC is more critical for Amiez, C., Kostopoulos, P., Champod, A. S., &new learning than for familiar routines. LPFC Petrides, M. (2006). Local morphologydamage preferentially affects new learning; ani- predicts functional organization of the dorsalmals and humans can still engage in complex premotor region in the human brain. Journalbehaviors as long as they learned them before of Neuroscience, 26, 2724–2731.the damage occurred (Dias et al., 1997; Knight, Asaad, W.F., Rainer, G. & Miller, E.K. (1998) Neural1984; Shallice, 1982; Shallice & Evans, 1978). activity in the primate prefrontal cortex during associative learning. Neuron, 21, 1399–1407.LPFC neurons also show more selectivity during Asaad, W. F., Rainer, G., & Miller, E. K. (2000). Task-new learning than during the performance of specific neural activity in the primate prefrontalfamiliar cue-response associations (Asaad et al., cortex. Journal of Neurophysiology, 84, 451–459.1998). Human imaging studies report greater Badre, D., & D’Esposito, M. (2007). Functionalblood flow to PMC than LPFC when subjects magnetic resonance imaging evidence for aare performing familiar versus novel tasks hierarchical organization of the prefrontal cortex.(Boettiger & D’Esposito, 2005) and greater Journal of Cognitive Neuroscience, 19, 2082–2099.LPFC activation when subjects are retrieving Badre, D., & Wagner, A. D. (2006). Computationalnewly learned rules versus highly familiar rules and neurobiological mechanisms underlying(Donohue, 2005). In addition, with increasing cognitive flexibility. Proceedings of the Nationaltask familiarity, there is a relative shift in blood Academy of Sciences, 103, 7186–7191.flow from areas associated with focal attention, Barbas, H., & Pandya, D. (1991). Patterns ofsuch as LPFC, to motor regions (Della-Maggiore connections of the prefrontal cortex in the rhesus monkey associated with cortical architecture. In:& McIntosh, 2005). Therefore, it may be that H. S. Levin, H. M. Eisenberg, & A. L. Benton (Eds.),LPFC is primarily involved in new learning, but Frontal lobe function and dysfunction (pp. 35–58).with familiarity, rules become more strongly estab- New York: Oxford University Press.lished in downstream motor system structures. Berg, E. A. (1948). A simple objective test for measuring flexibility in thinking. Journal of General Psychology, 39, 15–22.CONCLUSION Boettiger, C.A. & D’Esposito, M. (2005) FrontalAs behavior has become more complex, it has networks for learning and executing arbitraryinevitably become more difficult to control. The stimulus-response associations. Journal ofanimal that is reliant on conditioning parses the Neuroscience, 25, 2723–2732.world into two behaviors: approach or avoid. The Brass, M., & von Cramon, D. Y. (2004). Selectionanimal that learns arbitrary relationships, however, for cognitive control: A functional magnetic resonance imaging study on the selection ofhas a far more difficult task. Presented with a single task-relevant information. Journal ofstimulus, it must determine the salient features. Neuroscience, 24, 8847–8852.Should it pay attention to its color, its shape, or Braver, T. S., & Bongiolatti, S. R. (2002). The role ofsome other feature? Even once the animal has frontopolar cortex in subgoal processing duringidentified the relevant sensory dimension, it must working memory. Neuroimage, 15, 523–536.then select among the myriad potential motor Braver, T. S., Reynolds, J. R., & Donaldson, D. I.responses. A hierarchical organization of cognitive (2003). Neural mechanisms of transient andcontrol, with increasing abstraction at each level of sustained cognitive control during taskthe hierarchy, is one potential solution to the switching. Neuron, 39, 713–726.

EXECUTIVE CONTROL CIRCUITS 419Bunge, S. A., Kahn, I., Wallis, J. D., Miller, E. K., & di Pellegrino, G., Ciaramelli, E., & Ladavas, E. Wagner, A. D. (2003). Neural circuits subserving (2007). The regulation of cognitive control the retrieval and maintenance of abstract rules. following rostral anterior cingulate cortex Journal of Neurophysiology, 90, 3419–3428. lesion in humans. Journal of CognitiveBunge, S. A., & Zelazo, P. D. (2006). A brain-based Neuroscience, 19, 275–286. account of the development of rule use in Dias, R., Robbins, T. W., & Roberts, A. C. (1996). childhood. Current Directions in Psychological Dissociation in prefrontal cortex of affective Science, 15, 118–121. and attentional shifts. Nature, 380, 69–72.Bush, E. C., & Allman, J. M. (2004). The scaling of Dias, R., Robbins, T.W. & Roberts, A.C. (1997) frontal cortex in primates and carnivores. Dissociable forms of inhibitory control within Proceedings of the National Academy of prefrontal cortex with an analog of the Wisconsin Sciences, 101, 3962–3966. Card Sort Test: restriction to novel situations andBussey, T.J., Muir, J.L., Everitt, B.J. & Robbins, T.W. independence from ‘‘on-line’’ processing. Journal (1996) Dissociable effects of anterior and posterior of Neuroscience, 17, 9285–9297. cingulate cortex lesions on the acqui-sition of a Donohue, S.E., Wendelken, C., Crone, E.A. & Bunge, conditional visual discrimination: facilitation of S.A. (2005) Retrieving rules for behavior from early learning vs. impairment of late learning. long-term memory. Neuroimage, 26, 1140–1149. Behavioural Brain Research, 82, 45–56. Dreher, J. C., & Berman, K. F. (2002). FractionatingBussey, T.J., Muir, J.L., Everitt, B.J. & Robbins, the neural substrate of cognitive control T.W. (1997) Triple dissociation of anterior processes. Proceedings of the National Academy cingulate, posterior cingulate, and medial of Sciences, 99, 14595–14600. frontal cortices on visual discrimination tasks Fellows, L. K., & Farah, M. J. (2005). Is anterior using a touchscreen testing procedure for the cingulate cortex necessary for cognitive control? rat. Behavioral Neuroscience, 111, 920–936. Brain, 128, 788–796.Christoff, K., & Keramatian, K. (2008). Abstraction of Funahashi, S., Bruce, C. J., & Goldman-Rakic, P. mental representation: Theoretical conisderations S. (1989). Mnemonic coding of visual space in and neuroscientific evidence. In: S. A. Bunge & J. the monkey’s dorsolateral prefrontal cortex. D. Wallis (Eds.), Neuroscience of rule-guided Journal of Neurophysiology, 61, 331–349. behavior. New York: Oxford University Press. Fuster, J. M. (2002). Cortex and mind. Oxford:Clarke, H. F., Dalley, J. W., Crofts, H. S., Robbins, T. Oxford University Press. W., & Roberts, A. C. (2004). Cognitive inflexi- Fuster, J. M., & Alexander, G. E. (1971). Neuron bility after prefrontal serotonin depletion. activity related to short-term memory. Science, Science, 304, 878–880. 173, 652–654.Clarke, H. F., Walker, S. C., Crofts, H. S., Dalley, J. Gabriel, M., Kubota, Y., Sparenborg, S., Straube, K. W., Robbins, T. W., & Roberts, A. C. (2005). & Vogt, B.A. (1991) Effects of cingulate cortical Prefrontal serotonin depletion affects reversal lesions on avoidance learning and training- learning but not attentional set shifting. induced unit activity in rabbits. Experimental Journal of Neuroscience, 25, 532–538. Brain Research, 86, 585–600.Collins, P., Roberts, A. C., Dias, R., Everitt, B. J., & Gaffan, D., Easton, A., & Parker, A. (2002). Interaction Robbins, T. W. (1998). Perseveration and strategy of inferior temporal cortex with frontal cortex and in a novel spatial self-ordered sequencing task for basal forebrain: Double dissociation in strategy nonhuman primates: Effects of excitotoxic lesions implementation and associative learning. Journal and dopamine depletions of the prefrontal cortex. of Neuroscience, 22, 7288–7296. Journal of Cognitive Neuroscience, 10, 332–354. Genovesio, A., Brasted, P. J., Mitz, A. R., & Wise, S. P.Constantinidis, C., Franowicz, M. N., & Goldman- (2005). Prefrontal cortex activity related to Rakic, P. S. (2001). The sensory nature of abstract response strategies. Neuron, 47, 307–320. mnemonic representation in the primate pre- Giurfa, M., Zhang, S., Jenett, A., Menzel, R. & frontal cortex. Nature Neuroscience, 4, 311–316. Srinivasan, M.V. (2001) The concepts ofDella-Maggiore, V. & McIntosh, A.R. (2005) Time ‘sameness’ and ’difference’ in an insect. Nature, course of changes in brain activity and functional 410, 930–933. connectivity associated with long-term Halsband, U., & Passingham, R. (1982). The role of adaptation to a rotational transformation. premotor and parietal cortex in the direction of Journal of Neurophysiology, 93, 2254–2262. action. Brain Research, 240, 368–372.

420 PRIMATE NEUROETHOLOGYHarlow, H. F. (1949). The formation of learning Mansouri, F. A., Matsumoto, K., & Tanaka, K. sets. Psychological Review, 56, 51–65. (2006). Prefrontal cell activities related toHawkins, J., & Blakeslee, S. (2007). On intelligence. monkeys’ success and failure in adapting to New York: Henry Holt & Co. rule changes in a Wisconsin Card Sorting TestHerman,L.M.,&Gordon,J.A.(1974).Auditorydelayed analog. Journal of Neuroscience, 26, 2745–2756. matching in the bottlenose dolphin. Journal of the Milner, B. (1963). Effects of different brain lesions on ExperimentalAnalysis of Behavior, 21,19–26. card sorting. Archives of Neurology, 9, 100–110.Hoshi, E., Shima, K., & Tanji, J. (2000). Neuronal Mishkin, M., Prockop, E. S., & Rosvold, H. E. (1962). activity in the primate prefrontal cortex in the One-trialobjectdiscriminationlearninginmonkeys process of motor selection based on two behavioral with frontal lesions. Journal of Comparative and rules. Journal of Neurophysiology, 83, 2355–2373. Physiological Psychology, 55, 178–181.Hoshi, E., & Tanji, J. (2000). Integration of target and Monsell, S. (2003). Task switching. Trends in body-part information in the premotor cortex Cognitive Science, 7, 134–140. when planning action. Nature, 408, 466–470. Monsell, S., Sumner, P., & Waters, H. (2003). Task-Hunter, M. W., & Kamil, A. C. (1971). Object set reconfiguration with predictable and discrimination learning set and hypothesis unpredictable task switches. Memory and behavior in the northern blue jay (Cyanocitta Cognition, 31, 327–342. cristata). Psychonomic Science, 22, 271–273. Muhammad, R., Wallis, J. D., & Miller, E. K.Jacobsen, C. F. (1935). Functions of the frontal (2006). A comparison of abstract rules in the association area in primates. Archives of prefrontal cortex, premotor cortex, inferior Neurology and Psychiatry, 33, 558–569. temporal cortex, and striatum. Journal ofKastak, D., & Schusterman, R. J. (1994). Transfer of Cognitive Neuroscience, 18, 974–989. visual identity matching-to-sample in two Murray, E. A., Gaffan, E. A., & Flint, R. W., Jr. Californian sea lions (Zalophus californianus). (1996). Anterior rhinal cortex and amygdala: Animal Learning and Behavior, 22, 427–453. Dissociation of their contributions to memoryKerns, J. G., Cohen, J. D., MacDonald, A. W., 3rd, and food preference in rhesus monkeys. Cho, R. Y., Stenger, V. A., & Carter, C. S. Behavioral Neuroscience, 110, 30–42. (2004). Anterior cingulate conflict Nakahara, K., Hayashi, T., Konishi, S., & Miyashita, Y. monitoring and adjustments in control. (2002). Functional MRI of macaque monkeys Science, 303, 1023–1026. performing a cognitive set-shifting task. Science,Knight, R.T. (1984) Decreased response to novel 295, 1532–1536. stimuli after prefrontal lesions in man. Nissen, H. W., Blum, J. S., & Blum, R. A. (1948). Electroencephalography and Clinical Neuro- Analysis of matching behavior in chimpanzees. physiology, 59, 9–20. Journal of Comparative and PhysiologicalKoechlin, E., Basso, G., Pietrini, P., Panzer, S., & Psychology, 41, 62–74. Grafman, J. (1999). The role of the anterior Oden, D. L., Thompson, R. K., & Premack, D. prefrontal cortex in human cognition. Nature, (1988). Spontaneous transfer of matching by 399, 148–151. infant chimpanzees (Pan troglodytes). JournalKoechlin, E., Ody, C., & Kouneiher, F. (2003). The of Experimental Psychology: Animal Behavior architecture of cognitive control in the human Processes, 14, 140–145. prefrontal cortex. Science, 302, 1181–1185. Pandya, D. N., & Yeterian, E. H. (1990). PrefrontalKubota, K., & Niki, H. (1971). Prefrontal cortical unit cortex in relation to other cortical areas in activity and delayed alternation performance in rhesus monkey: Architecture and connections. monkeys. Journal of Neurophysiology, 34, 337–347. Progress in Brain Research, 85, 63–94.MacDonald, A. W., 3rd, Cohen, J.D., Stenger,V.A., & Passingham, R. E., Myers, C., Rawlins, N., Carter, C. S. (2000). Dissociating the role of the Lightfoot, V., & Fearn, S. (1988). Premotor dorsolateral prefrontal and anterior cingulate cortex in the rat. Behavioral Neuroscience, 102, cortex in cognitive control. Science, 288, 1835– 101–109. 1838. Pasupathy, A. & Miller, E.K. (2005) Different timeMansouri, F. A., Buckley, M. J., & Tanaka, K. courses of learning-related activity in the prefrontal (2007). Mnemonic function of the dorsolateral cortex and striatum. Nature, 433, 873–876. prefrontal cortex in conflict-induced behavioral Pepperberg, I. M. (1987). Interspecies communi- adjustment. Science, 318, 987–990. cation: A tool for assessing conceptual abilities

EXECUTIVE CONTROL CIRCUITS 421 in the African Grey parrot (Psittacus arithacus). decisions in the medial frontal cortex. Trends in In: G. Greenberg & E. Tobach (Eds.), Cognition, Cognitive Science, 8, 410–417. language and consciousness: Integrative levels Sakai, K., & Passingham, R. E. (2003). Prefrontal (pp. 31–56). Hillsdale, NJ: Lawrence Erlbaum interactions reflect future task operations. Associates Inc. Nature Neuroscience, 6, 75–81.Perret, E. (1974). The left frontal lobe of man and Semendeferi, K., Armstrong, E., Schleicher, A., the suppression of habitual responses in verbal Zilles, K., & Van Hoesen, G. W. (2001). categorical behavior. Neuropsychologia, 12, Prefrontal cortex in humans and apes: A 323–330. comparative study of area 10. AmericanPetrides, M. (1982). Motor conditional associative- Journal of Physical Anthropology, 114, 224–241. learning after selective prefrontal lesions in Semendeferi, K., Lu, A., Schenker, N., & Damasio, the monkey. Behavioural Brain Research, 5, 407– H. (2002). Humans and great apes share a large 413. frontal cortex. Nature Neuroscience, 5, 272–276.Petrides, M., & Pandya, D. N. (2007). Efferent Shallice, T. (1982) Specific impairments of association pathways from the rostral planning. Philosophical Transactions of the prefrontal cortex in the macaque monkey. Royal Society London B Biological Sciences, 298, Journal of Neuroscience, 27, 11573–11586. 199–209.Rabbitt, P. M. (1966). Errors and error correction Shallice, T. & Evans, M.E. (1978) The involvement in choice-response tasks. Journal of Experi- of the frontal lobes in cognitive estimation. mental Psychology, 71, 264–272. Cortex, 14, 294–303.Ramnani, N., & Owen, A. M. (2004). Anterior Stoet, G., & Snyder, L. H. (2003). Executive control prefrontal cortex: Insights into function from and task-switching in monkeys. Neuropsychologia, anatomy and neuroimaging. Nature Review 41, 1357–1364. Neurosciences, 5, 184–194. Stoet, G., & Snyder, L. H. (2007). Correlates ofRao, S. C., Rainer, G., & Miller, E. K. (1997). stimulus-response congruence in the posterior Integration of what and where in the primate parietal cortex. Journal of Cognitive Neuroscience, prefrontal cortex. Science, 276, 821–824. 19, 194–203.Rehkamper, G., & Zilles, K. (1991). Parallel evolution Wallis, J. D., Anderson, K. C., & Miller, E. K. in mammalian and avian brains: Comparative (2001a). Single neurons in prefrontal cortex cytoarchitectonic and cytochemical analysis. Cell encode abstract rules. Nature, 411, 953–956. Tissue Research 263, 3–28. Wallis, J. D., Dias, R., Robbins, T. W., &Restle, F. (1958). Toward a quantitative description Roberts, A. C. (2001b). Dissociable contributions of learning set data. Psychological Review, 64, of the orbitofrontal and lateral prefrontal cortex of 77–91. the marmoset to performance on a detourRolls, E. T., & Baylis, L. L. (1994). Gustatory, reaching task. European Journal of Neuroscience, olfactory, and visual convergence within the 13, 1797–1808. primate orbitofrontal cortex. Journal of Wallis, J. D., & Miller, E. K. (2003). From rule to Neuroscience, 14, 5437–5452. response: Neuronal processes in the premotorRomanski, L. M., Bates, J. F., Goldman-Rakic, P. S. and prefrontal cortex. Journal of (1999). Auditory belt and parabelt projections Neurophysiology, 90, 1790–1806. to the prefrontal cortex in the rhesus White, I. M., & Wise, S. P. (1999). Rule-dependent monkey. Journal of Comparative Neurology, neuronal activity in the prefrontal cortex. 403, 141–157. Experimental Brain Research, 126, 315–335.Romo, R., Brody, C. D., Hernandez, A., & Lemus, Wilson, B., Mackintosh, N. J., & Boakes, R. A. L. (1999). Neuronal correlates of parametric (1985). Transfer of relational rules in matching working memory in the prefrontal cortex. and oddity learning by pigeons and corvids. Nature, 399, 470–473. Quarterly Journal of Experimental Psychology,Rushworth, M. F., Nixon, P. D., Eacott, M. J., & 37B, 313–332. Passingham, R. E. (1997). Ventral prefrontal ¨ Witthoft, W. (1967) Absolute anzahl und Verteilung cortex is not essential for working memory. der zellen im hirn der honigbiene. Zeitschrift fur Journal of Neuroscience, 17, 4829–4838. Morphologie der Tiere, 61, 160–184.Rushworth, M. F., Walton, M. E., Kennerley, S. W., Wynne, C. D. L. (2004). Do animals think? & Bannerman, D. M. (2004). Action sets and Princeton, NJ: Princeton University Press.

CHAPTER 22Reinventing Primate Neuroscience for theTwenty-First CenturyTodd M. PreussThe manner in which an organism perceives and relatively uniform across species, an idea thatresponds to features of its environment reflects can be traced to the influential work of Karlthe state of the neural systems that make up its Lashley (Lashley, 1931, 1949), who famouslybrain. If we want to address behavior from a argued that most of the cortex is functionallyperspective that is meaningful in terms of both ‘‘equipotential.’’ Subsequent work has refinedthe mechanisms of behavior and evolutionary the allometric treatment of brain-size data, buthistory, we have to consider the organization of the core premise of the enterprise—that brainthe neural systems that constitute brains, how organization is conserved and scalable across athat organization varies across individuals and wide variety of mammalian species—has largelyacross species, and how that organization gone unchallenged. Ralph Holloway, with hischanged over evolutionary time. insistence that evolution could reorganize As a group, primatologists have paid rela- brains internally, so that brains of the same sizetively little attention to evolutionary changes in could have different functional capacities, hasthe internal organization of brains, focusing been a conspicuous exception (Holloway, 1966).instead on changes in size. Size has virtues as a To me, Holloway’s critique has alwaysscientific variable: It can be measured in both seemed eminently sensible. The molecular, cel-extant and extinct species, and it is unquestion- lular, and histological organization of the cere-ably easier to measure the size of a brain than to bral cortex, and its patterns of neuronalexamine its contents. Moreover, there have connectivity, are incredibly complex, and soseemed good reasons to think that mammalian there would seem to be enormous opportunitybrains (or primate brains, anyway) are just larger for evolutionary change: Why, in the service ofor smaller versions of a common brain, and that modifying behavior, shouldn’t evolution havesize itself is what one really needs to study in tweaked the membrane potentials of cells, fororder to understand the relationship between instance, or built connections between neuronsthe evolution of the brain and its functional that didn’t previously communicate, or reorga-capacities. This view was systematized by Harry nized cell ensembles to serve novel cognitiveJerison, in his classic synthesis of brain-size stu- functions?dies (Jerison, 1973), and it established the model Yet, it has to be admitted that if these kinds offor primatological/anthropological approaches changes did take place in mammalian evolution,to brain evolution. Jerison’s approach was the neuroscience community has done a verygrounded in the idea that the internal organiza- good job of keeping them under wraps.tion of the cortex (the largest component of the Modern experimental neuroscience is stronglybrain in primates and many other mammals) is focused on the apparent similarities between422

REINVENTING PRIMATE NEUROSCIENCE FOR THE TWENTY-FIRST CENTURY 423taxa. The cortex, in particular, has come to be Furthermore, and not surprisingly, the varia-regarded as essentially invariant in its internal tions in cortical organization are not random,organization across species, and has been said to but follow phylogenetic patterns: The internalhave a ‘‘basic uniformity of structure’’ (Rockel organization of cerebral cortex in bushbabieset al., 1980) and a ‘‘generality of functional orga- and rhesus macaques—two primates that differnization’’ (Creutzfeldt, 1977). In considering the markedly in brain size—is much more similarappeal of these claims, it is important to bear in than in bushbabies and tree shrews—animalsmind that most neuroscientists work with a very with brains about the same size as bushbabies,limited range of taxa, typically one or a few of the but which are not primates. This is not to sayaccepted ‘‘model’’ species, which today means that there aren’t features of cortical organizationprimarily rats or mice. If you study rodent that are widely shared among mammals bybrains, you have every incentive to believe that virtue of common ancestry (e.g., the existencethere are no important brain characteristics of of some manner of columnar and laminar dis-other mammals (including humans) that are not tribution of cortical neurons), but the idea thatpresent in rodents. And if most of your collea- there was an ancestral organization from whichgues also study rodents, who’s to say you’re modern forms have descended with modifica-wrong? As one speaker I heard recently put it, tion is a very different proposition than thewe can use rats as experimental stand-ins for claim that important features of cortical organi-humans because, after all, ‘‘brains are brains.’’ zation are invariant across mammalian groups,The generality of results obtained in model spe- which is the main point of ‘‘basic uniformity.’’cies is simply taken for granted and is not Accepting the fact that cortical organizationregarded as being in need of empirical validation varies across changes one’s view of what students(Logan, 1999; Preuss, 2000a). of primate brain evolution should be doing. For Nevertheless, if one sheds the blinders of the one thing, it means that a primary or exclusivemodel-animal paradigm and looks in detail at focus on brain size will not do, for if the internalthe primary data available from a variety of organization of brains varies in important ways,mammals, it is quite apparent that there is then trying to understand the functional capa-remarkable phyletic variation in the organiza- cities of different species by measuring endocra-tion of cerebral cortex: The claim of ‘‘basic uni- nial capacity is like trying to understand whatformity’’ is simply false. These data have been different computers can do by measuring theextensively reviewed elsewhere (e.g., DeFelipe boxes they came in. To be sure, you might learnet al., 2002, 2007; Elston, 2007; Hof & something by a comparison of computer boxes ifSherwood, 2007; Preuss, 1995, 2000, 2001; you start out with a good understanding of theRilling, 2006; Sherwood & Hof, 2007; architecture of computers, and if the computersSherwood et al., 2003), and these reviews paint are two versions of the same model. But if youa very clear picture: Cortex varies across mam- don’t have that kind of knowledge, you’re notmalian taxa at just about any level or dimension going to get very far.of organization one cares to consider, from Acknowledging that internal brain organiza-transmitters and receptors; to cell types, cell tion varies means that we must document thosemorphologies, and cell numbers; to the ways in variations in a systematic way and apply thewhich different cell types are organized into modern analytic tools of comparative biology tolayers; to the ways cells in different layers are reconstruct brain evolution based on those varia-interconnected to form local processing archi- tions. That is, we need to know how brain orga-tectures; to the ways local cell assemblies are nization varies across taxa and to map thoseorganized into modules and areas (the larger- variations onto the phylogenetic tree. Doingorder histological units of the cortex); to the this, it is possible to reconstruct the brain organi-ways areas are assembled into higher-order zation of ancestors, which are represented at theinformation-processing systems through pat- internal nodes of the phylogenetic tree, and to useterns of long intracortical connections. ancestral organizations as starting points for

424 PRIMATE NEUROETHOLOGYassessing evolutionary changes in daughter from that of apes or other primates, apart fromlineages. These analytic techniques are now well the fact that it is extremely large (Crick & Jones,developed and widely applied in fields of research 1993; Preuss, 2000b, 2006).as diverse as molecular biology and ethology; The task of addressing primate brain evolu-indeed, the burgeoning science of comparative tion at such fine levels of neurobiological andgenomics, which is founded on the identification phylogenetic detail might seem daunting, if notof homologous gene sequences in different taxa, impossible. It is important to recognize, there-would scarcely be possible without the modern fore, that there is at least one area of research intools of phylogenetic analysis. which neuroscientists have been equal to the An emphasis on studying the internal orga- task: Primate visual systems have been examinednization of brains also implies a primary in considerable detail from a comparative stand-emphasis on the study of living (extant) taxa. I point, and it is possible to reconstruct manydon’t mean by this to suggest that paleoneur- features of the primate visual system evolution.ology is futile, but rather to assert that what we I want to review this matter in some detail,can expect to glean from paleoneurological stu- because it illustrates both the promise of thedies of extinct species will depend on what we science and the limitations that must be over-know about the actual brain biology of their come in order to obtain comparably detailedclosest living relatives. In addition, we need to analyses of other brain systems.pick and choose the extant taxa to be studied If we are unusually well informed about thevery carefully. Since we can’t study every species evolution of primate visual systems, it is becausein great detail, we must choose species that max- visual science has a particularly strong compara-imize the amount of information they provide tive tradition and because the influential work ofabout evolutionary history. Certainly, we want Le Gros Clark (1959) stressed the importance ofto identify the shared, derived characteristics of vision in primate evolution. Le Gros Clark’sat least the major branches of the primate tree— banner was taken up by Diamond and his col-strepsirrhines (lemurs, lorises, and bushbabies), leagues, who helped promote studies of treetarsiers, platyrrhines (New World monkeys), shrews and bushbabies (e.g., Diamond & Hall,and catarrhines (Old World monkeys, apes, 1969; Harting et al., 1972, 1973), studies thatand humans), so we need to study members of have been essential for understanding primateeach of these groups. Understanding what fea- visual system evolution. This rich history oftures of brain organization distinguish primates study makes it possible to identify likely primatefrom other mammals requires outgroup evi- specializations at almost every level of visualdence, and thus, in addition to studying pri- system organization (for comprehensive reviewsmates, was also need to compare primates to of the primary literature, see Allman, 1977;other mammals, especially the groups of mam- Casagrande & Kaas, 1994; Casagrande et al.,mals most closely related to primates, which, on 2007; Kaas & Preuss, 1993; Preuss, 2004,current evidence, include tree shrews (mamma- 2007a,b; Rosa & Krubitzer, 1999; Ross &lian order Scandentia) and colugos (i.e., flying Martin, 2007). One of the first and most exhaus-lemurs; order Dermoptera) (e.g., Murphy et al., tively documented primate specializations is the2001). Of course, the characteristics of the distinctive pattern of projections from the retinahuman lineage and of the hominoid (ape) to the superior colliculus, whereby each colli-clade of which our lineage is a part are culus represents only the contralateral visualmatters of special interest and importance. field. The unusual lamination in the thalamicUnderstanding this part of primate evolutionary lateral geniculate nucleus (LGN) of primateshistory requires comparing humans and apes, reflects a pattern of input segregation differentand particularly the ape species to which we from other mammals. In primates, the LGNhumans are most closely related: chimpanzees projects nearly exclusively to a single corticaland bonobos. Currently, we know very little area, the primary visual area (V1; also knownabout what distinguishes the human brain as striate cortex), whereas in other mammals

REINVENTING PRIMATE NEUROSCIENCE FOR THE TWENTY-FIRST CENTURY 425that have been studied, LGN projections are could look like, the unhappy fact is that wemore widespread. One consequence of this is known rather little about most other regions ofthat higher-order visual areas are more depen- the brain. We know that primates vary in thedent upon V1 for driving physiological activity in organization of somatosensory and motorprimates than they are in nonprimates. Within cortex in ways that parallel changes in manualarea V1 of primates, there is a distinctive pattern reaching, grasping, and manipulative abilitiesof compartmentation in the upper layers, such (Hinkley et al., 2007; Kaas, 2007; Padberg et al.,that inputs that ultimately arise from the W class 2007; Wise, 2007). We have evidence that pri-of cells in the retina terminate in small patches mates evolved new motor areas along with a(‘‘blobs’’) that are rich in cytochrome oxidase, a large suite of new areas in the classical higher-metabolic enzyme. Blobs are absent in tree order association regions of the frontal, parietal,shrews, rodents, and other mammals, with the and temporal lobes (Preuss, 2007a,b). We alsoexception of carnivores, in which they evolved know of changes in cortical cell biology, his-independently. Primates also possess a large tology, and receptor distribution, although innumber of higher-order visual areas, as do some most cases the comparative coverage has notother mammals, but only in primates are these been sufficient to make possible detailed phyleticareas organized by their interconnections into reconstructions. Many aspects of primate braintwo distinct processing streams, a dorsal pathway organization remain to be explored from a com-related to spatial attention and action and a ven- parative perspective.tral pathway related to object recognition. Why can’t we simply extend the experimental In addition to the shared, derived features of approaches that have proven so successful inprimates, we also know something about the documenting the evolution of the visual systemvisual-system specializations of particular pri- to other systems? Consider that much of themate groups. Early in the radiation of the hap- comparative visual system research has involvedlorine primates (the group that includes tarsiers invasive and terminal experimental procedures,and anthropoids), there was adaptation to a and that to be applied on anything more than adiurnal activity pattern and the retina was reor- small scale, these procedures would require anganized, with the evolution of a true fovea and infrastructure of animal colonies housing a tax-with the replacement of rods in the central retina onomically diverse set of primate species andby an abundance of miniaturized cones. These close primate relatives. In the United States,changes improved spatial resolution in bright where concentration of research around a fewlight and created the conditions required for oppo- ‘‘model’’ species seems to be a much highernent interactions between cones, which make color priority than maintaining a diversity of researchdiscrimination possible. Independent duplication species, the number of primate species (andof the ancestral medium/long-wavelength cone other species relevant to primate evolution)opsin in the New World and Old World anthro- available for research is declining: The pastpoid led to the evolution of trichromatic vision in decade has seen the elimination of all the apethese groups. We know of additional modifica- species from the National Primate Researchtions of the laminar organization of area V1 (in Centers with the exception of chimpanzees,hominoid evolution) and of the compartmental and the National Committee for Researchorganization of area V1 (in human evolution), Resources (NCRR; the branch of the Nationalbut the functional implications of these changes Institutes of Health [NIH] that supports theare as yet poorly understood. National Primate Research Centers) has decided to maintain a moratorium on breeding the chimpanzees it holds (Cohen, 2007a), whichTHE NEW NEUROSCIENCE means that the chimpanzees will eventually beTECHNOLOGIES eliminated, too. The few remaining breedingIf the visual system gives us a glimpse of what a colonies of critically important taxa like galagosmature comparative neuroscience of primates and tree shrews, historically supported by NIH

426 PRIMATE NEUROETHOLOGYfunding, are unlikely to be maintained. To the because of their intrinsic magnetic properties andextent that nonhuman primates will continue to because they are the most abundant nuclei in thehave a role in a neuroscience based on tradi- brain, so the most commonly used MRI techni-tional experimental methods, that future ques are based on proton imaging. In the presencebelongs mainly to a few macaque and baboon of a strong background magnetic field, protonsspecies and to marmosets. If we continue to rely align their spin axes along that field; in fact, theirprimarily on experimental approaches requiring spin axes precess around a central axis much likeinvasive and terminal procedures, it seems unli- the wobble of a spinning top. External electromag-kely that we will have access to the diversity of netic pulses tuned to appropriate ‘‘resonant’’ fre-taxa required to reconstruct primate brain quencies that depend on the strength of theevolution. background field cause the axes to flip out of the Does the science of primate brain evolution plane of the background magnetic field and simul-have a future? If it does, it will have to come to taneously synchronize the precession phases of theterms with the degradation of the animal individual nuclei. Following the EM pulse, theresources upon which comparative brain studies protons relax, their precessions dephasing at anhave heretofore relied. One way to do this is to exponential rate and their axes returning to align-take greater advantage of the variety of new ment with the background field at an exponential‘‘low-impact’’ techniques available to neuros- rate. As protons relax, they emit EM waves at theircientists, techniques that either involve nonin- resonant frequencies. The rate of return to align-vasive or minimally invasive methods for ment with the background field is known as T1;studying living organisms or employ biological the rate of dephasing of precessions is known asmaterials that can be collected without harming T2. Different tissues have different T1 and T2the individual or that can be collected after values, values that depend on the concentrationdeath. These include neuroimaging, histological of hydrogen and on the organization of differenttechniques, and genomic and other molecular hydrogen-containing molecules in the tissue. Thisbiological methods. provides the basis for the tissue contrast one sees in MR images, such as, for example, the contrastComparative Neuroimaging between gray matter and white matter in the brain. Typical structural MRI scans reflect both T1 andNeuroimaging is a set of methods for remotely T2 but weight them differently (Fig. 22.1A).probing the structure, chemistry, and physiology Increasing the strength of the backgroundof the brain. These include magnetic resonance magnetic field increases the signal-to-noiseimaging (MRI), positron emission tomography ratio of the MR signal obtained from individual(PET), computer-assisting tomography (CAT), brain volume elements (voxels), which meansnear-infrared spectroscopy, electroencephalo- that stronger magnets make it possible tography (EEG), and transcranial magnetic stimu- obtain higher spatial resolution scans withoutlation (TMS). Currently, MRI and PET offer the sacrificing image quality. Current clinical scan-greatest potential for comparative neuroima- ners are generally 1.5 to 3 Tesla (T), and aging, and will be the focus of discussion here. It number of universities now have experimentalis convenient to distinguish structural and func- research scanners of 9.4T or more. Early scan-tional modes of neuroimaging, categories that ners, with low field strength, were best suited forcut across the MRI/PET distinction. the study of large species, such as humans. Indeed, MRI is unusual among neuroscientificStructural Imaging with MRI techniques in that its first large-scale applica-MRI works by using magnetic fields and electro- tions were in humans, proving valuable as amagnetic (EM) pulses to manipulate the spins of diagnostic tool for localizing tumors, strokes,atomic nuclei, and thereby induce them to emit and aneurisms, as well as for carrying out mor-electromagnetic signals. The strongest signals can phometric studies of human brain structure.be elicited from hydrogen nuclei (protons), both The progressive improvement of the technology

REINVENTING PRIMATE NEUROSCIENCE FOR THE TWENTY-FIRST CENTURY 427Figure 22.1 T1- and T2-weighted structural neuroimaging. (A) T1-weighted coronal image from achimpanzee acquired in vivo with a 3T scanner. (B) T2-weighted images from the fixed brain of an aye-aye (Daubentonia madagascariensis) acquired in a 9T scanner. The image sets in B were reconstructed toshow the intersection of a coronal plane and a parasagittal plane. T1- and T2-weighted images showdifferent patterns of tissue contrast: Gray matter is darker than white matter in T1, whereas white matterappears very dark in T2-weighted images, which resemble myelin-stained histological sections. T1-weightedscans are commonly used for rapid acquisition of structural images in vivo with high spatial resolution.Fixed brains can be scanned for very long periods, however, and T2-weighted images can be acquired withvery high spatial resolution and excellent contrast between gray matter and white matter, showingremarkable anatomical detail even in the relatively small brain of this lemur. Figure 22.1B from Kaufman,J. A., Ahrens, E. T., Laidlaw, D. H., Zhang, S., & Allman, J. M. (2005). Anatomical analysis of an aye-ayebrain (Daubentonia madagascariensis, primates: Prosimii). combining histology, structural magneticresonance imaging, and diffusion-tensor imaging. Anatomical Records Part A: Discovery of Molecular,Cellular, and Evolutionary Biology, 287, 1026–1037. Used with permission.of MRI has made it possible to extend MRI (Rilling & Seligman, 2002; Schenker et al.,morphometry to species with smaller and 2005; Schoenemann et al., 2005) and of thesmaller brains, so that MRI has become an major fiber bundles interconnecting the twoimportant tool in comparative neuroscience. hemispheres, the anterior commissure andMoreover, while we usually think of MRI as an corpus callosum (Rilling & Insel, 1999a); asin vivo technique, it can be used with fixed well as studies of cerebral asymmetries of corticalbrains as well and therefore can be used to mass and sulcal morphology (Gilissen, 2001;study archival brain material (Fig. 22.1B). Hopkins et al., 1998). To date, MRI morpho- There is already an extensive literature of metry has been used mainly to study hominoidscomparative MRI morphometry. Examples and anthropoids and other large-brain creatures,include studies comparing the absolute and rela- but it is certainly possible to image smaller-braintive volumes of cerebral lobes among hominoids animals. One of the advantages of postmortem(Rilling & Insel, 1999b; Rilling & Seligman, 2002; imaging is that you can fit relatively small brainsSemendeferi et al., 1997, 2002); the volumes of into high field-strength scanners (7T or higher)the cerebellum and its constituents among pri- and scan them for a long time to obtain very highmates and between primates and cetaceans resolution, as, for example, in the recent study of(MacLeod et al., 2003; Marino et al., 2000; a Daubentonia brain by Kaufman et al. (2005).Rilling, 2007; Rilling & Insel, 1998); the sizes of (Fig. 22.1B).white matter systems of the brain, including the Despite these technical improvements, thewhite matter of the different cerebral lobes study of certain major constituent structures

428 PRIMATE NEUROETHOLOGYof the brain, notably cortical areas and thalamic There are at least two additional variants ofnuclei, has proven difficult with current MRI MRI that are potentially extremely valuable fortechniques. Identifying and demarcating (‘‘seg- comparative studies. One is diffusion-tensormenting’’) these structures is important, imaging (DTI), the attraction of which lies inbecause unlike whole lobes and hemispheres, its ability to map fiber tracts noninvasively, andthese are real functional units of the brain. thus overcome (to some extent) the limitationsSulci, unfortunately, are not very reliable of standard connectivity techniques. Sinceguides to the borders of cortical areas in cross- brains are collections of neural systems, consti-species comparisons (Preuss, 2007b). The pro- tuted by the interconnections of neurons andblem of segmenting cortical areas and nuclei in neuron-containing structures, understandingMRI is compounded by the fact that the orga- connectivity and how it varies across species isnization of even well-studied taxa like rhesus fundamental to comparative neuroscience. Themacaques is still not completely worked out. most sensitive and reliable experimental techni-The widespread reliance on Brodmann’s map ques for tracing connections in the brain haveof human cortical areas (Brodmann, 1909) in involved the injection of chemical substancesfunctional MRI studies suggests a level of that are taken up by neurons and transportedknowledge about human cortex that we just either proximally (back to the cell body) or dis-do not have: Brodmann’s map is inaccurate in tally (out to the terminations of axons) or both.important respects (Abbott, 2003; Eickhoff Injecting the substances involves invasive sur-et al., 2005a; Toga et al., 2006). Moreover, gical procedures; confirming the location of theresearchers typically assign Brodmann’s num- injection site and determining where the tracerbers to loci of functional activity based solely on was transported to requires sacrificing the sub-their coordinates in one of the standardized ject and examining the brain microscopically.brain spaces currently available for human Obviously, these techniques are inappropriatebrain mapping, rather than on an independent for use in humans, or in rare or endangeredassessment of the histological architecture of species, such as chimpanzees. As a result, mostsubjects’ brains—precisely because it has been of what we think we know about the connec-difficult to do something like a cytoarchitec- tivity of the human brain represents extrapola-tonic or myeloarchitectonic parcellation of the tion from experimental studies in macaquecortex using MRI. One approach to resolving monkeys rather than results obtained directlythis limitation is to make architectonic maps from studies of human beings (Crick & Jones,from brains that have been scanned, then sec- 1993). DTI provides the opportunity to studytioned and analyzed histologically, and then connectivity directly in humans, and can be usedregistered back into standardized brain space; in other species as well (Fig. 22.2).individual subjects’ MRI scans can then be DTI is based on the ability of MRI to detectregistered into the same space, and cortical the direction of diffusion of water molecules (seeareas identified on a probabilistic basis reviews by Mori & Zhang, 2006, and Ramnani(Eickhoff et al., 2005b, 2007b). Ultimately, it et al., 2004). Since water molecules tend to dif-may prove possible to make individualized fuse along, rather than across, nerve fibers, localbrain maps: Current imaging techniques can anisotropies in water diffusion can be used todetect variations in the strength of the MR reconstruct the three-dimensional organizationsignal across the laminae of the cortex, and in of fiber bundles (Kaufman et al., 2005). DTI canprinciple, these intensity variations could be be done in vivo, but also with fixed, archivalused to distinguish cortical areas just as laminar brain tissue. Studies of well-preserved tissuevariations in cell and fiber staining are used to yield results similar to in vivo studiesdistinguish cortical areas in classical histolo- (D’Arceuil et al., 2007; Guilfoyle et al., 2003;gical architectonic mapping (Clare & Bridge, Sun et al., 2003), because DTI is based on the2005; Clark et al., 1992; Eickhoff et al., 2005a; physics of water diffusion, rather than on theWalters et al., 2003). biological activity of living tissue.

Figure 22.2 Diffusion-tensor imaging (DTI) can be used to reconstruct white matter pathways. A–C illustrate theprinciples of DTI in the brain of Daubentonia, as demonstrated by Kaufman et al. (2005). (A) A histological sectionthrough the cingulum bundle (CB), corpus callosum (CC), and caudate nucleus (Cd) stained for myelin. The cingulumbundle and corpus callosum are major fiber tracts, and therefore are stained darkly; the caudate, a gray matter structure,has a light appearance. (B) A ‘‘color map’’ at the same level, obtained from DTI scans obtained before the brain wassectioned. DTI color maps code the principal direction of fibers in each brain voxel. In this map, red representsanteroposteriorly oriented fibers, green represents fibers oriented superoinferiorly, and blue represents mediolaterallyoriented fibers. The prominent blue territory corresponds to the corpus callosum, which carries fibers passingmediolaterally between the two hemispheres. The cingulum bundle, which carries fibers that pass between the frontallobe and posterior cortical regions, starts out red at posterior levels and then turns green as it turns inferiorly anterior tothe genu of the corpus callosum. (C) Histological confirmation of the DTI results: Plots of fiber directions observed inmyelin-stained sections at different levels of the cingulum bundle match the directionality indicated by the DTI results inB. (D) Results of a DTI study comparing the course and targets of the arcuate fasciculus in the cerebral hemispheres ofhumans and chimpanzees, based on Rilling et al. (2008). These maps illustrate fiber pathways reconstructed fromprincipal-direction data. Note that in these figures, all fibers and terminations are rendered as shades of green, whatevertheir orientation. In humans, the arcuate fasciculus interconnects language areas of the inferior frontal lobe and parietaland temporal cortex. Chimpanzees also possess an arcuate fasciculus, but it lacks the strong connections with thetemporal lobe seen in humans. In humans, the relevant temporal areas are involved in the representation of wordmeaning. abSF, ascending branch of the Sylvian fissure; AOS, anterior occipital sulcus; CS, central sulcus; FOS, fronto-orbital sulcus; hbSF, horizontal branch of the Sylvian fissure; IFS, inferior frontal sulcus; IPS, intraparietal sulcus; ITS,inferior temporal sulcus; LCaS, lateral calcarine sulcus; LuS, lunate sulcus; PoCS, postcentral sulcus; PrCS, precentralsulcus; SF, Sylvian fissure; SFS, superior frontal sulcus; STS, superior temporal sulcus. A–C from Kaufman, J. A., Ahrens,E. T., Laidlaw, D. H., Zhang, S., & Allman, J. M. (2005). Anatomical analysis of an aye-aye brain (Daubentoniamadagascariensis, primates: Prosimii) combining histology, structural magnetic resonance imaging, and diffusion-tensor imaging. Anatomical Records Part A: Discovery of Molecular, Cellular, and Evolutionary Biology, 287, 1026–1037.D based on Rilling, J. K., Glasser, M. F., Preuss, T. M., Ma, X., Zhao, T., Hu, X., et al. (2008). The evolution of the arcuatefasciculus revealed with comparative DTI. Nature Neuroscience, 11, 426–428. Used with permission. 429

430 PRIMATE NEUROETHOLOGYFigure 22.2 (Continued) DTI tracing has some important limitations useful tool for segmenting the areas and nuclei ofcompared to traditional tracer studies. The spatial the brain (Behrens & Johansen-Berg, 2005),resolution of DTI is relatively coarse (with voxel because brain areas differ in their patterns ofsizes usually in the range of 0.5 to 2.0 mm), so at connectivity as well as their histologicalpresent this is not a method that can trace connec- organization.tions from cell to cell. In addition, accurate recon- Functional Imagingstruction of fiber tracts depends on the coherenceof fiber bundles, so DTI tracks quite well through Neuroimaging can also be employed to monitorthe white matter, but less well as fibers pass into the functional activity of living brains, using agray matter, where fibers disperse and travel in variant of the MRI technique, functional MRImultiple directions. Finally, DTI is vulnerable to (fMRI), and with PET. Both approaches areboth false positives and negatives, so that currently based on the fact that changes in neural activitythe best use of DTI is not to provide an exhaustive are accompanied by changes in regional bloodaccounting of the structures with which a given flow and glucose utilization. Functional imagingbrain region is connected (which is what tracer maps these regional metabolic changes.studies provide), but rather to assess the prob- Functional MRI does this by tracking changesability that fibers pass between a specific region in blood flow; as brain regions increase orof interest and other voxels in the brain. decrease their activity level, and blood flowNotwithstanding these limitations, DTI is increases or decreases accordingly, the relativeproving to be extremely valuable as a noninvasive amounts of oxygenated and deoxygenatedtool for tracing connections, and it is now being hemoglobin in those regions change: Regionsused to directly compare humans and nonhuman with high blood flow have higher levels of oxy-primates and thus to shed light on human brain genated hemoglobin. The change can beevolution. Recent studies of note include compar- detected with MRI because deoxygenated andisons of frontal lobe connections with subcortical oxygenated hemoglobin have different magneticstructures (Croxson et al., 2005; Ramnani et al., properties.2006; Tomassini et al., 2007); comparisons of Whereas MRI works by the external applica-the interconnections of higher-order cortical tion of magnetic fields and EM pulses, PETregions in humans, chimpanzees, and macaques derives information by introduction into the(Rilling et al., 2008); and comparisons of amyg- subject (orally or by injection) of radioactivelydala connectivity (Hecht et al., 2008). The ability labeled substances that accumulate in the brain.of DTI to track fiber pathways also makes this a These substances incorporate short half-life,

REINVENTING PRIMATE NEUROSCIENCE FOR THE TWENTY-FIRST CENTURY 431positron-emitting isotopes. An emitted positron (FDG), a radiolabeled glucose analog, this PETtravels only a few millimeters on average before technique has been used in humans (who are notencountering an electron and the two are anni- usually anesthetized during scanning), as well ashilated, yielding a pair of photons traveling in in nonhuman primates (Rilling et al., 2001,nearly opposite directions. These photons reach 2004, 2007). PET is currently the only techniquedetectors arrayed around the subject, and by available for directly comparing behavior-comparing the location of the detector elements related patterns of brain functional activity inactivated and the time required to reach the humans and chimpanzees (Fig. 22.3).detector array, their points of origin can bedetermined. In this way, regional variations in Functional Connectivitythe accumulation of the radiolabeled substance The need to anesthetize nonhuman subjectscan be reconstructed. PET can be used as an during scanning means that PET, rather thananatomical tool, for example, to determine the fMRI, is in most cases the practical optiondistribution of specific receptor types using for comparing the relationship betweenappropriate ligands, but its principal use has regional brain activity and behavior acrossbeen to determine function-related differences primate species. Functional MRI does have ain regional brain activity using radiolabeled use in comparative studies, however. Regionsoxygen isotopes or glucose analogs. of the brain that are coupled synaptically While widely used in human studies, func- tend to have temporally correlated patternstional imaging has found rather limited applica- of metabolic activity: They tend to be activetions in studies of nonhuman primates. In part, in synchrony. Functional MRI, which mea-this is because subject movement degrades the sures levels of functional activity in indivi-images obtained in MRI and PET scanners. dual brain voxels, provides data that can beMost human subjects can restrain their head used infer patterns of connectivity. This tech-movements upon request even as they make nique, called functional connectivity MRI (orfinger taps or other motor responses in behavioral fcMRI), is now being used extensively totasks, but awake animals need to be extensively study humans (e.g., Biswal et al., 1995; Foxtrained, as well as physically restrained, in order et al., 2005; Hampson et al., 2002; Lowe etto be suitable as subjects. This can be done in al., 1998; Seeley et al., 2007; Xiong et al.,some cases, mainly with highly trained, chair- 1999). Typically, human subjects rest quietlyrestrained macaques (e.g., Howell et al., 2001; in the scanner while MRI records sponta-Logothetis et al., 1999), but scanning awake, neous brain activity; the subjects are not per-unrestrained individuals is not a practical option forming any particular task. Interestingly,for most nonhuman primate species. Certainly, much of this spontaneous brain activity isyou do not want to try to scan a powerful animal retained in shallow sleep and light anesthesialike an adult chimpanzee unrestrained in your (Peltier et al., 2005), and as a result, themultimillion-dollar MRI or PET facility. technique has been employed to compare One way to escape this limitation is to sepa- humans and macaques (Vincent et al.,rate the time period during which the subject is 2007). It should be possible to extend it toawake and behaving and taking up a radiola- other nonhuman primate species as well. Thebeled marker, from the time period of PET scan- fcMRI technique could prove an importantning, in which the distribution of the ligand in complement to DTI for providing informa-the brain is determined. Provided the tracer tion about connectivity for species in whichaccumulates to an asymptotic level during the traditional, invasive methods cannot be used.behavioral testing period and doesn’t declinesignificantly during the time required to obtain Comparative Histology and Cell Biologythe PET scan, it makes little difference whetherthe PET scan is done with the subject awake or Neuroimaging gives us a window into the brain;anesthetized. Using [18F]-fluorodeoxyglucose histological and cell-biological techniques take

432 PRIMATE NEUROETHOLOGYFigure 22.3 Comparative functional neuroimaging of resting-state cognitive activity using the in vivofludeoxyglucose-positron emission tomography (FDG-PET) technique in humans and chimpanzees, basedon Rilling et al. (2007). In humans, numerous studies have shown that specific regions of cortex are highlyactive when subjects simply rest in the scanner and allow their thoughts to wander freely, including themedial frontal cortical (MFC) areas representing self-referential and emotional processes, medial posteriorcortical (MPFC) areas involved in episodic memory, and lateral areas in the posterior parietal (PP) lobe andthe cortex in and around the posterior superior temporal sulcus (STS) involved in language and consciousawareness. Rilling and colleagues replicated previous resting-state activity results in humans, and showedthat chimpanzees exhibit patterns of activity similar, but not identical, to that of humans. The amount ofFDG incorporated into different parts of the brain is color coded; the brightest voxels denote the highestactivity. The same FDG-PET technique can be used with more structured behavioral tasks to comparefunctional brain activity across species. V, visual cortex. From Rilling, J. K., Barks, S. K., Parr, L. A., Preuss,T. M., Faber, T. L., Pagnoni, G., et al. (2007). A comparison of resting-state brain activity in humans andchimpanzees. Proceedings of the National Academy of Sciences, 104, 17146–17151. Used with permission.us into the very substance of the brain, and at and cell compartments, new methods of quanti-scales of organization that can be smaller than tative analysis, and new techniques for preser-those achievable in normal practise with neuroi- ving tissue.maging (Balter, 2007). Because tissue studies are Scientists in the nineteenth century discoveredas old as experimental biology itself and almost that different components of neural tissue couldalways involve studies of tissue obtained post- be distinguished by chemical treatment of themortem, one might not imagine this to be a tissue; these histochemical methods includehotbed of new methods and an avenue to new stains that reveal cell bodies (e.g., the Nissldiscovery about the brain. In fact, the amount stain) and myelinated axons. New histochemicaland quality of information that can be gleaned stains were developed in the twentieth century,from brain tissue has increased dramatically over including stains for enzymes that are highly abun-the past several decades, driven by the introduc- dant in the nervous system, such as cytochrometion of new techniques for labeling specific tissue oxidase and acetylcholinesterase. These staining

REINVENTING PRIMATE NEUROSCIENCE FOR THE TWENTY-FIRST CENTURY 433methods are still widely employed in experi- distribution across brain regions can be usedmental neuroscience. Increasingly, however, neu- to segment (or ‘‘parcellate,’’ in the argot ofroscientists are making use of a rapidly growing architectonics) the brain. So, for example,set of ligand molecules that bind selectively to simple histochemical stains for cytochromespecific molecules of interest in the brain. Since oxidase and acetylcholinesterase have beendifferent molecules tend to be localized within used to map cortical areas (e.g., Stepniewskaparticular cell types (e.g., pyramidal cells, inter- et al., 1993; Tootell et al., 1985), and with theneurons, astrocytes) and to particular compart- proliferation of antibodies and other ligands, aments within those cell types (e.g., nucleus, cell number of these have proven useful for sub-body, synaptic spines), mapping the distribution dividing the cortex and subcortical brainof specific molecules can tell us a great deal about regions. Investigators try to combine evidencethe cellular and subcellular organization of brain from multiple sources to obtain reliable andtissue. The ligands employed in these studies meaningful parcellations (e.g., Kaas, 1987;include naturally occurring chemicals, including Toga et al., 2006). For example, my colleaguesa variety of plant lectins that have affinities for the and I found that a plant lectin, Wisteria flour-carbohydrate moieties of proteins, and synthetic abunda agglutinin, and an antibody to a cal-compounds, including neuroactive drugs that cium-binding protein, calbindin, are bothmimic neurotransmitters, binding to specific useful tools for distinguishing nuclei of thereceptor molecules on neurons and glia. But by thalamus (Cola et al., 2005; Preuss et al.,far the most widely used ligands are antibodies. 1998), and that an antibody to neurofilamentAntibody production for research and clinical use protein (SMI-32) is valuable for distinguishingis now a large industry, turning out antibodies to subdivisions of motor cortex (Preuss et al.,a substantial fraction of the proteins present in 1997). Our experiences with these ligands arethe human brain. So, in a typical immunohisto- paralleled by those of many other workers, andchemical (or immunocytochemical) procedure, the use of lectins and antibodies is now aone bathes tissue sections in a solution containing standard feature of brain mapping in humansantibodies to a protein of interest, then immerses and nonhuman primates. Karl Zilles andthe sections in a second solution that tags the his coworkers have made extensive use ofprimary antibody with a marker that can be another class of ligands, synthetic radiolabeledvisualized under a microscope, such as a fluores- receptor-binding substances, in comprehensivecent dye or a chemical that can be processed to human brain mapping studies (for reviews, seeyield a visible precipitate, typically diaminobenzi- Amunts & Zilles, 2001; Eickhoff et al., 2007a;dine (DAB). Zilles et al., 2002). This impressive body of What can we do with these methods? One work should eventually supplant Brodmann’suse is to segment the gray matter into structu- (Brodmann, 1909) widely employed, butrally and functional meaningful subunits. Since deeply flawed, map of human cortex (Abbott,gray matter, in its natural state, tends to be 2003). The same ligands can be used in com-quite homogeneous in appearance, histologists parative studies (e.g., Zilles et al., 1996),have explored chemical procedures that differ- although to date they have not been widelyentially stain or label subunits of gray matter. employed in this capacity. With the develop-Classically, the histochemical stains for Nissl ment of new techniques for differentiallyand myelin were employed to try to subdivide labeling brain regions have come new quanti-gray matter based on regional differences in the tative analytic techniques for identifying bor-size and density of cell bodies (as in the ders between regions (e.g., Annese et al., 2004;familiar work of Brodmann, 1909) and the Geyer et al., 1996; Hackett et al., 2001).density and geometric pattern of myelinated Histological techniques have much more tofiber distribution (as, for example, by Vogt & offer to comparative neuroscience than seg-Vogt, 1919). In principle, however, any mole- menting major brain divisions, as important ascule that varies in its density or spatial that task may be. They also provide clues to

434 PRIMATE NEUROETHOLOGYphyletic differences at finer levels of organiza- results in the overestimation of cell numbers andtion, of which there are many examples (for densities. This problem of ‘‘lost caps’’ has longreviews, see DeFelipe et al., 2002, 2007; Hof & been recognized, and various correction factorsSherwood, 2007; Preuss, 1995, 2001; Sherwood & were developed to mitigate the problem, most ofHof, 2007). We know of differences in the which now appear to be inadequate. Beginning inlaminar organization and compartmental orga- the 1960s, new and simple techniques have beennization of homologous cortical areas, as exem- developed that resolve the lost-caps problem,plified by the modifications of layer 4A of which, along with efficient sampling schemes,primary visual cortex that occurred in ape and yield unbiased and reliable estimates of the num-human evolution (Preuss & Coleman, 2002; bers and densities of cells and other small parti-Preuss et al., 1999) (Fig. 22.4). There are differ- cles. These so-called ‘‘design-based stereology’’ences in the biochemical phenotypes of homo- techniques have become the standard for quanti-logous cell types: For example, layer 5 pyramidal tative studies in the neurosciences (Gundersen etcells in the cingulate cortex of great apes and al., 1988; Schmitz & Hof, 2005; West, 1993).humans express the calcium-binding protein Thus, it is only very recently that we have hadcalretinin, unlike other primates (Hof et al., estimates of neocortical neuron number for2001). The morphologies of neurons vary humans (19–23 billion, varying with age andacross taxa: great apes and humans possess a sex; Pakkenberg & Gundersen, 1997), based ondistinctive class of large, spindle-shaped layer 5 methods that are generally acknowledged to becells (Von Economo cells) in anterior cingulate accurate. Stereological techniques for unbiasedand fronto-insular cortex; these are especially estimation of cell numbers have recently beenlarge and numerous in humans, and are not introduced into comparative primate brain stu-observed in primates outside the great ape- dies by Sherwood and colleagues in their studieshuman clade (Allman et al., 2005; Nimchinsky of glia/neuron ratios in anthropoids (Sherwoodet al., 1999). Glial cell morphologies also vary et al., 2006) and of the densities of different inter-across primate taxa (Colombo et al., 2004). The neuron types in anthropoid visual cortexdendritic branching patterns of pyramidal cells (Sherwood et al., 2007). Sherwood and colleaguesdiffer between taxa (Elston, 2003), as do the have also introduced modern phylogeneticgeometries of certain classes of axonal termina- regression techniques (independent contrasts;tions (Raghanti et al., 2007, 2008). The sizes of e.g., Garland et al., 1992) to comparative studiesparticular types of neurons can vary in ways that relating variations in cell numbers and density toare not plausibly explained by differences in brain size in primates. Simple regressions can bebrain size, and presumably reflect functional confounded by the presence of different phyloge-specializations (Sherwood et al., 2003). It is netic groups, with different scaling relations andalso possible to study the local connectivity of different sample sizes, in a common analysis.brain regions in fixed tissue, over a distance of Independent contrasts provide a means to distin-several millimeters, using lipophilic dyes guish within-group and between-group contribu-(Galuske et al., 2000; Mufson et al., 1990; tions to scaling.Tardif & Clarke, 2001), but comprehensive com- It is noteworthy that most of the studies citedparative studies have not yet been undertaken previously were carried out using unperfusedwith this technique. tissue collected and fixed postmortem by One of the most important developments in simple immersion in fixative. Often in thesemodern histology is the advent of techniques for studies, furthermore, there were delays of severalaccurately estimating numbers and densities of hours postmortem before the brain wascells. Because the process of sectioning tissue removed and placed in fixative. This contrastsresults in fractions of individual cells being pre- with the usual procedures employed withsent in two or more adjacent tissue sections, experimental animals, which are overdosedmerely counting the number of cells in a section with anesthetic and then perfused transcardiallyand multiplying by the thickness of the section with fixative at the point of death, so as to

Figure 22.4 The neural architecture of homologous cortical areas can vary widely across taxa, asillustrated by recent studies of the primary visual area (area V1; area 17) of anthropoid primates. Asdemonstrated by Preuss and Coleman (2002), patterns of evolutionary change in the brain can bereconstructed in considerable detail when appropriate comparative studies are carried out, and thesechanges include hominoid (ape-human) and human specializations of layer 4A. (A)Photomicrographs of Nissl-stained sections from area V1 in four primate species: a New Worldmonkey (Saimiri sciureus, squirrel monkey); an Old World monkey (Macaca mulatta, rhesusmacaque); and two hominoids (Pan troglodytes, chimpanzee; Homo sapiens, human). These sectionsspan the full thickness of the cortex, from layer 1 to the white matter (WM). Nissl staining does notreveal dramatic differences in the organization of area V1 across anthropoid species. Scale bars are500 microns. (B) Marked differences in the histology of anthropoid V1 can be demonstrated withpreparations other than Nissl staining. These panels, which focus on layer 4A, show results using fourdifferent procedures: histochemistry for cytochrome oxidase (CO), a mitochondrial enzyme expressedat high levels in metabolically active neurons and neurites; immunocytochemistry for the calbindin(CB), a calcium-binding protein expressed primarily by interneurons; immunocytochemistry fornonphosphorylated neurofilament protein (NPNF), which is expressed strongly by pyramidal cells;and immunocytochemistry with monoclonal antibody Cat-301, which labels an extracellular matrixproteoglycan expressed primarily by interneurons. In the CO panel, note the prominent, thin band ofCO staining in layer 4A of Saimiri and Macaca, which is lacking in Pan (denoted by the filledarrowhead) and Homo. This difference may reflect evolutionary changes in the distribution of visualinputs to layer 4A (Preuss et al., 1999). In the CB panel, Saimiri and Macaca exhibit light stainingin layer 4A, while Pan (filled arrowhead) has stronger labeling. Homo has even stronger labeling, and 435

Figure 22.4 (Continued) unlike Pan, the labeling is distributed in irregular patches (open arrowhead).NPNF and Cat-301 labeling in Homo also shows a patchy pattern, with irregular darkly stained and lightlystained compartments. Double-labeling studies show that NPNF and Cat-301 densely label the samecompartments, whereas CB is concentrated in the other set of compartments (Preuss & Coleman, 2002).Pan and Homo also differ from the monkeys in having dense NPNF labeling of the neuropil in layers 2 and 3(denoted by the asterisk in Pan). The photographs are not to a common scale, each being rescaled as neededto show the four divisions of layer 4A in parallel. (C) A reconstruction of layer 4A evolution in anthropoidprimates based on published data about connectivity and histology from 13 anthropoid genera. Thebranching relationships between anthropoids depicted here is from Purvis (1995) and is relativelyuncontroversial. Different types of data are presented by an alphabetic code: (a) the presence or absenceof direct projections from LGN to 4A, (b) the presence or absence of a CO-dense band, (c) level ofexpression of calbindin in 4A, (d) pattern of expression of NPNF, and (e) the pattern of expression ofCat-301. Parsimony analysis suggests that ancestral anthropoids possessed an organization of layer 4Asimilar to that found in modern Saimiri and Macaca, in which afferents from the parvocellular layers of thelateral geniculate nucleus (P-LGN) terminate directly in layer 4A, which is reflected by the presence of adiscrete band of CO labeling in this layer. Cone-shaped clusters of pyramidal cell bodies and their apicaldendrites in layer 4B punctuate layer 4A, giving layer 4A a ‘‘honeycomb’’ appearance, when sectioned in theplane of the cortical surface. Expression of CB and of the Cat-301 antigen was probably low in layer 4A ofancestral anthropoids, and expression of NPNF and MAP2 (a microtubule-associated protein stronglyexpressed in neurons) was concentrated in the cone-like pyramidal cell clusters. Ancestral organization wasmodified in the evolution of owl monkeys (Aotus), which lost the direct P-LGN projections to layer 4A,along with their associated CO-dense band. The most dramatic modifications of the macaque- and squirrelmonkey–like pattern of layer 4A took place in hominoid evolution, however. The CO-dense band was lost inhominoid phylogeny prior to the divergence of orangutans (Pongo) from the African ape clade; this mayreflect the loss of P-LGN inputs (as in Aotus). By the same time point, there was increased expression of CBin layer 4A. Additional specializations occurred in the lineage leading to Homo sapiens, after its separationfrom the Pan lineage: Humans evolved a distinctive type of tissue compartmentation in layer 4A, in whichcompartments filled with small cells that stain densely for CB are embedded in a meshwork of neurons andneuropil that express NPNF and the proteoglycan labeled by Cat-301. Figure modified from Preuss, T. M., &Coleman, G. Q. (2002). Human-specific organization of primary visual cortex: Alternating compartmentsof dense Cat-301 and calbindin immunoreactivity in layer 4A. Cerebral Cortex, 12, 671–691. 436

REINVENTING PRIMATE NEUROSCIENCE FOR THE TWENTY-FIRST CENTURY 437minimize postmortem tissue degradation and students of primate brain evolution. There areensure uniform tissue fixation. Certainly such two sides to comparative molecular research,procedures improve tissue quality, and certain one focused on understanding the phyletic rela-molecules (especially RNA and small peptides) tionships between species, genes, and proteins,may be so labile as to degrade within minutes to and the other focused on reconstructing thehours if the brain is not fixed at death. Yet the evolutionary transformations of molecules andquality of tissue obtained and fixed postmortem molecular systems in particular evolutionarycan be remarkably good for many purposes lineages. Here, I am concerned principally with(Lewis, 2002). For many molecules, the critical the latter. To date, much of the research carriedissue appears to be overfixation rather than post- out in this vein has focused on the humanmortem collection. For decades, the common lineage, and particularly on molecular changespractice of pathologists and histologists was to potentially relevant to human brain evolution,store tissue samples in fixative at room tempera- but the same methods can be brought to bear onture. Prolonged exposure to fixative can alter the other primate lineages and other biologicalstructure of proteins so that antibodies don’t systems.bind to them well. In recent years, a variety of The most conspicuous recent development inprocedures for restoring the antigenicity of comparative molecular biology is the compara-tissue (antigen retrieval techniques) have been tive genomics movement. Comparative geno-developed that can greatly improve the staining mics is the product of the invention of rapid,of tissue that has been stored in fixative for long automated techniques for sequencing DNA,periods (months to years). Perhaps even more which made possible the first whole-genomevaluable, however, has been the development of sequencing projects involving humans andcryopreservative solutions containing ethylene select model organisms such as mice and fruitglycol—antifreeze, that is (Hoffman & Le, flies. As sequencing methods have improved and2004; Watson et al., 1986). Using these solutions, sequencing capacity has accumulated, the cost oftissue sections, tissue blocks, and even whole sequencing has come down and more and morebrains can be removed from fixative and stored species have been added to the list of organismsat –20°C in common household-type freezers. being ‘‘genomed,’’ with varying degrees of accu-Tissue quality is remarkable—antigenicity and racy and completeness. As of this writing, thereeven enzyme activity are retained for years (it’s are sequencing projects for 13 primate species,not yet clear for exactly how many years). Even including several strepsirrhines, a tarsier, a Newrelatively labile and fixation-sensitive molecules, World monkey (Callithrix jacchus), several Oldlike cytochrome oxidase, can be preserved with World monkeys from the genera Macaca andcryopreservative solution (Preuss & Coleman, Papio, a hylobatid, the Sumatran orang, gorilla,2002). Antigen retrieval and cryopreservation common chimpanzee, and human. In addition,techniques greatly enhance the value of brain one species each from the orders Scandentia (treetissue archives. shrews) and Dermoptera (flying lemurs) are being sequenced. Interestingly, the rationale offered for selecting particular species to be genomedComparative Genomics and Other includes not only their status as experimentalComparative Molecular Approaches models but also the desire to cover major branchesComparative molecular biology has roots that of the phylogenetic tree in order to facilitate thego back at least as far as the work of Nutall evolutionary reconstruction of gene sequences(1904), who identified differences in the immu- and genomes (see, e.g., http://genome.wustl.edu/nogenicity of blood serum derived from dif- genomes/view/cynocephalus_volans/). The rapidferent species. Recent advances in the growth in information about gene and proteintechnology of nucleic acid sequencing have pro- structure and function, spanning a remarkablefoundly transformed this field, however, and variety of organisms, has necessitated the devel-provide much new information of value to opment of computational and Web-based

438 PRIMATE NEUROETHOLOGYmethods for organizing and exploring informa- Most of these regulatory changes, they argued,tion, with the result that in silico analyses are now involved the rearrangement of genes on chro-as much a part of molecular biology as in vivo and mosomes, resulting in changes in the location ofin vitro studies. coding genes with respect to regulatory The development of technologies to sequence sequences. A small number of rearrangementsnucleic acids rapidly and cheaply has had con- of the right sorts of genes could yield profoundsequences that extend far beyond the mere accu- phenotypic changes, transforming a rathermulation of gene sequence data: They have led chimp-like ancestor into a human.(directly and indirectly) to remarkable new The sequencing of the human genome laidexplorations of virtually every dimension of the foundation for carrying out the researchmolecular genetics, including not only protein- program implicit in the analysis of King andcoding sequences but also the organization of Wilson (1975). Sequence information made itnontranscribed regions of the DNA that influ- possible to construct gene microarrays (geneence gene expression and of the nontranslated chips) that allow one to quantify the amountssmall RNAs that regulate how messenger RNAs of mRNAs present in tissue samples, and thus to(mRNAs) are translated into proteins. They have identify gene expression differences betweenalso made it possible to document in exquisite humans, chimpanzees, and other primates. Thedetail how evolution rearranged the coding and microarrays employed in most of the first studiesnoncoding elements in chromosomes, and so of this type consisted of probe sequences repre-augment the work of classical comparative cyto- senting about 10,000 expressed human genesgenetics. The information from sequencing pro- (nearly half the total complement of expressedjects has also provided the basis for constructing genes). To date, studies comparing humans andmRNA microarrays that make it possible to chimpanzees have focused on adult brain tissueidentify genes differentially expressed in homo- ´ (Caceres et al., 2003; Enard et al., 2002; Gu & Gu,logous tissues under different experimental con- 2003; Khaitovich et al., 2004; Oldham et al.,ditions, or in some cases, in homologous tissues 2006; Uddin et al., 2004; but see also Karamanof different species. et al., 2003). The data generated by these dif- Rather than review these techniques in detail, ferent studies show a high level of consistency,I want to consider some recent findings in com- and there is a good rate of validation for indivi-parative genomics that require us to reconsider dual genes when examined with independentour common understanding of the relationship techniques (Preuss et al., 2004). Functional ana-between genetic and phenotypic evolution, an lysis of the expression differences is problematic,understanding strongly influenced by the classic in part because the functions of many of thepaper of King and Wilson (1975). King and genes differentially expressed in the cortex areWilson were struck by the very close similarity ´ so poorly understood. However, Caceres andin the macromolecular sequences of humans and ´ colleagues (Caceres et al., 2003; Preuss et al.,chimpanzees: greater than 99% correspondence 2004) noted a higher than expected number ofin the amino acid sequences of proteins, for expression changes in genes involved in cellexample. Yet, it seemed to them there was a growth and maintenance in the human lineage,very large gap between the phenotypes of and identified a number of genes involved inhumans and chimpanzees, too large to be neuronal or synaptic functions that underwentaccounted for by of the small number of changes expression changes in human evolution.in protein sequences. To explain this disparity, One limitation of the use of microarrays forthey proposed that most of the important evolu- comparative studies like these is that the micro-tionary changes in the human lineage involved arrays are composed of probe sequences basedchanges in gene expression; that is, in the on human-specific sequences; thus, in instancesamount of mRNA and protein that specific in which there are species differences in the shortgenes produce and in the spatial distribution sequences used to construct the probe sets, theand developmental timing of that production. human mRNA transcripts will bind more

REINVENTING PRIMATE NEUROSCIENCE FOR THE TWENTY-FIRST CENTURY 439strongly to the probes than their nonhuman segmental duplications (resulting in sequenceshomologs, creating the false impression of with copy numbers in the range of 2 to 50) inincreased gene expression in humans. The hominoid and human evolution (Bailey &sequence differences between humans and chim- Eichler, 2006). Approximately 33% of the dupli-panzees are sufficiently small that one can delete cates present in humans are absent in chimpan-from the analysis probes that aren’t identical in zees (Cheng et al., 2005). We seem to havethe two species. The differences between maca- systematically underestimated the global genomicques and humans (and chimpanzees) are much differences between species, even such closelygreater, however, and one would have to elim- related species as humans and chimpanzeesinate so many probes as to make analysis (Sikela, 2006). It may be true that the alignable,impractical. The noise in the outgroup data orthologous gene sequences of humans andmakes it difficult to determine the direction of chimpanzees are about 98% to 99% similar, butcharacter change in the ingroups; thus, we may these do not comprise the entire genome.be able to state with confidence, for example, The gain and loss of gene copies has impor-that gene X is expressed at a higher level in tant implications. When genes that are copied orhumans than in chimpanzees, but we can be lost are very similar in structure and function,less sure whether the gene was upregulated in we might think of phyletic differences in copyhuman evolution or downregulated in chim- number as affecting gene dosage. As has long bepanzee evolution. The new technology of mass understood (Ohno, 1970), however, gene dupli-resequencing (Bentley, 2006), which can be used cation followed by structural divergence is theto quantify the numbers of mRNA copies pre- main mechanism by which evolution has gener-sent in tissue samples, should be ‘‘species ated new genes—genes that have new functions,agnostic,’’ in the sense that its precision should or are localized in different tissues or cell com-not depend on the species being studied (pro- partments, or are expressed at different points invided that adequate sequence information is development (Conrad & Antonarakis, 2007).available), and therefore should be extremely Given the apparent large number of lineage-spe-valuable for comparative studies. cific duplications in primate evolution The advent of techniques for characterizing (including hominoid and hominin evolution),phyletic differences in gene expression differ- we have to take seriously the fact that there areences is by any measure a real breakthrough. species-specific genes, and take seriously theParallel developments in genomics, however, possibility that these may have played a signifi-also make it clear that although changes in gene cant role in human evolution (Fortna et al.,regulation are important in evolution, there is 2004; Nahon, 2003; Sikela, 2006). Two genemore to the evolution of gene expression than groups that have attracted interest in thischromosomal rearrangements and more to the regard are the morpheus family (Eichler et al.,genetics of species differences than changes in 2001; Johnson et al., 2001) and the DUF1220gene expression. Genomes are systems in flux, domain (Popesco et al., 2006); not only wereand numerous gene duplications, insertions, and these gene segments amplified in human evolu-deletions occurred independently in the human tion, but also some of their members underwentand chimpanzee lineages, as well as in other strong positive selection at the sequence level.primate lineages (Eichler et al., 2001; Fortna DUF1220-containing proteins are expressedet al., 2004; Nahon, 2003). Thus, Varki and strongly (although not exclusively) in the brainAltheide (2005) estimated that humans and (Popesco et al., 2006).chimpanzees are %96%, rather than 98% to New genes can also arise by means other than99%, similar at the nucleotide level. Even that duplication and divergence. Hayakawa and col-number, however, doesn’t really capture the full leagues (2005) have identified a human-specificmagnitude of the genetic differences between gene, SIGLEC11, a segment of which is homo-lineages. The studies of Eichler and colleagues, logous to part of the chimpanzee SIGLEC11in particular, have identified large numbers of gene, while another segment is homologous to

440 PRIMATE NEUROETHOLOGYpart of human SIGLEC16, which is a pseudo- isn’t limited to the diversity of genes: We knowgene. Human SIGLEC11 is expressed in micro- that homologous gene transcripts can be pro-glia (Hayakawa et al., 2005), the cells that cessed to yield different functional proteins bymediate immune and inflammatory responses the alternative splicing of different exonsin the brain. (Calarco et al., 2007) and that homologous pro- Nor do all the new or modified genes code for teins can undergo different patterns of posttran-proteins: Some code for transcribed, but slational processing and modification that affectuntranslated, RNAs. Pollard and colleagues their functions (e.g., Brooks, 2004; Gearing et al.,(2006a), screening for sequences that are highly 1996; Rosen et al., 2006; Varki, 2006).conserved in vertebrate evolution but that The magnitude of macromolecular diversityunderwent rapid change in the human lineage, constitutes something of a conundrum. Underidentified over 200 elements, most of which are the old view that humans and chimps are 99%in noncoding DNA and many of which are identical, the role of comparative genomicslocated near transcription factor and DNA- seemed to be to identify the relatively few geneticbinding genes, that is, genes that regulate gene differences that explain the phenotypic differencesexpression. The element that showed the highest between humans and chimpanzees. Now, it seems,rate of change in the human lineage, however, we’ve got lots of genetic differences betweencodes for part of an expressed RNA, HAR1F humans and chimpanzees, involving many dif-(Pollard et al., 2006b). Pollard and colleagues ferent dimensions of the macromolecular organi-demonstrated that HAR1F is expressed by zation, and the differences between more distantlyCajal-Retzius cells, which also express a protein, related taxa must be even greater. Yet, as discussedreelin, that plays a central role in organizing previously, we know very little about how humanthe laminar arrangement of neurons in the and ape brains differ, or about how hominoiddeveloping cortex. The secondary structure of brains differ from those of other primates.HAR1F appears to be unlike that of other Where, then, are all the human-specific phe-known expressed RNAs, including the class of notypes? Presumably, they’re still out there for ussmall (%22 nucleotide) RNAs known as to discover. This suggests a new role for compara-microRNAs. MicroRNAs are thought to play tive genomics and molecular biology: phenotypean important role in sculpting developmental- discovery (Preuss et al., 2004). In genetics, typi-stage and tissue-specific patterns of gene expres- cally, one identifies a distinct phenotype and thension; in the brain they have been implicated in tracks down the gene or genes that contribute tothe control of synapse formation (Klein et al., it. Phenotype discovery reverses the process,2005). As with other classes of genes, there are taking known genetic differences as startinglineage-specific (including human-specific) points for idenfying previously unknown pheno-microRNAs (Berezikov et al., 2006; Zhang types. One way to do this is to investigate theet al., 2007). consequences of a sequence change or expres- It is clear, then, that genetic differences sion-level changes in tissues that express thebetween humans and chimpanzees—animals gene in question. For example, using microarraysseparated by only %6 million years from their and polymerase chain reaction to comparecommon ancestor—are much greater than has humans, chimpanzees, and macaques, my collea-commonly been supposed (Cohen, 2007b). Far gues and I found evidence that the genes for thefrom being 99% identical, with only a smattering extracellular matrix proteins thrombospondin 2of nucleotide differences in coding and non- (THBS2) and thrombospondin 4 (THBS4) werecoding regions separating us, we now know upregulated in human brain evolution, THBS4that humans and chimpanzees differ along ´ dramatically so (Caceres et al., 2007) (Fig. 22.5).many different dimensions of genomic organi- Thrombspondins are proteins of special interestzation, and that there are likely a nontrivial to neuroscientists because in model systems theynumber of human-specific and chimpanzee-spe- play a critical role in inducing neurons to makecific genes. Moreover, macromolecular diversity synapses (Christopherson et al., 2005; Susman

Figure 22.5 Evidence for increased expression of thrombospondin messenger RNA (mRNA) and proteinsin human brain evolution. (A) Comparative microarray experiments indicate that expression of the genesfor THBS4 and THBS2 is stronger in forebrain regions of humans (Hs: Homo sapiens) than of chimpanzees(Pt: Pan troglodytes). Values plotted are average mRNA hybridization intensities and standard errors.Asterisks indicate significant differences at the 0.05 (*) and 0.01 (**) levels. Brain regions surveyedinclude FCx, frontal cortex; TCx, temporal cortex: VCx, primary visual cortex; ACCx, anterior cingulatecortex; Cau, caudate nucleus; and Cb, cerebellum. (B) Comparative reverse transcriptase-polymerase chainreaction (RT-PCR) experiments support the conclusion that expression of the THBS4 and THBS2 genes infrontal cortex increased in the human lineage, after the divergence of humans and chimpanzees. Expressionvalues from humans (Hs), chimpanzees (Pt), and two species of macaque monkeys, Macaca mulatta (Mm)and Macaca nemestrina (Mn), were used to estimate expression levels at all three internal nodes of thephylogeny using linear parsimony and squared-change parsimony techniques. Branches that showsignificant differences between the observed values for the living species and estimated value for theancestral node are marked with asterisks. Observed expression values represent the mean number (–SEM) of thrombospondin gene transcripts in the sample per 1,000 b-actin transcripts in the sample. (b-actin serves as a control gene to which results from each sample are normalized.) (C) Comparative Westernblotting results confirm that the increased expression of thrombospondin mRNA in humans wasaccompanied by increased protein levels in the cortex of the frontal pole (FP). Samples were comparedfrom three individual humans (Hs), three chimpanzees (Pt), and three macaques (Mm) The density andwidth of the bands reflects the amount of protein present in a sample. b-Tubulin (TUBB) was used asloading control to normalize for differences in the total amount of protein in the samples. Staining forTHBS4 and THBS2 was stronger in the humans than in chimpanzees or macaques, THBS4 very markedly so.(D) Immunocytochemistry for THBS4 protein in frontal polar cortex revealed stronger labeling in humansthan in chimpanzees or macaques. The lower panel shows labeling through the full thickness of the cortex;the upper panel shows labeling in the upper part of cortical layer 3 at higher magnification. THBS4 antibodylabeled numerous pyramidal cell bodies in all species, as well as the neuropil region surrounding the cellbodies. Labeling of the neuropil, which is rich in synapses, was particularly strong in humans. Scale bars:upper panel, 50 mm; lower panel, 250 mm. Figures are modified from Caceres, M., Suwyn, C., Maddox, M., ´Thomas, J. W., & Preuss, T. M. (2007). Increased cortical expression of two synaptogenic thrombospondinsin human brain evolution. Cerebral Cortex, 17, 2312–2321. 441

442 PRIMATE NEUROETHOLOGYFigure 22.5 (Continued)et al., 2007). Thrombospondins have other func- that humans modified their propensity to maketions, however, and are expressed by a variety of and unmake synapses, the results justify furthercell types, so the mere fact that thrombospondin explorations of the comparative synaptic biologymRNA was upregulated in human brain tissue of humans and nonhuman primates. If the bio-isn’t by itself strong evidence humans underwent chemical machinery of synaptogenesis were,evolutionary changes in the biochemical control indeed, modified during human evolution, weof synapse formation. Moreover, it is possible that should expect there to have been changes inthe upregulation of thrombospondin mRNA is other genes and proteins involved in this process,not matched by an increase in thrombospondin and we are currently investigating several addi-protein. So, we used Western blotting with anti- tional candidate gene expression changes thatbodies to THBS4 and THBS2 proteins to verify may be relevant. A change in the synaptogenicthat they are, indeed, expressed more strongly in machinery of the cortex should also be mani-humans than in chimps or macaques. Finally, fested at physiological, cognitive, and behavioralusing immunocytochemistry with antibodies to levels. What these changes might be are notTHBS4, we showed that the most conspicuous clear—no one, to my knowledge, has previouslydifference in the labeling of prefrontal cortex proposed that there might be something unusualacross species was the greater density in humans about human synaptic machinery. One possibi-of labeling in the neuropil space—the territory lity is that evolution modified the substrates ofsurrounding neuronal cell bodies that is filled human working memory, which is at the heart ofwith fine axonal and dendritic processes, and human reasoning capacities and involves cir-the place where most synapses are located. cuitry spanning much of the cortex (Baddeley,While we have not yet directly demonstrated 2003).

REINVENTING PRIMATE NEUROSCIENCE FOR THE TWENTY-FIRST CENTURY 443 This example illustrates how genomic differ- evolution facing extinction, neuroscientistsences can suggest previously unsuspected phe- need to make a commitment to species preserva-notypes. It also illustrates how the different tion. This means not only conservation in thetechniques discussed in this chapter can be wild but also the provision for captive animalused in combination, in this case, the combina- facilities: Both are essential. I take it there’s notion of comparative genomics, molecular need to explain to biologists why species con-biology, and histology. Conceivably, if we were servation in the wild is important, but I do thinkto obtain additional evidence pointing to a it’s important to emphasize how important cap-human specialization of synaptic mechanisms, tive facilities are for experimental behavioral,we could test our working hypothesis with com- cognitive, and neuroimaging studies. Simplyparative neuroimaging. put, the opportunities for doing properly controlled experiments in the wild are extremely limited compared to those in captive situations.CONCLUSIONS We would never take seriously theories ofThe new tools of comparative neuroimaging, human psychology or brain function that arecomparative histology, and comparative mole- not based on well-controlled experiments: Whycular biology discussed in this chapter can serve would we suppose that we can do without themto energize the study of primate brain evolution when trying to understand other species? If wein the twenty-first century. What makes these think that captive situations are too artificial totools so valuable is not only that they permit us yield valid results, we should work to make themto explore dimensions of brain organization more naturalistic. We need a better interactionthat were previously out of reach, but also that between field and captive studies and scientiststhey make it possible to study species that are with experience in both contexts. Certainly, theotherwise out of reach. This is because they can impending loss of access to chimpanzees forbe used in vivo, or make use of material even the kinds of low-impact investigationsacquired using noninvasive or minimally inva- that we routinely carry out in humans will havesive procedures in living individuals or from a catastrophic effect on our ability to understandsamples acquired postmortem. As a result, not human brain organization, and we need to doonly can they tap the resources of primate everything in our power to ensure that futureresearch centers, they can also make available generations of scientists will be able to studyfor study the more varied collections of species these unique and remarkable animals (Preuss,present in zoos and wildlife parks, greatly 2006).enhancing the value of these resources. I don’t As the foregoing discussion has made clear,mean to imply that the development of low- tissue collections are potentially of enormousimpact technologies renders invasive and term- value for understanding primate brain evolu-inal methods irrelevant or obsolete: We would tion. Existing collections include thebe very hard pressed to sense of DTI or fcMRI Comparative Mammalian Brain Collectionresults without the results of experimental (www.brainmuseum.org), consisting of materialtract-tracing and physiological recording studies originally assembled by W. Welker at thein macaques, squirrel monkeys, owl monkeys, University of Wisconsin and J.I. Johnston, Jr.,and galagos. Nevertheless, the new technologies at Michigan State University, and currentlyoffer new opportunities for the growth of neuros- housed in the National Museum of Health andcientific understanding and the more complete Medicine. There is also the Comparativeintegration of neuroscience with the other beha- Neurobiology of Aging Resource, organized byvioral and biological sciences, provided that we J.M. Erwin as part of the Great Ape Agingmuster the appropriate resources. Project, which serves as a clearinghouse for What are those appropriate resources? With great ape brain tissue. These efforts are extremelyso many of the mammalian and primate species valuable, but we need to do more. Faced as wecrucial for understanding primate brain are with the extinction of so many primate

444 PRIMATE NEUROETHOLOGYspecies, we must make every reasonable effort to that all living species, including humans, arecollect tissue from as many individuals of as the products of equally long evolutionary his-many primate species as possible, and ensure tories. Moreover, we understand that each spe-that specimens are properly curated and made cies is a mosaic of characteristics: some sharedavailable to qualified investigators. We need to with many other species by virtue of ancientidentify and catalog brain material held in ad common ancestry, some shared with a narrowerhoc collections. Fixed brains from as many range of species by virtue of more recentspecies as possible should be scanned with common ancestry, and some that are unique tostructural MRI and DTI protocols, and the the species, the products of recent evolutionarydata placed in a permanent database of primate history. If we want to know what features ofbrain morphology and connectivity, accessible organization are widespread among mammals,online. MRI scans of diverse fish species are cur- we understand that we must study a variety ofrently being collected under the auspices of the different mammalian species to reconstructDigital Fish Library (www.digitalfishlibrary.org), ancestral organization. Similarly, if we want tounder the sponsorship of the National Science know what features of organization constituteFoundation—we should be able to do at least as the defining specializations of a particular spe-much with primates. cies, we understand that we need to compare the As essential as these resources and technolo- target species to other species, and that the mostgies are to primate neuroscience, they will be of informative comparisons are with closely relatedlittle value unless we have researchers who are species. If the species in question is Homoexpert in both neurobiology and evolutionary sapiens, the most informative species are chim-biology. We need to recognize a distinct profes- panzees and the other great apes. No one withsional category—evolutionary neuroscience— even the most rudimentary training in modernjust as 25 years ago we recognized molecular evolutionary biology would presume that oneevolution as a distinct field, and provide fellow- could take results from a few rodent speciesships and stipends to support students in this and expect them, as a rule, to generalize to alldiscipline, as well as dedicated funding for com- mammals, or take results from a few rodent orparative brain research. even primate species or expect them to extend We need to recognize this special class of with high fidelity to humans.scientists to bridge the gap between the two Yet it is upon such boggy ground that thevery ways of conceptualizing the role of animal enterprise of modern neuroscience largely rests.studies in biological research. It’s not too much Most branches of neuroscience have adopted theof a stretch to say that there are currently two model-animal paradigm, an approach thatbiologies—one that takes phyletic diversity ser- assumes that results obtained in a few modeliously and one that doesn’t. Those of us who species will be broadly applicable to other spe-would characterize ourselves as neuroethologists cies. Neuroscientists expend enormous effort toor primatologists or zoologists are likely to be ensure that their experimental results havefamiliar with the principles of modern evolution internal validity—that is, that their results arebiology. As modern evolutionists, most of us reliable or repeatable under the same set of con-would disavow those views of evolution that ditions—but often pay little more than lip ser-were so popular in the nineteenth and early vice to external validity, or generalizability. Iftwentieth centuries, specifically, that evolution you believe that rodents are truly archetypicaltakes the form of a phylogenetic scale, with mammals, then external validity becomes a non-humans at the top (Lovejoy, 1964; Richards, issue (Logan, 1999, 2002, 2005). But if you1987), or the idea that the multitude of species understand that species vary, that mammalianare really just minor variants of a few basic brains are not just smaller or larger versions of aanimal ‘‘types’’ (Desmond, 1982; Richards, common brain type, you have to take external1992). Evolutionary biologists today take the validity seriously. And the way to take externalview that evolution is a branching tree, and validity seriously is to do comparative studies.

REINVENTING PRIMATE NEUROSCIENCE FOR THE TWENTY-FIRST CENTURY 445I’m not suggesting that we stop studying rodents The lesson of genomics is that comparativeor rhesus monkeys. Rather, I’m suggesting that biology is good biology, and that studying manyneuroscientists need to study those species and species is better than studying a few species. Ofmany more, and do so within a rigorous com- course, we can’t study every species in great depth,parative framework. Only in this way can we and I don’t doubt that obtaining useful neurobio-make well-founded claims about ‘‘basic’’ fea- logical information from any given species typi-tures of brain organization. It is also the only cally requires more cost and effort than obtainingway we can make room for the serious consid- useful genetic information from that same species.eration of the ways species and groups of species But neuroscience can no longer afford to live out-differ from one another, such as, for example, side the scope of comparative biology and to behow primates differ from other animals or how hamstrung by the concentration of effort andhumans differ from other primates. resources on a few ‘‘model’’ species. It is not Perhaps it is too much to hope that the neu- merely desirable or intellectually pleasing that neu-rosciences can be reconciled with modern evolu- roscience anchor itself with the core principles thattionary biology. Consider, however, the case guide the rest of biology—it is essential. It’s aof comparative genomics. One need only question of good biology versus bad biology:browse through the Entrez Gene Web site whether we’re going to make room for phyletic(www.ncbi.nlm.nih.gov/sites/entrez), sponsored variations in brain organization—including theby the National Center for Biotechnology variations that make humans human—or whetherInformation (NCBI; a branch of NIH), to we’re going to content ourselves with the fictionappreciate the extent to which genomics and that ‘‘brains are brains.’’related branches of molecular science have beenevolutionized. At the core of the NCBI databaseis the tree of life (www.ncbi.nlm.nih.gov/sites/ ACKNOWLEDGMENTSentrez?db=taxonomy), and although the tax- The author is grateful to acknowledge the sup-onomy browser warns us not to regard it as a port of the James S. McDonnell Foundationsubstitute for the primary taxonomic literature (JSMF 21002093), the National Aging Instituteon phyletic relationships, the effort is remark- (NIH P01AG026423), and the Yerkes Base Grantably comprehensive and thoroughly grounded (NIH RR-00165).in modern phylogenetics. Data for thousandsof genes and gene domains are presented, all inappropriate phylogenetic context (Benson et al., REFERENCES2000; Wheeler et al., 2000). Abbott, A. (2003). Neuroscience: A new atlas of the Genomics didn’t start out with a strong com- brain. Nature, 424, 249–250.mitment to evolutionary diversity. The roots of Allman, J. M. (1977). Evolution of the visual systemgenomics are in experimental genetic studies of in the early primates. In: J. M. Sprague & A. N.fruit flies and mice, animals that were aggressively Epstein (Eds.), 7(pp. 1–53). New York:promoted as stand-ins for animals generally Academic.(Kohler, 1994; Rader, 2004). Genetics, however, Allman, J. M., Watson, K. K., Tetreault, N. A., &has never been saddled with quite as small a Hakeem, A. Y. (2005). Intuition and autism: Anumber of model species as the neurosciences, possible role for Von Economo neurons. Trendsand as sequence data for different species began in Cognitive Science, 9, 367–373. Amunts, K., & Zilles, K. (2001). Advances into accumulate, the differences between organisms cytoarchitectonic mapping of the humanbecame a fact that couldn’t be ignored, if for no cerebral cortex. Neuroimaging Clinics of Northother reason than that the differences had to be America, 11, 151–169, vii.dealt with systematically in order to correctly iden- Annese, J., Pitiot, A., Dinov, I. D., & Toga, A. W.tify the commonalities. Genomics thus embraced (2004). A myelo-architectonic method for thethe modern evolutionary concepts and methods structural classification of cortical areas.that make sense of the diversity of life. Neuroimage, 21, 15–26.

446 PRIMATE NEUROETHOLOGYBaddeley, A. (2003). Working memory: Looking of alternative splicing differences between back and looking forward. Nature Reviews humans and chimpanzees. Genes and Neuroscience, 4, 829–839. Development, 21, 2963–2975.Bailey, J. A., & Eichler, E. E. (2006). Primate Casagrande, V. A., & Kaas, J. H. (1994). The segmental duplications: Crucibles of evolution, afferent, intrinsic, and efferent connections of diversity and disease. Nature Reviews Genetics, primary visual cortex in primates. In: A. Peters 7, 552–564. & K. Rockland (Eds.), (pp. 201–259).Balter, M. (2007). Neuroanatomy. Brain evolution New York: Plenum. studies go micro. Science, 315, 1208–1211. Casagrande, V. A., Khaytin, I., & Boyd, J. (2007).Behrens, T. E., & Johansen-Berg, H. (2005). The evolution of parallel visual pathways in the Relating connectional architecture to grey brains of primates. In: J. H. Kaas & T. M. Preuss matter function using diffusion imaging. (Eds.), Evolution of nervous systems. Vol. 4: Philosophical Transactions of the Royal Society Primates (pp. 87–108). Oxford: Elsevier. of London Series B: Biological Science, 360, Cheng, Z., Ventura, M., She, X., Khaitovich, P., 903–911. Graves, T., Osoegawa, K., et al. (2005). ABenson, D. A., Karsch-Mizrachi, I., Lipman, D. J., genome-wide comparison of recent Ostell, J., Rapp, B. A., & Wheeler, D. L. (2000). chimpanzee and human segmental GenBank. Nucleic Acids Research, 28, 15–18. duplications. Nature, 437, 88–93.Bentley, D. R. (2006). Whole-genome re-sequencing. Christopherson, K. S., Ullian, E. M., Stokes, C. C., Current Opinions in Genetics and Development, 16, Mullowney, C. E., Hell, J. W., Agah, A., et al. 545–552. (2005). Thrombospondins are astrocyte-Berezikov, E., Thuemmler, F., van Laake, L. W., secreted proteins that promote CNS Kondova, I., Bontrop, R., Cuppen, E., et al. synaptogenesis. Cell, 120, 421–433. (2006). Diversity of microRNAs in human and Clare, S., & Bridge, H. (2005). Methodological issues chimpanzee brain. Nature Genetics, 38, relating to in vivo cortical myelography using 1375–1377. MRI. Human Brain Mapping, 26, 240–250.Biswal, B., Yetkin, F. Z., Haughton, V. M., & Hyde, Clark, V. P., Courchesne, E., & Grafe, M. (1992). In J. S. (1995). Functional connectivity in the vivo myeloarchitectonic analysis of human motor cortex of resting human brain using striate and extrastriate cortex using magnetic echo-planar MRI. Magnetic Resonance in resonance imaging. Cerebral Cortex, 2, 417–424. Medicine, 34, 537–541. Cohen, J. (2007a). Animal studies. NIH to endBrodmann, K. (1909). Vergleichende chimp breeding for research. Science, 316, 1265. Lokalisationslehre der Grosshirnrhinde. Leipzig: Cohen, J. (2007b). Evolutionary biology. Relative Barth. (Reprinted as Brodmann’s Localisation in differences: The myth of 1%. Science, 316, 1836. the cerebral cortex, translated and edited by L. J. Cola, M. G., Seltzer, B., Preuss, T. M., & Cusick, Garey, London: Smith-Gordon, 1994). C. G. (2005). Neurochemical organization ofBrooks, S. A. (2004). Appropriate glycosylation of chimpanzee inferior pulvinar complex. Journal recombinant proteins for human use: of Comparative Neurology, 484, 299–312. Implications of choice of expression system. Colombo, J. A., Sherwood, C. C., & Hof, P. R. Molecular Biotechnology, 28, 241–255. (2004). Interlaminar astroglial processes in the ´Caceres, M., Lachuer, J., Zapala, M. A., Redmond, cerebral cortex of great apes. Anatomy and J., Kudo, L., Geschwind, D., et al. (2003). Embryology (Berlin), 208, 215–218. Elevated gene expression levels distinguish Conrad, B., & Antonarakis, S. E. (2007). Gene human from non-human primate brains. duplication: A drive for phenotypic diversity Proceedings of the National Academy of and cause of human disease. Annual Review of Sciences, 100, 1330–1335. Genomics and Human Genetics, 8, 17–35. ´Caceres, M., Suwyn, C., Maddox, M., Thomas, Creutzfeldt, O. D. (1977). Generality of functional J. W., & Preuss, T. M. (2007). Increased structure of the neocortex. Naturwiss cortical expression of two synaptogenic Naturwissenschaften, 64, 507–517. thrombospondins in human brain evolution. Crick, F., & Jones, E. (1993). Backwardness of Cerebral Cortex, 17, 2312–2321. human neuroanatomy. Nature, 361, 109–110.Calarco, J. A., Xing, Y., Caceres, M., Calarco, J. P., Croxson, P. L., Johansen-Berg, H., Behrens, T. E., Xiao, X., Pan, Q., et al. (2007). Global analysis Robson, M. D., Pinsk, M. A., Gross, C. G., et al.

REINVENTING PRIMATE NEUROSCIENCE FOR THE TWENTY-FIRST CENTURY 447 (2005). Quantitative investigation of connections Elston, G. N. (2007). Specializations of the of the prefrontal cortex in the human and macaque neocortical pyramidal cell during primate using probabilistic diffusion tractography. Journal evolution. In: J. H. Kaas & T. M. Preuss (Eds.), of Neuroscience, 25, 8854–8866. Evolution of nervous systems. Vol. 4: PrimatesD’Arceuil, H. E., Westmoreland, S., & de (pp. 191–242). Oxford: Elsevier. Crespigny, A. J. (2007). An approach to high Enard, W., Khaitovich, P., Klose, J., Zollner, S., resolution diffusion tensor imaging in fixed Heissig, F., Giavalisco, P., et al. (2002). Intra- primate brain. Neuroimage, 35, 553–565. and interspecific variation in primate geneDeFelipe, J., Alonso-Nanclares, L., & Arellano, J. I. expression patterns. Science, 296, 340–343. (2002). Microstructure of the neocortex: Fortna, A., Kim, Y., MacLaren, E., Marshall, K., Comparative aspects. Journal of Neurocytology, Hahn, G., Meltesen, L., Brenton, M., et al. 31, 299–316. (2004). Lineage-specific gene duplication andDeFelipe, J., Alonso-Nanclares, L., Arellano, J., loss in human and great ape evolution. PLoS ´˜ Ballestero-Yanez, I., Benevides-Piccione, R., & Biology, 2, E207. ˜ Munoz, A. (2007). Specializations of the Fox, M. D., Snyder, A. Z., Vincent, J. L., Corbetta, cortical microstructure of humans. In: J. H. M., Van Essen, D. C., & Raichle, M. E. (2005). Kaas & T. M. Preuss (Eds.), Evolution of The human brain is intrinsically organized into nervous systems. Vol. 4: Primates (pp. 168– dynamic, anticorrelated functional networks. 190). Oxford: Elsevier. Proceedings of the National Academy ofDesmond, A. J. (1982). Archetypes and ancestors: Sciences, 102, 9673–9678. Palaeontology in Victorian London, 1850–1875. Galuske, R. A., Schlote, W., Bratzke, H., & Singer, London: Blond & Briggs. W. (2000). Interhemispheric asymmetries of theDiamond, I. T., & Hall, W. C. (1969). Evolution of modular structure in human temporal cortex. neocortex. Science, 164, 251–262. Science, 289, 1946–1949.Eichler, E. E., Johnson, M. E., Alkan, C., Tuzun, Garland, T., Harvey, P., & Ives, A. (1992). E., Sahinalp, C., Misceo, D., et al. (2001). Procedures for the analysis of comparative Divergent origins and concerted expansion data using phylogenetically independent of two segmental duplications on contrasts. Systematic Biology, 41, 18–32. chromosome 16. Journal of Heredity, 92, Gearing, M., Tigges, J., Mori, H., & Mirra, S. S. 462–468. (1996). Abeta40 is a major form of beta-Eickhoff, S. B., Paus, T., Caspers, S., Grosbras, M. amyloid in nonhuman primates. Neurobiology H., Evans, A. C., Zilles, K., et al. (2007b). and Aging, 17, 903–908. Assignment of functional activations to Geyer, S., Ledberg, A., Schleicher, A., Kinomura, S., probabilistic cytoarchitectonic areas revisited. Schormann, T., Burgel, U., et al. (1996). Two Neuroimage, 36, 511–521. different areas within the primary motor cortexEickhoff, S. B., Schleicher, A., Scheperjans, F., of man. Nature, 382, 805–807. Palomero-Gallagher, N., & Zilles, K. (2007a). Gilissen, E. (2001). Structural symmetries and Analysis of neurotransmitter receptor asymmetries in human and chimpanzee distribution patterns in the cerebral cortex. brains. In: ‘D. Falk & K. R. Gibson (Eds.), Neuroimage, 34, 1317–1330. Evolutionary anatomy of the primate cerebralEickhoff, S. B., Stephan, K. E., Mohlberg, H., Grefkes, cortex (pp. 187–215). Cambridge: Cambridge C., Fink, G. R., Amunts, K., et al. (2005b). A new University Press. SPM toolbox for combining probabilistic Gu, J., & Gu, X. (2003). Induced gene expression in cytoarchitectonic maps and functional imaging human brain after the split from chimpanzee. data. Neuroimage, 25, 1325–1335. Trends in Genetics, 19, 63–65.Eickhoff, S., Walters, N. B., Schleicher, A., Kril, J., Guilfoyle, D. N., Helpern, J. A., & Lim, K. O. Egan, G. F., Zilles, K., et al. (2005a). High- (2003). Diffusion tensor imaging in fixed brain resolution MRI reflects myeloarchitecture and tissue at 7.0 T. NMR in Biomedicine, 16, 77–81. cytoarchitecture of human cerebral cortex. Gundersen, H. J., Bagger, P., Bendtsen, T. F., Human Brain Mapping, 24, 206–215. Evans, S. M., Korbo, L., Marcussen, N.,Elston, G. N. (2003). Cortex, cognition and the cell: et al. (1988). The new stereological tools: New insights into the pyramidal neuron and Disector, fractionator, nucleator and point prefrontal function. Cerebral Cortex, 13, 1124–1138. sampled intercepts and their use in

448 PRIMATE NEUROETHOLOGY pathological research and diagnosis. Apmis, search for more suitable parameters. American 96, 857–881. Anthropologist, 68, 103–121.Hackett, T. A., Preuss, T. M., & Kaas, J. H. (2001). Hopkins, W. D., Marino, L., Rilling, J. K., & Architectonic identification of the core region MacGregor, L. A. (1998). Planum temporale in auditory cortex of macaques, chimpanzees, asymmetries in great apes as revealed and humans. Journal of Comparative Neurology, by magnetic resonance imaging (MRI). 441, 197–222. Neuroreport, 9, 2913–2918.Hampson, M., Peterson, B. S., Skudlarski, P., Howell, L. L., Hoffman, J. M., Votaw, J. R., Gatenby, J. C., & Gore, J. C. (2002). Detection Landrum, A. M., & Jordan, J. F. (2001). An of functional connectivity using temporal apparatus and behavioral training protocol to correlations in MR images. Human Brain conduct positron emission tomography (PET). Mapping, 15, 247–262. Neuroimaging in conscious rhesus monkeys.Harting, J. K., Glendenning, K. K., Diamond, I. T., Journal of Neuroscience Methods, 106, 161–169. & Hall, W. C. (1973). Evolution of the primate Jerison, H. J. (1973). Evolution of the brain and visual system: Anterograde degeneration studies intelligence. New York: Academic Press. of the tecto-pulvinar system. American Journal Johnson, M. E., Viggiano, L., Bailey, J. A., Abdul- of Physical Anthropology, 38, 383–392. Rauf, M., Goodwin, G., Rocchi, M., et al.Harting, J. K., Hall, W. C., & Diamond, I. T. (1972). (2001). Positive selection of a gene family Evolution of the pulvinar. Brain, Behavior, and during the emergence of humans and African Evolution, 6, 424–452. apes. Nature, 413, 514–519.Hayakawa, T., Angata, T., Lewis, A. L., Mikkelsen, T. Kaas, J. H. (1987). The organization of neocortex in S., Varki, N. M., & Varki, A. (2005). A human- mammals: Implications for theories of brain specific gene in microglia. Science, 309, 1693. function. Annual Review of Psychology, 38,Hecht, E., Gutman, D., Glasser, M., Mascaro, J., 129–151. Hamann, S., Preuss, T. M., et al. (2008). Kaas, J. H. (2007). The evolution of sensory and Comparing amygdala connectivity between motor systems in primates. In: J. H. Kaas & monkeys, apes, and humans using diffusion T. M. Preuss (Eds.), Evolution of nervous tensor imaging [Abstract]. American Journal of systems. Vol. 4: Primates (pp. 35–57). Oxford: Physical Anthropology. Elsevier.Hinkley, L., Padberg, J., Krubitzer, L., & Disbrow, Kaas, J. H., & Preuss, T. M. (1993). Archontan E. (2007). Hand use and the evolution of affinities as reflected in the visual system. In: F. S. posterior parietal cortex in primates. In: J. H. Szalay, M. J. Novacek & M. C. McKenna (Eds .), Kaas & T. M. Preuss (Eds.), Evolution of nervous (pp. 115–128). New York: Springer Verlag. systems. Vol. 4: Primates (pp. 407–415). Oxford: Karaman, M. W., Houck, M. L., Chemnick, L. G., Elsevier. Nagpal, S., Chawannakul, D., Sudano, D., et al.Hof, P. R., Nimchinsky, E. A., Perl, D. P., & (2003). Comparative analysis of gene- Erwin, J. M. (2001). An unusual population expression patterns in human and African of pyramidal neurons in the anterior cingulate great ape cultured fibroblasts. Genome cortex of hominids contains the calcium- Research, 13, 1619–1630. binding protein calretinin. Neuroscience Kaufman, J. A., Ahrens, E. T., Laidlaw, D. H., Letters, 307, 139–142. Zhang, S., & Allman, J. M. (2005). AnatomicalHof, P. R., & Sherwood, C. C. (2007). The analysis of an aye-aye brain (Daubentonia evolution of neuron classes in the neocortex of madagascariensis, primates: Prosimii) mammals. In: J. H. Kaas & L. A. Krubitzer combining histology, structural magnetic (Eds.), Evolution of nervous systems: A resonance imaging, and diffusion-tensor comprehensive reference. Vol. 3: Mammals imaging. Anatomical Records Part A: Discovery (pp. 113–124). Amsterdam: Elsevier. of Molecular, Cellular, and Evolutionary Biology,Hoffman, G. E., & Le, W. W. (2004). Just cool it! 287, 1026–1037. Cryoprotectant anti-freeze in immuno- Khaitovich, P., Muetzel, B., She, X., Lachmann, M., cytochemistry and in situ hybridization. Hellmann, I., Dietzsch, J., et al. (2004). Regional Peptides, 25, 425–431. patterns of gene expression in human andHolloway, R. L., Jr. (1966). Cranial capacity, neural chimpanzee brains. Genome Research, 14, reorganization, and hominid evolution: A 1462–1473.

REINVENTING PRIMATE NEUROSCIENCE FOR THE TWENTY-FIRST CENTURY 449King, M. C., & Wilson, A. C. (1975). Evolution at Mori, S., & Zhang, J. (2006). Principles of two levels in humans and chimpanzees. Science, diffusion tensor imaging and its applications 188, 107–116. to basic neuroscience research. Neuron, 51,Klein, M. E., Impey, S., & Goodman, R. H. (2005). 527–539. Role reversal: The regulation of neuronal gene Mufson, E. J., Brady, D. R., & Kordower, J. H. expression by microRNAs. Current Opinions in (1990). Tracing neuronal connections in Neurobiology, 15, 507–513. postmortem human hippocampal complexKohler, R. E. (1994). Lords of the fly: Drosophila with the carbocyanine dye DiI. Neurobiology genetics and the experimental life. Chicago: and Aging, 11, 649–653. University of Chicago Press. Murphy, W. J., Eizirik, E., O’Brien, S. J., Madsen,Lashley, K. S. (1931). Mass action in cerebral O., Scally, M., Douady, C. J., et al. (2001). function. Science, 73, 245–254. Resolution of the early placental mammalLashley, K. S. (1949). Persistent problems in the radiation using Bayesian phylogenetics. evolution of mind. Quarterly Review of Biology, Science, 294, 2348–2351. 24, 28–42. Nahon, J. L. (2003). Birth of ’human-specific’ genesLe Gros Clark, W. E. (1959). The antecedents of during primate evolution. Genetica, 118, man. Edinburgh: Edinburgh University Press. 193–208.Lewis, D. A. (2002). The human brain revisited: Nimchinsky, E. A., Gilissen, E., Allman, J. M., Perl, Opportunities and challenges in postmortem D. P., Erwin, J. M., & Hof, P. R. (1999). A studies of psychiatric disorders. neuronal morphologic type unique to humans Neuropsychopharmacology, 26, 143–154. and great apes. Proceedings of the NationalLogan, C. A. (1999). The altered rationale for the Academy of Sciences, 96, 5268–5273. choice of a standard animal in experimental Nuttall, G. (1904). Blood immunity and blood psychology: Henry H. Donaldson, Adolf relationships. Cambridge: Cambridge Meyer and ’the’ albino rat. History of University Press. Psychology, 2, 3–24. Ohno, S. (1970). Evolution by gene duplication.Logan, C. A. (2002). Before there were standards: Berlin, New York: Springer-Verlag. The role of test animals in the production of Oldham, M. C., Horvath, S., & Geschwind, D. empirical generality in physiology. Journal of the H. (2006). Conservation and evolution of History of Biology, 35, 329–363. gene coexpression networks in human andLogan, C. A. (2005). The legacy of Adolf Meyer’s chimpanzee brains. Proceedings of the comparative approach: Worcester rats and the National Academy of Sciences, 103, 17973– strange birth of the animal model. Integrative 17978. Physiological and Behavioral Science, 40, Padberg, J., Franca, J. G., Cooke, D. F., Soares, J. G., 169–181. Rosa, M. G., Fiorani, M., Jr., et al. (2007).Logothetis, N. K., Guggenberger, H., Peled, S., & Parallel evolution of cortical areas involved in Pauls, J. (1999). Functional imaging of the skilled hand use. Journal of Neuroscience, 27, monkey brain. Nature Neuroscience, 2, 555–562. 10106–10115.Lovejoy, A. O. (1964). The great chain of being: A Pakkenberg, B., & Gundersen, H. J. (1997). study of the history of an idea. Cambridge: Neocortical neuron number in humans: Effect Harvard University Press. of sex and age. Journal of ComparativeLowe, M. J., Mock, B. J., & Sorenson, J. A. (1998). Neurology, 384, 312–320. Functional connectivity in single and multislice Peltier, S. J., Kerssens, C., Hamann, S. B., Sebel, P. echoplanar imaging using resting-state S., Byas-Smith, M., & Hu, X. (2005). Functional fluctuations. Neuroimage, 7, 119–132. connectivity changes with concentration ofMacLeod, C. E., Zilles, K., Schleicher, A., Rilling, sevoflurane anesthesia. Neuroreport, 16, J. K., & Gibson, K. R. (2003). Expansion of the 285–288. neocerebellum in Hominoidea. Journal of Pollard, K. S., Salama, S. R., King, B., Kern, A. D., Human Evolution, 44, 401–429. Dreszer, T., Katzman, S., et al. (2006a). ForcesMarino,L.,Rilling,J.K.,Lin,S.K.,&Ridgway,S.H.(2000). shaping the fastest evolving regions in the Relative volume of the cerebellum in dolphins and human genome. PLoS Genetics, 2, e168. comparison with anthropoid primates. Brain, Pollard, K. S., Salama, S. R., Lambert, N., Lambot, Behavior,andEvolution,56,204–211. M. A., Coppens, S., Pedersen, J. S., et al.

450 PRIMATE NEUROETHOLOGY (2006b). An RNA gene expressed during and calbindin immunoreactivity in layer 4A. cortical development evolved rapidly in Cerebral Cortex, 12, 671–691. humans. Nature, 443, 167–172. Preuss, T. M., Gray, D., & Cusick, C. G. (1998)Popesco, M. C., Maclaren, E. J., Hopkins, J., Subdivisions of the motor and somatosensory Dumas, L., Cox, M., Meltesen, L., et al. (2006). thalamus of primates revealed with Human lineage-specific amplification, Wisteria floribunda agglutinin histochemistry. selection, and neuronal expression of Somatosensory and Motor Research, 15, 211–219. DUF1220 domains. Science, 313, 1304–1307. Preuss, T. M., Qi, H., & Kaas, J. H. (1999).Preuss, T. M. (1995). The argument from animals Distinctive compartmental organization of to humans in cognitive neuroscience. In: M. S. human primary visual cortex. Proceedings of the Gazzaniga (Ed.), The cognitive neurosciences National Academy of Sciences, 96, 11601–11606. (pp. 1227–1241). Cambridge, MA: MIT Press. Preuss, T. M., Stepniewska, I., Jain, N., & Kaas, J. H. (Reprinted in M. S. Gazzaniga [Ed.], Cognitive (1997). Multiple divisions of macaque precentral neuroscience: A reader. Oxford: Blackwell motor cortex identified with neurofilament Publishers, 2000.) antibody SMI-32. Brain Research, 767, 148–153.Preuss, T. M. (2000a). Taking the measure of Purvis, A. (1995). A composite estimate of primate diversity: Comparative alternatives to the phylogeny. Philosophical Transactions of the model-animal paradigm in cortical Royal Society of London Series B: Biological neuroscience. Brain, Behavior, and Evolution, Science, 348, 405–421. 55, 287–299. Rader, K. A. (2004). Making mice: StandardizingPreuss, T. M. (2000b). What’s human about the animals for American biomedical research, human brain? In M. S. Gazzaniga (Ed.), The new 1900–1955. Princeton, NJ: Princeton cognitive neurosciences. Second edition (pp. University Press. 1219–1234). Cambridge, MA: MIT Press. Raghanti, M. A., Stimpson, C. D., Marcinkiewicz,Preuss, T. M. (2001). The discovery of cerebral J. L., Erwin, J. M., Hof, P. R., & Sherwood, C. C. diversity: An unwelcome scientific revolution. (2007). Differences in cortical serotonergic In: D. Falk & K. Gibson (Eds.), Evolutionary innervation among humans, chimpanzees, and anatomy of the primate cerebral cortex (pp. macaque monkeys: A comparative study. 138–164). Cambridge: Cambridge University Cerebral Cortex . Press. Raghanti, M. A., Stimpson, C. D.,Preuss, T. M. (2004). Specializations of the human Marcinkiewicz, J. L., Erwin, J. M., Hof, P. visual system: The monkey model meets human R., & Sherwood, C. C. (2008). Cholinergic reality. In: J. H. Kaas & C. E. Collins (Eds.), The innervation of the frontal cortex: Differences primate visual system (pp. 231–259). Boca among humans, chimpanzees, and macaque Raton, FL: CRC Press. monkeys. Journal of Comparative Neurology,Preuss, T. M. (2006). Who’s afraid of Homo 506, 409–424. sapiens? Journal of Biomedical Discovery and Ramnani, N., Behrens, T. E., Johansen-Berg, H., Collaboration, 1, 17. Richter, M. C., Pinsk, M. A., Andersson, J. L.,Preuss, T. M. (2007a). Evolutionary specializations et al. (2006). The evolution of prefrontal inputs of primate brain systems. In: M. J. Ravosa & M. to the cortico-pontine system: Diffusion Dagosto (Eds.), Primate origins: Evolution and imaging evidence from Macaque monkeys and adaptations (pp. 625–675). New York: Springer. humans. Cerebral Cortex, 16, 811–818.Preuss, T. M. (2007b). Primate brain evolution in Ramnani, N., Behrens, T. E., Penny, W., & Matthews, phylogenetic context. In: J. H. Kaas & T. M. P. M. (2004). New approaches for exploring Preuss (Eds.), Evolution of nervous systems. anatomical and functional connectivity in the Vol. 4: Primates (pp. 1–34). Oxford: Elsevier. human brain. Biological Psychiatry, 56, 613–619.Preuss, T. M., Caceres, M., Oldham, M. C., & Richards, R. J. (1987). Darwin and the emergence of Geschwind, D. H. (2004). Human brain evolutionary theories of mind and behavior. evolution: Insights from microarrays. Nature Chicago: University of Chicago Press. Review Genetics, 5, 850–860. Richards, R. J. (1992). The meaning of evolution.Preuss, T. M., & Coleman, G. Q. (2002). Human- Chicago: University of Chicago Press. specific organization of primary visual cortex: Rilling, J. K. (2006). Human and nonhuman Alternating compartments of dense Cat-301 primate brains: Are they allometrically scaled

REINVENTING PRIMATE NEUROSCIENCE FOR THE TWENTY-FIRST CENTURY 451 versions of the same design? Evolution International Conference on Alzheimer’s Disease Anthropology, 15, 65–77. and Related Disorders (ICAD), Abstract,Rilling, J. K. (2007). The evolution of the Presentation P2-0005 cerebellum in anthropoid primates. In: J. H. Ross, C. F., & Martin, R. D. (2007). The role of Kaas & T. M. Preuss (Eds.), Evolution of vision in the origin and evolution of primates. nervous systems. Vol. 4: Primates In: J. H. Kaas & T. M. Preuss (Eds.), Evolution of (pp. 149–156). Oxford: Elsevier. nervous systems. Vol. 4: Primates (pp. 60–78).Rilling, J. K., Barks, S. K., Parr, L. A., Preuss, T. M., Oxford: Elsevier. Faber, T. L., Pagnoni, G., et al. (2007). A Schenker, N. M., Desgouttes, A. M., & comparison of resting-state brain activity in Semendeferi, K. (2005). Neural connectivity humans and chimpanzees. Proceedings of the and cortical substrates of cognition in National Academy of Sciences, 104, hominoids. Journal of Human Evolution, 49, 17146–17151. 547–569.Rilling, J. K., Glasser, M. F., Preuss, T. M., Ma, X., Schmitz, C., & Hof, P. R. (2005). Design-based Zhao, T., Hu, X., et al. (2008). The evolution of stereology in neuroscience. Neuroscience, 130, the arcuate fasciculus revealed with 813–831. comparative DTI. Nature Neuroscience, 11, Schoenemann, P. T., Sheehan, M. J., & Glotzer, 426–428. L. D. (2005). Prefrontal white matter volumeRilling, J. K., & Insel, T. R. (1998). Evolution of is disproportionately larger in humans than in the cerebellum in primates: Differences in other primates. Nature Neuroscience, 8, relative volume among monkeys, apes and 242–252. humans. Brain, Behavior, and Evolution, 52, Seeley, W. W., Menon, V., Schatzberg, A. F., 308–314. Keller, J., Glover, G. H., Kenna, H., et al.Rilling, J. K., & Insel, T. R. (1999a). Differential (2007). Dissociable intrinsic connectivity expansion of neural projection systems in networks for salience processing and primate brain evolution. Neuroreport, 10, executive control. Journal of Neuroscience, 27, 1453–1459. 2349–2356.Rilling, J. K., & Insel, T. R. (1999b). The primate Semendeferi, K., Damasio, H., Frank, R., & neocortex in comparative perspective using Van Hoesen, G. W. (1997). The evolution magnetic resonance imaging. Journal of of the frontal lobes: A volumetric analysis Human Evolution, 37, 191–223. based on three-dimensional reconstructionsRilling, J. K., & Seligman, R. A. (2002). A of magnetic resonance scans of human and quantitative morphometric comparative ape brains. Journal of Human Evolution, 32, analysis of the primate temporal lobe. Journal 375–388. of Human Evolution, 42, 505–533. Semendeferi, K., Lu, A., Schenker, N., & Damasio,Rilling, J. K., Winslow, J. T., & Kilts, C. D. (2004). H. (2002). Humans and great apes share a large The neural correlates of mate competition in frontal cortex. Nature Neuroscience, 5, 272– dominant male rhesus macaques. Biological 276. Psychiatry, 56, 364–375. Sherwood, C., & Hof, P. (2007). The evolution ofRilling, J. K., Winslow, J. T., O’Brien, D., Gutman, D. neuron types and cortical histology in apes and A., Hoffman, J. M., & Kilts, C. D. (2001). Neural humans. In: J. Kaas & T. Preuss (Eds.), correlates of maternal separation in rhesus Evolution of nervous systems: A comprehensive monkeys. Biological Psychiatry, 49, 146–157. reference. Volume 4: Primates (pp. 355–378).Rockel, A. J., Hiorns, R. W., & Powell, T. P. S. Amsterdam: Elsevier. (1980). The basic uniformity of structure of Sherwood, C. C., Lee, P. W., Rivara, C. B., the neocortex. Brain, 103, 221–224. Holloway, R. L., Gilissen, E. P., Simmons, R.Rosa, M. G., & Krubitzer, L. A. (1999). The M., et al. (2003). Evolution of specialized evolution of visual cortex: Where is V2? pyramidal neurons in primate visual and Trends in Neuroscience, 22, 242–248. motor cortex. Brain, Behavior, and Evolution,Rosen, R. F., Levine, H., III, Murphy, M. P., Preuss, T. 61, 28–44. M., Ghiso, J. A., Farberg, A., et al. (2006). Cerebral Sherwood, C. C., Raghanti, M. A., Stimpson, C. D., Ab accumulation in Alzheimer’s disease, aged Bonar, C. J., de Sousa, A. A., Preuss, T. M., et al. humans, and aged nonhuman primates. (2007). Scaling of inhibitory interneurons in

452 PRIMATE NEUROETHOLOGY areas v1 and v2 of anthropoid primates as chimpanzees and humans as revealed by revealed by calcium-binding protein genome-wide phylogenetic analysis of brain immunohistochemistry. Brain, Behavior, and gene expression profiles. Proceedings of the Evolution, 69, 176–195. National Academy of Sciences, 101, 2957–Sherwood, C. C., Stimpson, C. D., Raghanti, M. A., 2962. Wildman, D. E., Uddin, M., Grossman, L. I., Varki, A. (2006). Nothing in glycobiology makes et al. (2006). Evolution of increased glia-neuron sense, except in the light of evolution. Cell, 126, ratios in the human frontal cortex. Proceedings 841–845. of the National Academy of Sciences, 103, Varki, A., & Altheide, T. K. (2005). Comparing the 13606–13611. human and chimpanzee genomes: Searching forSikela, J. M. (2006). The jewels of our genome: The needles in a haystack. Genome Research, 15, search for the genomic changes underlying the 1746–1758. evolutionarily unique capacities of the human Vincent, J. L., Patel, G. H., Fox, M. D., Snyder, A. Z., brain. PLoS Genetics, 2, e80. Baker, J. T., Van Essen, D. C., et al. (2007).Stepniewska, I., Preuss, T. M., & Kaas, J. H. Intrinsic functional architecture in the (1993). Architectonics, somatotopic anaesthetized monkey brain. Nature, 447, 83–86. organization, and ipsilateral cortical Vogt, C., & Vogt, O. (1919). Allgemeine Ergebnisse connections of the primary motor area (M1) unserer Hirnforschung. Journal of Psychology of owl monkeys. Journal of Comparative and Neurology, 25, 221–473. Neurology, 330, 238–271. Walters, N. B., Egan, G. F., Kril, J. J., Kean, M.,Sun, S. W., Neil, J. J., & Song, S. K. (2003). Relative Waley, P., Jenkinson, M., et al. (2003). In vivo indices of water diffusion anisotropy are identification of human cortical areas using equivalent in live and formalin-fixed mouse high-resolution MRI: An approach to cerebral brains. Magnetic Resonance in Medicine, 50, structure-function correlation. Proceedings of 743–748. the National Academy of Sciences, 100,Susman, M. W., Eroglu, C., Chakraborty, C., 2981–2986. Hubermann, A. D., Green, E. M., Annis, D., Watson, R. E., Jr., Wiegand, S. J., Clough, R. W., & et al. (2007). Identification of the Hoffman, G. E. (1986). Use of cryoprotectant thrombospondin receptor that promotes CNS to maintain long-term peptide synaptogenesis. Society of Neuroscience, immunoreactivity and tissue morphology. Abstract 571.2. Peptides, 7, 155–159.Tardif, E., & Clarke, S. (2001). Intrinsic West, M. J. (1993). New stereological methods for connectivity of human auditory areas: A counting neurons. Neurobiology and Aging, 14, tracing study with DiI. Eur Journal of 275–285. Neuroscience, 13, 1045–1050. Wheeler, D. L., Chappey, C., Lash, A. E., Leipe,Toga, A. W., Thompson, P. M., Mori, S., Amunts, D. D., Madden, T. L., Schuler, G. D., et al. K., & Zilles, K. (2006). Towards multimodal (2000). Database resources of the National atlases of the human brain. Nature Reviews Center for Biotechnology Information. Nucleic Neuroscience, 7, 952–966. Acids Research, 28, 10–14.Tomassini, V., Jbabdi, S., Klein, J. C., Behrens, T. Wise, S. P. (2007). Evolution of ventral premotor E., Pozzilli, C., Matthews, P. M., et al. (2007). cortex and the primate way of reaching. In: J. H. Diffusion-weighted imaging tractography- Kaas & T. M. Preuss (Eds.), Evolution of nervous based parcellation of the human lateral systems. Vol. 4: Primates (pp. 157–165). Oxford: premotor cortex identifies dorsal and ventral Elsevier. subregions with anatomical and functional Xiong, J., Parsons, L. M., Gao, J. H., & Fox, P. specializations. Journal of Neuroscience, 27, T. (1999). Interregional connectivity to 10259–10269. primary motor cortex revealed using MRITootell, R. B., Hamilton, S. L., & Silverman, M. S. resting state images. Human Brain Mapping, (1985). Topography of cytochrome oxidase 8, 151–156. activity in owl monkey cortex. Journal of Zhang, R., Peng, Y., Wang, W., & Su, B. (2007). Neuroscience, 5, 2786–2800. Rapid evolution of an X-linked microRNAUddin M, Wildman DE, Liu G, Xu W, Johnson cluster in primates. Genome Research, 17, RM, Hof PR, et al. (2004). Sister grouping of 612–617.

REINVENTING PRIMATE NEUROSCIENCE FOR THE TWENTY-FIRST CENTURY 453Zilles, K., Schlaug, G., Geyer, S., Luppino, G., Zilles, K., Schleicher, A., Palomero-Gallagher, N., & Matelli, M., Qu, M., et al. (1996). Anatomy and Amunts, K. (2002). Quantitative analysis of cyto- transmitter receptors of the supplementary and receptor architecture of the human brain. In: motor areas in the human and nonhuman J. Mazziotta & A. Toga (Eds.), Brain mapping. primate brain. Advances in Neurology, 70, 29–43. The methods. (pp. 573–602). Oxford: Elsevier.

CHAPTER 23Ethologically Relevant Movements Mapped on theMotor CortexMichael S. A. GrazianoDOES MOTOR CORTEX The hypothesis of a segregated map of move-DECOMPOSE MOVEMENT INTO ment components was taken to its extreme byMUSCLES, OR INTEGRATE Asanuma and colleagues in their studies of theMUSCLES INTO MOVEMENTS? cat and monkey cortex (e.g., Asanuma & Rosen, 1972; Asanuma & Sakata, 1967; Asanuma &The discovery of motor cortex is one of the land- Ward, 1971). Rather than stimulating the sur-mark events in neuroscience. In 1870, Fritsch face of the cortex with large electrodes, Asanumaand Hitzig found that brief pulses of electricalstimulation applied to the cortex of a dog brain and colleagues inserted a microelectrode into theevoked muscle twitches. The twitch-evoking cortex and stimulated through its exposed tip.sites were arranged across the surface of the The volume of brain tissue directly affected bycortex to form a rough map of the body (Fig. stimulation was probably less than half a milli-23.1). This map of the body was studied further meter in diameter. The stimulation current wasin monkeys (e.g., Beevor & Horsely, 1887; reduced to its threshold, the level at which theFerrier, 1874). Almost 70 years after the initial evoked movement was barely detectable. Usingdiscovery of motor cortex, Penfield published this method, Asanuma and colleagues reportedhis now famous study of the human motor that each stimulation site in cortex was con-cortex (Penfield & Boldrey, 1937), confirming nected to a single muscle or in some cases twothat the human brain also contained a topo- muscles. They argued for the presence of a seg-graphic map of body parts (Fig. 23.2). regated mapping of muscles on the cortexAlthough a map the body could be discerned in (Asanuma, 1975).the motor cortex, the map was not cleanly seg- An alternative explanation exists for theregated into separate muscles or rotations of results of Asanuma and colleagues. In whatseparate joints. Instead, the topography was could be termed the ‘‘iceberg’’ interpretation,blurred and overlapped. This intermixing of each site in cortex contributes to a complexmuscles within the motor cortex map was movement that recruits many muscles thatreported in every major study. As shown in cross many joints. As part of the coordinationFigure 23.2, however, Penfield’s iconic homun- of that movement, some muscles are moreculus appeared to show a clean, segment-by- active than others. By lowering the electricalsegment map of the body. Perhaps Penfield’s stimulation to threshold, by the definition ofsummary diagram inadvertently contributed to ‘‘threshold’’ the evoked movement is reducedthe now widespread and inaccurate idea of a until most of it is no longer detectable andsimple map of body parts laid out on the only the one or two most strongly recruitedmotor cortex. muscles are still detectable above the noise.454

ETHOLOGICALLY RELEVANT MOVEMENTS MAPPED ON THE MOTOR CORTEX 455 Neck Foreleg Hindleg FaceFigure 23.1 The map of stimulation-evoked movements in a dog brain. Each point indicates theapproximate location of a movement center. Stimulation at or near each point evoked movements of theindicated body part. Adapted from Fritsch, G., & Hitzig, E. (1870). Uber die elektrishe Erregbarkeit desGrosshirns. Arch. f. Anat., Physiol und wissenchaftl. Mediz., Leipzig, 300–332. [On the electrical excitabilityof the cerebrum. Translated by G. von Bonin. In: W. W. Nowinski (Ed.), Some papers on the cerebral cortex(pp. 73–96). Springfield, IL: Thomas.]Used with permission.Only the tip of the movement iceberg is cortex, and even each neuron in cortex, affected ameasured. In this interpretation, the use of complex set of muscles.threshold stimulation by its very definition One reasonable hypothesis based on theproduces the misleading illusion of a muscle- results of Cheney and Fetz is that the over-by-muscle map. lapping representation of muscles in cortex Are single muscles, or combinations of muscles, reflects the overlapping use of muscles incontrolled by the motor cortex map? The question normal movement. In support of this hypoth-was answered unambiguously by Cheney and Fetz esis, Nudo and colleagues (1996) found that(1985). They recorded the activity of neurons in when a monkey practices a task that requiresthe motor cortex of monkeys and the activity of the coordinated use of two arm joints, themuscles in the arm and hand. When a neuron in motor cortex develops a larger overlap in thecortex fired an action potential, after a latency of representation of the muscles that actuate thoseapproximately 5 ms, a minute effect could be joints. Martin and colleagues further exploredobserved at the muscles. By averaging the data the role of experience in the development of anover thousands of neuronal spikes, a clear signal overlapping somatotopy (Chakrabarty &could be obtained. This method of spike-triggered Martin, 2000; Martin et al., 2005). They usedaveraging allowed the experimenters to determine microstimulation to map the motor cortex inwhich muscles were directly affected by the firing cats. They found that at birth, the representa-of a neuron in cortex. The results showed that each tion in motor cortex was mainly nonoverlap-cortical neuron could affect the activity of many ping. Separate joints of the forelimb weremuscles crossing many joints. The motor cortex represented in discrete patches in cortex.did not contain a muscle-by-muscle map as sug- During development, as the kitten learned togested by Asanuma and colleagues, but instead perform complex behaviors that required coor-integrated the action of muscles. Each locus in dination among joints, the representations in

456 PRIMATE NEUROETHOLOGYFigure 23.2 The motor homunculus of the human brain. Each point in motor cortex was electricallystimulated and the evoked muscle twitch was noted. Although each cortical point could activate manymuscles, a rough body plan could be discerned. Adapted from Penfield, W., & Rasmussen, T. (1950). Thecerebral cortex of man. A clinical study of localization of function. New York: Macmillan.cortex developed the highly overlapped prop- in which the relevant muscles are represented inerty characteristic of the adult. Individual an integrated fashion.joints were no longer typically represented in Recently we electrically stimulated the motorseparate patches. If the kitten was prevented cortex in monkeys and found the often con-from practicing complex, integrated move- firmed, overlapping map of muscles (Cooke &ments, the motor map did not develop the Graziano, 2004a,b; Graziano et al., 2002, 2005).normal overlap of representations. However, we then extended the electrical stimu- These results suggest that during experience lation to a duration that was behaviorally rele-the motor cortex is trained on, and comes to vant. We stimulated for half a second at a time,reflect, the movement repertoire of the animal. approximating the duration of a monkey’sIf an animal has a need to control individual reaching or grasping, instead of stimulating formuscles (if such an unlikely condition ever the more typical duration of less than 50 ms.exists), the animal might well develop a motor Neurons in motor cortex are not normallycortex map that topographically separates the active in 50-ms bursts but instead, to a firstmuscles. In the more common case that an approximation, are active throughout the dura-animal has a need to control many muscles and tion of a movement. This use of electrical stimu-joints in a coordinated fashion, such as for lation on a behavioral time scale is a standardreaching toward an object or manipulating an method used to study brain areas outside ofobject, its motor cortex develops a topography motor cortex. It is responsible for fundamental

ETHOLOGICALLY RELEVANT MOVEMENTS MAPPED ON THE MOTOR CORTEX 457results such as the mapping of eye movement stimulation on a behaviorally relevant time scaleareas of the brain, the discovery of emotional had not been systematically explored.and motivational functions of the hypothalamus, Using the longer stimulation in motor cortex,and the demonstration that cortical sensory pro- we found that the muscle twitches unfolded intocessing causes sensory perception (e.g., Bruce et complex actions. Stimulation caused the monkeyal. 1985; Caggiula & Hoebel, 1966; Hess, 1957; to perform movements that were common in itsHoebel, 1969; King & Hoebel, 1968; Robinson, normal repertoire. Some of these movements are1972; Robinson & Fuchs, 1969; Salzman et al., drawn in Figure 23.3, traced from video footage.1990; Schiller & Stryker, 1972). Within motor They included ethologically relevant behaviorscortex research, however, the technique of such as closing the hand in a grip while bringing Climbing/leaping Hand in Reach to grasp lower space Manipulate in central space Defense Chewing/ licking Hand to mouthFigure 23.3 Action zones in the motor cortex of the monkey. Seven common categories of movementevoked by electrical stimulation of the cortex on the behaviorally relevant time scale of 0.5 seconds. Imagestraced from video frames. Each image represents the final posture obtained at the end of the stimulation-evoked movement. Within each action zone, movements of similar behavioral category were evoked. Basedon results from Graziano, M. S. A., Taylor, C. S. R., & Moore, T. (2002). Complex movements evoked bymicrostimulation of precentral cortex. Neuron, 34, 841–851; and Graziano, M. S. A., Aflalo, T., & Cooke, D.F.. (2005). Arm movements evoked by electrical stimulation in the motor cortex of monkeys. Journal ofNeurophysiology, 94, 4209–4223. Used with permission.

458 PRIMATE NEUROETHOLOGYthe hand to the mouth and opening the mouth; The evoked movements were also roughlyextending the hand away from the body with the arranged across the cortex according to thegrip opened as if in preparation to grasp an object; location in space to which the movement wasbringing the hand inward to a region just in front directed. The height of the hand was mostof the chest while shaping the fingers, as if to clearly mapped across the cortical surface.manipulate an object; squinting the facial muscles Stimulation of the lower (ventral) regions ofwhile turning the head sharply to one side and cortex commonly drove the hand into upperflinging up the arm, as if to protect the face from space, and stimulation of upper (dorsal)an impending impact; and moving all four limbs regions of cortex commonly drove the handas if leaping or climbing. The behavioral reper- into lower space (Fig. 23.4). Again, an impor-toire of the animal seemed to be rendered onto the tant aspect of the animal’s action repertoire wascortical sheet. mapped across the cortex.Figure 23.4 Progression of spatial locations to which hand movements are directed. Within the armrepresentation of the monkey motor cortex, electrical stimulation in dorsal cortex tended to drive the handinto lower space; stimulation in ventral cortex tended to drive the hand into upper space; and stimulation inintermediate cortical locations tended to drive the hand to intermediate heights. Each image is a tracing ofthe final posture obtained at the end of a stimulation-evoked movement. Each dotted line shows thetrajectory of the hand during the 0.5-second stimulation train. Dots show the position of the hand in 30-msincrements. These trajectories show the convergence of the hand from disparate starting locations toward afinal location. Based on results from Graziano, M. S. A., Taylor, C. S. R., & Moore, T. (2002). Complexmovements evoked by microstimulation of precentral cortex. Neuron, 34, 841–851; and Graziano, M. S. A.,Aflalo, T., & Cooke, D. F.. (2005). Arm movements evoked by electrical stimulation in the motor cortex ofmonkeys. Journal of Neurophysiology, 94, 4209–4223. Used with permission.

ETHOLOGICALLY RELEVANT MOVEMENTS MAPPED ON THE MOTOR CORTEX 459 Just how closely the stimulation-evoked Penfield (Fig. 23.2). The traditional topographicmovements mimic natural movements is still scheme, however, does not capture the actualuncertain. The muscles are activated in combi- pattern of overlaps, fractures, re-representations,nations that apparently mimic normal behavior. and multiple areas separated by fuzzy borders. AHowever, the temporal pattern of muscle commonly accepted parcellation of motor cortexactivity is time-locked to the stimulation train into functionally different areas is shown inand often does not have the smooth, bell-shaped Figure 23.5 (e.g., Dum & Strick, 2002; He et al.,modulation of normal movement (Cooke & 1995; Luppino et al., 1991; Matelli et al., 1985;Graziano, 2004a; Taylor et al., 2002). It is there- Matsuzaka et al., 1992; Preuss et al., 1996;fore probably not correct to think of the stimu- Rizzolatti & Luppino, 2001). This scheme, how-lation as triggering a complete and natural ever, also suffers from the problem of overlap.motor plan. Rather, the motor cortex network Although the motor cortex is clearly heteroge-appears to have learned the statistics of natural neous, the changes across the cortical sheet arebehavior, and the electrical stimulation reveals more graded than absolute. A clean division intothat statistical structure and the manner in separate areas with separate functions is almostwhich different zones of cortex emphasize dif- certainly incorrect, especially since the specificferent parts of the movement repertoire. functions supposedly localized to these distinct areas are largely unknown and the functions that have been tested tend to be found to some degreeDIMENSIONALITY REDUCTION in all motor areas tested.AS A THEORY OF TOPOGRAPHIC Based on our stimulation results, we pro-ORGANIZATION posed a different underlying topographic prin-As described previously, a traditional view of the ciple: the reduction of the many-dimensionalmotor cortex is that it contains a map of the space of the animal’s movement repertoirebody. This map was famously depicted by onto the two-dimensional surface of the cortex 3 cingulae motor areas on medial wall SMA Central sulcus SEF Pre-SMA Primary motor PMDr PMDc F7 F2 FEF F4 F5 PMVr PMVc Arcuate sulcusFigure 23.5 Some commonly accepted divisions of the cortical motor system of the monkey. PMDr,dorsal premotor cortex, rostral division, also sometimes called Field 7 (F7); PMDc, dorsal premotor cortex,caudal division, also sometimes called Field 2 (F2); PMVr, ventral premotor cortex, rostral division, alsosometimes called Field 5 (F5); PMVc, ventral premotor cortex, caudal division, also sometimes called Field 4(F4); SMA, supplementary motor area; SEF, supplementary eye field, a part of SMA; pre-SMA,presupplementary motor area; FEF, frontal eye field.

460 PRIMATE NEUROETHOLOGY(Aflalo & Graziano, 2006). In this principle, local field, as in Figure 23.5. The details of how andcontinuity is optimized. Information processors why different movement types were naturallythat need to interact are arranged physically near pushed to different zones in cortex are describedeach other in cortex, presumably gaining a con- more fully in Graziano and Aflalo (2007). Thenectional advantage. One could term this prin- theory of a dimensionality reduction, in whichciple of cortical organization the rule of ‘‘like the animal’s movement repertoire was flattenedattracts like.’’ However, perfect continuity is onto the cortex, was astonishingly successful innot possible because the movement repertoire explaining the organizational intricacies of ais highly dimensional and the cortical sheet is large swath of cortex, totaling about 20% of theonly two-dimensional. Therefore, the optimum macaque cortical mantle.solution for local continuity is not a perfectsolution, but rather a compromise of many con-flicting constraints. ACTION ZONES WITHIN MOTOR In our proposal, the map of actions in Figure CORTEX23.3 is not by itself correct. It is present in the The following sections describe in detail thedata, but the pattern is noisy and approximate. complex movements evoked by stimulation ofThe map of the body shown in Figure 23.2 is also different zones within the motor cortex of mon-not by itself correct, but is noisy and blurred, as keys, and the manner in which these movementsall motor cortex researchers have noted. The resemble actions in the monkey’s normal beha-map of hand location shown in Figure 23.4 is vioral repertoire.also an approximate one, present statistically butoverlapping and blurred. The height of the hand Hand-to-Mouth Movementsis most clearly mapped across the cortical sur-face, but even this dimension is noisily mapped. Stimulation within a restricted zone in the pre-Our proposal is that many potential ways to central gyrus evoked a characteristic hand-to-parameterize movement are rendered onto the mouth movement. Five components werecortical sheet simultaneously, resulting in a typical of this movement. The grip aperturecompromise that does not neatly follow any closed in the hand contralateral to the elec-single scheme. trode; the forearm supinated and the wrist We tested a mathematical model (Aflalo & flexed, such that the grip was aimed at theGraziano, 2006; Graziano & Aflalo, 2007) that mouth; the elbow flexed and the shouldercollapsed an approximate description of the rotated such that the hand moved precisely tomonkey’s movement repertoire onto a two- the mouth; the mouth opened; and when thedimensional sheet following the principle of head was released from the headbolt andmaximizing local continuity. The model fol- allowed to turn freely, stimulation caused alowed the method of Kohonen for self-orga- rotation of the head to a forward-facing posi-nizing maps (Kohonen, 1982, 2001). The tion, contributing to the alignment of themodel was able to reconstruct organizational mouth and the hand. These five movementfeatures of the cortical motor system, including components occurred simultaneously in a coor-the outlines of the body map as in Figure 23.2, dinated fashion resembling the monkey’s ownthe clustering of movement categories as in voluntary hand-to-mouth movements.Figure 23.3, and an approximate mapping of Although the movements resembled volun-hand position as in Figure 23.4. In addition, tary actions in some respects, they clearly werethe dimensionality reduction resulted in a not true voluntary movements of the monkey’srough division of the motor cortex into zones but were driven by the stimulation. Typically,with different properties, closely following the the movement could be obtained on every sti-outlines of the primary motor area, dorsal and mulation at short latency with mechanical relia-ventral premotor areas, supplementary motor bility for hundreds of trials, with no adaptationarea, frontal eye field, and supplementary eye or degradation. Similar movements could be

ETHOLOGICALLY RELEVANT MOVEMENTS MAPPED ON THE MOTOR CORTEX 461evoked in anesthetized animals, though the behavior that was mechanically produced bymovements were weaker and required greater the stimulated circuitry without intelligentcurrent. A short stimulation, such as a 100-ms flexibility.stimulation, evoked the initial part of the action, a Not all sites within the hand-to-mouth zoneslight closing of the hand, a slight twitching of the resulted in the same movement. For example,hand upward in the direction of the face, and a depending on the cortical site, stimulationslight opening of the mouth. This truncated drove the hand to one side of the mouth ormovement, by itself, makes no behavioral sense. the other, and caused the mouth to open moreIt is best described as a twitch. It makes sense, on the side that the hand approached, as if thehowever, if interpreted as the initial segment of a monkey were placing a piece of food into thelarger movement that has not had time to unfold. side of the jaw, as the animals often do inLonger stimulations, such as for 300 ms, allowed normal behavior. Not only did the exact posi-more of the movement to unfold, but rarely tion of the hand vary from site to site, but theallowed the hand to reach the mouth. Yet longer type of hand grip also varied. For some stimu-stimulation of 500 ms almost always allowed the lation sites the hand shaped into an apparenthand to reach the mouth in an apparent comple- precision grip, the thumb against the side of thetion of the movement. Stimulations longer than forefinger (typical of a macaque precision grip).500 ms, such as those of 1,000 ms, typically For other stimulation sites, the hand shapedcaused the hand, arm, and mouth to freeze at into what we called a hamburger grip, the fourthe final configuration, as if the movement had fingers against each other and opposed to thebeen completed and the activated circuit were thumb, with a gap between, as if for gripping amaintaining the final posture. When the stimula- larger object. These variations suggested thattion train was extended beyond 1 second, almost the zone of cortex was not uniform and notalways the animal appeared to overcome the sti- dedicated to producing a single movement,mulation effects and take back some degree of but instead probably contributed to a range ofcontrol of its arm. Once the stimulation train movements that fell within the large class ofstopped, however, and then was reinitiated, the interactions between the hand and the mouth.hand would move directly back to the mouth. In normal monkey behavior, the hand is often If the monkey was reaching toward a piece of brought to the mouth to put in food, take outfood at the time of stimulation onset, the hand food, manipulate a piece of food that is in thewould close on empty air and come to the mouth, scratch the lips, pick at the teeth, pushmouth. If the monkey had just grasped a piece food out of the cheek pouches, and so on.of food, stimulation would drive the clenched It is unlikely that the collection of componentshand to the mouth and cause the hand to freeze in a hand-to-mouth movement co-occurred byat the mouth, the food securely gripped in the chance. Even putting aside the specific combina-fingers and the mouth stuck open, until the end tion of body parts, the hand closes rather thanof the stimulation train, at which time the opens (50% chance); the mouth opens ratheranimal would finally be released from the stimu- than closes (50% chance); the forearm supinates,lation-evoked posture and put the food in its aiming the grip at the mouth, rather than pro-mouth. If an obstacle was placed between the nates, aiming the grip away from the mouthhand and the mouth, stimulation caused the (50% chance); the hand moves within about 5hand to move along a direct path toward the cm of the mouth, a ball of space accounting formouth and bump against the obstacle, pressing about 1% of the total workspace of the hand (1%against it throughout the remainder of the sti- chance); and the head turns to a forward posi-mulation, without moving intelligently around tion, within about 5% of its range of motion (5%the obstacle. Therefore, although the stimulation chance). Multiplied, these conservatively esti-evoked a movement of great complexity and mated probabilities yield a P value of 0.00005.coordination, the complexity was also limited. We must dispense with the occasionally sug-The movement resembled a fragment of gested interpretation that the evoked

462 PRIMATE NEUROETHOLOGYmovements are chance collections of twitches contralateral side; a squinting of the musculaturerather than meaningful fragments of the beha- surrounding the eye; a lifting of the upper lip in avioral repertoire. facial grimace that wrinkled the cheek upward In all monkeys tested, the hand-to-mouth toward the eye; a folding of the contralateral earsites were clustered in a lateral, anterior zone against the side of the head; a shrugging of theprobably within the ventral premotor cortex. shoulder, either stronger on or exclusively on theWhether they are in the caudal or rostral divi- contralateral side; a rapid turning of the headsion is unclear. Every monkey tested thus far had away from the contralateral side; a rapid liftinga hand-to-mouth zone, but the exact location of the arm, sweeping the hand and forearm intovaried somewhat, especially in the rostrocaudal the contralateral space near the face as ifdimension. Our current interpretation is that blocking or wiping away a potential threat; andthe hand-to-mouth sites are more likely to lie a centering movement of the eyes (Cooke &within a ventral anterior part of F4 as defined by Graziano, 2004a; Graziano et al., 2002). TheseMatelli and colleagues (1985) and that the dorsal movement components match point for pointpart of F4 emphasizes a different type of action, the components of a normal defensive reactionthe defensive movement. such as when the monkey’s face is puffed with air (Cooke & Graziano, 2003). At other sites, neurons had a tactile receptiveDefensive Movements field on the arm and hand and a visual responseIn a specific zone in the precentral gyrus, neu- to objects looming toward the arm and hand.rons typically respond to tactile stimuli on the Stimulation caused a fast retraction of the handface and arms and to visual stimuli looming to the side or back of the torso. In general, thetoward the tactile receptive fields (Fogassi et al., movement evoked from a site within PZ seemed1996; Gentilucci et al., 1988; Graziano et al., appropriate for defending the part of the body1997; Rizzolatti et al., 1981). Some of the neu- covered by the tactile and visual receptive fieldsrons are trimodal, responding also to auditory of the neurons.stimuli in the space near their tactile receptive We observed apparent summation betweenfields (Graziano et al., 1999). Because of these the stimulation-evoked defensive-like move-distinctive sensory properties, we refer to this ments and actual defensive movements. In thecortical region as the polysensory zone (PZ). summation test, we lowered the stimulating cur-Although all monkeys tested have a PZ, it rent to a point near or below threshold until avaries among animals in size and precise posi- subtle movement was obtained only on sometion (Graziano & Gandhi, 2000). It is typically trials. We then puffed air on the monkey’s face,located just posterior to the bend in the arcuate or presented some other noxious stimulus suchsulcus. In the terminology scheme of Matelli and as a ping pong ball thrown at the animal, evokingcolleagues (1985), it probably corresponds to the a defensive reaction. Within a second after thedorsal part of premotor area F4 where similar actual defensive reaction, we then stimulated thepolysensory neurons have been reported site in PZ. Under this condition, the stimulation(Fogassi et al., 1996; Gentilucci et al., 1988). evoked a robust, super-threshold defensive reac- Stimulation within this zone evokes move- tion. The actual defensive movement seemed toments that closely resemble a natural defense of prime the system such that a low stimulatingthe body surface such as to an impending impact current in PZ could evoke a large effect.or unexpected touch. For example, at some sites, One possibility is that the stimulation of sitesthe neurons had tactile receptive fields on the in PZ evoked a noxious sensory percept to whichside of the face contralateral to the electrode and the monkey then reacted. This possibility is dif-visual receptive fields in the space near that side ficult or impossible to rule out because theof the face. Stimulation of these sites evoked a monkey cannot self-report. However, somedefensive action that included eight compo- observations suggest that it is unlikely.nents: a blink, stronger or exclusively on the Although the stimulation evoked an apparent

ETHOLOGICALLY RELEVANT MOVEMENTS MAPPED ON THE MOTOR CORTEX 463defensive reaction, as soon as the stimulation therefore, is an important confirmation andtrain ended the reaction ended and the monkey strengthens the findings from electricalreturned to feeding itself or playing with toys. stimulation.A brief stimulation, such as for 50 ms, evoked a In addition to evoking defensive-like move-correspondingly brief movement, shorter than ments by inducing bursts of neuronal activity,any behaviorally normal defensive reaction; a bicuculline also altered the monkey’s actuallong stimulation, such as for 1,000 ms, evoked a defensive reaction to an air puff directed at thecorrespondingly sustained movement that ter- face. After the injection of bicuculline into PZ,minated abruptly at the end of the stimulation. the monkey gave an exaggerated defensive reac-An actual noxious stimulus, such as an air puff or tion to the air puff. The magnitude of the defen-a ping pong ball thrown at the face, did not result sive reaction, as measured by facial musclein such tight time-locking to the stimulus, but activity, was approximately 45% larger afterinstead resulted in an extended reaction bicuculline injection than before injection.including general agitation and threats to the Even gently bringing a Q-tip toward the face,experimenter. Moreover, the defensive-like normally evoking little reaction from themovements evoked by stimulation could be still monkey, evoked a pronounced defensive reac-evoked under anesthesia, even when the tion in the monkey with a bicuculline-treatedanesthesia was so deep that the animal did not PZ. Muscle activity during chewing, threatreact to noxious stimuli. faces, and eyebrow movement were not elevated. In order to further test the role of PZ in the The effect was limited to the defensive reaction.coordination of defensive movements, we disin- When muscimol was injected into PZ, therebyhibited neuronal activity in PZ by injecting the inhibiting neuronal activity, the monkey’schemical bicuculline and inhibited neuronal defensive reaction to the air puff was reduced.activity by injecting the chemical muscimol The magnitude of the defensive reaction, as mea-(Cooke & Graziano, 2004b). sured by facial muscle activity, was approxi- When bicuculline was injected into PZ, not mately 30% smaller after muscimol injectiononly did the local neuronal activity increase, but than before injection. Injections into sur-the neurons also began to fire in intense sponta- rounding cortical tissue outside of PZ did notneous bursts of activity with approximately 5 to affect the defensive response to an air puff. These30 seconds between bursts. Each spontaneous chemical manipulations therefore strengthen theburst of neuronal activity was followed at short case for PZ as a sensory-motor interface relatedlatency by the standard set of defensive-like to the defense of the body surface, a corticalmovements, including blinking, squinting, flat- region to which the appropriate visual, tactile,tening the ear against the side of the head, ele- and auditory information is supplied, and fromvating the upper lip, shifting the head away from which emerges the motor command to producethe sensory receptive fields, shrugging the spatially directed defensive reactions.shoulder, rapidly lifting the hand into the spacenear the side of the head as if to block an Manipulation Movementsimpending impact, and centering the gaze.Chemical stimulation of neurons within PZ, Stimulation of another cluster of sites evoked antherefore, produced the same effect as electrical especially varied and complex set of movementsstimulation. This result may seem expected. If that involved the fingers, wrist, and often theelectrical stimulation of PZ evokes a set of move- arm and shoulder, contralateral to the electrode.ments, then surely chemical stimulation should The movements resembled the types of actionstoo. However, chemical stimulation is in some that monkeys typically make when manipu-ways a more specific manipulation, affecting lating, examining, or tearing objects. The fingerlocal neuronal receptors. It does not stimulate movements included an apparent precision gripfibers of passage or induce antidromic activa- (thumb against forefinger), a power grip (fist),tion. The result of chemical stimulation in PZ, or a splaying of the fingers. In some cases a

464 PRIMATE NEUROETHOLOGYsupination or pronation of the forearm partially flexed, as if the hand were reaching tooccurred, rotating the grip one direction or the a closer object. In all of these cases stimulationother. Also in some cases the wrist flexed or caused a convergence to the final posture from aextended. A common action for monkeys is to range of initial postures. These apparentsplay the fingers of one hand, orient the palm reaching sites tended to be located on the gyraltoward the face, and examine the splayed hand, surface just anterior to the ‘‘central space/perhaps searching for stray granules of food. manipulation’’ zone and dorsal to the ‘‘defen-This splayed-hand posture, with the palm sive’’ zone. Because of this relative location, theoriented toward the face, was often evoked on reach-related sites probably lie within the dorsalstimulation within this cortical zone. Monkeys premotor cortex, within its caudal divisioncommonly manipulate objects in a region of (PMDc), where a high proportion of neuronscentral space within about 10 cm of the chest. respond in relation to reaching movementsStimulation within this cortical zone often (e.g., Crammond & Kalaska, 1996; Hochermanevoked a movement of the shoulder and arm & Wise, 1991; Johnson et al., 1996; Messier &that brought the hand into this central region Kalaska, 2000). Typically stimulation of moreof space. A common action for monkeys when rostral sites did not evoke reliable or clearmanipulating objects is to tear the object or pull movements.it in two, the two hands pulling rapidly fromcentral space toward lateral space while the fore- Hand in Lower Spacearms supinate and the hands are tightly gripped. A commonly evoked movement involved a pla-Stimulation within this zone of cortex also cement of the hand in lower space near the feet,sometimes evoked just such a movement, typically with the forearm pronated such thatthough only in the contralateral limb. the palm faced down or inward toward the These sites were clustered in a posterior zone body. These stimulation-evoked movementsthat lay partly on the gyral surface and partly on resembled a common part of the monkey’sthe anterior bank of the central sulcus. This behavioral repertoire in which the hand wascluster probably corresponds to the traditional braced on the ground (Graziano et al., 2004).primary motor hand representation. It may also These sites were typically found just dorsal tocorrespond to the central hand region in the the central space/manipulation sites.motor cortex maps of Kwan and colleagues(1978) and Park and colleagues (2001). We sug- Mouth Movementsgested that this cortical zone may represent a‘‘manual fovea,’’ a repertoire of movements that The aforementioned movement categories wereis related to the manipulation of objects and that evoked from the large arm and hand sector ofis heavily biased toward but not exclusively lim- the lateral motor cortex. When we stimulated inited to hand locations in a central region of space cortex ventral to the arm and hand representa-in front of the chest (Graziano et al., 2004). tion, we obtained movement of the jaw, lips, and tongue, as expected on the basis of the standard body map described for the monkey motorReach-to-Grasp Movements cortex (Woolsey et al., 1952). The mouth move-For some cortical sites stimulation evoked an ments often appeared to be coordinated and ofapparent reach in which the wrist straightened, behavioral significance. For example, stimula-the fingers opened as if to grasp, the forearm tion of one site caused the jaw to attain a par-pronated to orient the grip outward, and the tially open position, the lips to purse slightlyhand extended away from the body. In some toward the contralateral side of the mouth, andcases the hand extended to a region of space as the tongue to move until the tip was placed in afar as 25 cm distant from the body, at arm’s contralateral and slightly protuberant position.length. In other cases the hand converged on a The final oral posture evoked from this sitelocation at a lesser distance, with the elbow resembled an action to acquire a bit of food

ETHOLOGICALLY RELEVANT MOVEMENTS MAPPED ON THE MOTOR CORTEX 465just outside the mouth on the contralateral side. and therefore the tail movements evoked byWe looked for but did not find any obvious stimulation of SMA are consistent with a pos-cortical map in the mouth representation in sible role in locomotion.terms of the spatial location around the mouth Stimulation within the SMA did not alwaystoward which the tongue and lip movement was evoke movements of all four limbs. For example,directed. We also did not obtain movements stimulation of another site in the right hemi-that looked like threat displays, fear grimaces, sphere caused the hips to turn to the left side,or any other social displays. It is likely, however, the left foot to reach out and down as if steppingthat we failed to discover many of the move- to a position lateral to and slightly behind thement types in the mouth representation, since body, and the left arm to reach to a lateral posi-we did not explore it as extensively as we did tion as if to grasp a support.the arm and hand representation. We rarely Although we sometimes tested stimula-obtained rhythmic chewing-like jaw move- tions extended to 1 second or more, we didments, perhaps because we usually stimulated not observe any cyclical stepping movements.for 0.5 seconds instead of the 3 seconds used by Instead, the movements resembled the com-Huang and colleagues (1989), who reported plex adjustments of body and limb often seenrhythmic chewing movements. when monkeys are navigating a complex environment. The climbing-like movements, however, were restricted by the primate chairClimbing/Leaping in which the animal was tested and thereforeIn a medial and anterior region, stimulation could never be compared directly to theevoked especially complex movements that normal climbing, leaping, or complex loco-involved bilateral action of the arms and legs, motor movements of a monkey.movements of the torso, and movements of thetail, often simultaneously from one site. Thesecomplex, whole-body sites correspond roughly COMPLEX MOVEMENTSto the supplementary motor area (SMA), a REPORTED IN OTHER STUDIEScortical region on the crown of the hemisphere Although our studies of motor cortex were theand extending slightly onto the lateral side, just first to systematically map the movement reper-anterior to the primary motor leg representa- toire onto the cortical surface, many experi-tion. Others have also obtained bilateral movements before and since have reported thatments of multiple body parts on stimulating in electrical stimulation of specific, motor-relatedthis area of cortex (e.g., Foerster, 1936; structures can evoke highly complex, species-Luppino et al., 1991; Penfield & Welch, 1951; typical behavior. Some of these studies areWoolsey et al., 1952). reviewed below. Subjectively, the movements resembledclimbing or leaping postures. For example, sti- Midbrainmulation of one site caused the left foot to pressdown against the floor of the primate chair; the In a now classic study, electrical stimulation of aright foot to lift and reach forward with the toes midbrain nucleus in the cat resulted in patternedshaped as if in preparation to grasp; the left hand locomotor behavior (Shik et al., 1969). The exactto reach toward a lower, lateral position while role of this mesencephalic locomotor nucleus, itsshaped as if in preparation to grasp; the right relationship to spinal and cortical control ofhand to reach toward a position above the head locomotion, is still unknown.while shaped as if in preparation to grasp; and Electrical stimulation has long been used tothe tail to curl to one side. The long-tailed maca- study maps of motor output in the superiorques in our experiments do not have prehensile colliculus or, as it is called in nonmammals, thetails. Instead, they use their long, stiff tails optic tectum. The map of saccadic eye move-mainly as balance devices during locomotion, ments in cats and monkeys is perhaps the best-

466 PRIMATE NEUROETHOLOGYknown stimulation result in the colliculus shoulder) and through the facial nucleus (for the(Guitton et al., 1980; Robinson, 1972; Schiller control of the head and mouth).& Stryker, 1972). However, other complexspecies-typical behaviors can be evoked. CortexStimulation of the optic tectum in salamandersevokes a coordinated movement in which the Several studies have confirmed the essential phe-animal orients to a spatial location, reaches out nomenon of complex movements evoked fromwith the forepaws, and opens the mouth as if to the motor cortex, and have now extended theacquire prey (Finkenstadt & Ewert, 1983). In results to other species of animals.rats, stimulation of the part of the map that In the rat, species-typical behavior can berepresents lower visual space evokes orienting evoked by stimulation of motor cortex. Brechtmovements of the head as if the animal were and colleagues (2004) found that intracellularacquiring an object on the ground in front of stimulation of a single cortical neuron evokedit, and stimulation of the part of the map that rhythmic whisking movements. Cramer andrepresents upper visual space evokes retracting, Keller (2006) suggested that the cortically con-defensive-like movements (Dean et al., 1989). trolled whisking actions are mediated by a pro-These movements are consistent with the exi- jection from the motor cortex to a subcorticalgencies of normal life for a rat, in which food is central pattern generator that in turn controlsfound on the ground in lower visual space and the whiskers.enemies attack from above. Haiss and Schwarz (2005) found that the rat motor cortex contains two adjacent zonesSpinal Cord related to the whiskers. Stimulation of oneGiszter and colleagues (1993) electrically stimu- zone on a behavioral time scale (500 ms)lated sites in the spinal cord of frogs and studied evoked rhythmic whisking similar to normalthe effect on the hindleg. The frog’s ankle was exploratory movements. Stimulation of thefixed in a range of different spatial locations. For other zone for 500 ms evoked a retraction ofeach ankle location, the force evoked by stimula- the whiskers on the contralateral side, a closuretion was measured. These stimulation-evoked of the contralateral eye, a facial grimace, andforces formed a convergent force field pointing sometimes a lifting of the contralateral forepawtoward a single location in space, suggesting that to the space beside the face. These results suggestif the ankle were free to move, the foot would that the rat motor cortex, like the monkey motormove to that spatial terminus. Different stimula- cortex, may be organized into zones that empha-tion sites resulted in convergent force fields size different ethologically useful actions, in thisaimed at different spatial locations. case exploratory whisking for one zone and Our results on stimulating the monkey motor defensive actions for the other zone.cortex are similar in that stimulation caused the Ramanathan and colleagues (2006) stimulatedlimb to converge from a range of initial locations the rat motor cortex on a behaviorally relevanttoward a specific final location. Presumably, the time scale (500 ms) and obtained reaching andcortical stimulation operates by recruiting spinal grasping movements of the forepaws. Moreover,circuitry. If the Giszter et al. result is applicable to when a reaching zone in cortex was lesioned, thethe monkey spinal cord, then our results may rat’s ability to reach was compromised. When thedepend on spinally mediated force fields. The cor- rat was retrained to reach, the motor cortex wastical stimulation, however, appears to recruit a found to have reorganized such that reachinghigher-order or more integrated version of the could be electrically evoked from new corticalspinal force fields. We typically found convergence sites. After this rehabilitation, the ability of theof many joints from different body segments. For rat to perform the behavior correlated with theexample, a hand-to-mouth movement involves a amount of cortex that, when stimulated, evokedcoordination of output that passes through the the behavior. These results strongly support thespinal cord (for the control of the hand, arm, and view that the motor cortex is organized to control

ETHOLOGICALLY RELEVANT MOVEMENTS MAPPED ON THE MOTOR CORTEX 467complex, meaningful behavior; that different resemble actions in the monkey’s normalbehaviors are emphasized in different regions of repertoire.cortex; and that these behaviors can be assessed The complex topography within motorthrough electrical stimulation. cortex roughly follows several different possible In the cat motor cortex, Ethier and colleagues organizational schemes, including a map of the(2006) found that stimulation on a behaviorally body, a spatial map of hand location, a clusterrelevant time scale (500 ms) evoked a variety of map of ethologically relevant categories offorepaw movements including apparent reaching action, and a set of commonly designated areasand hooking of the paw as if to acquire an object. with fuzzy borders and largely unknown func- Stepniewska and colleagues (2005) stimu- tions. One possible way to reconcile these manylated the motor and parietal cortex of galagos organizational schemes, each one of which cap-using the behaviorally relevant time scale of 500 tures some of the data, is to hypothesize that thems. They evoked complex movements that cortex is a best-fit rendering of the highly dimen-resembled fragments of the animal’s normal sional movement repertoire on the two-dimen-behavioral repertoire. Different categories of sional cortical sheet. In this view, variousmovement were evoked from different cortical individual dimensions that are of importancezones. The parietal lobe could be segmented into in the movement repertoire are mapped acrossfunctional zones including a hand-to-mouth cortex in noisy, partial topographies. No singlezone, a defensive zone, and a reaching zone. dimension of movement is perfectly mappedSimilar results were obtained in the motor across the cortical surface because of a compro-cortex, but were not studied in as much detail. mise among multiple constraints. The under- In macaque monkeys, stimulation of the ventral lying principle of organization, in this view, isintraparietal area (VIP) evokes movements that not a map of one or another dimension, but aresemble defensive reactions (Cooke & Graziano, rendering of the many-dimensional movement2004a; Cooke et al., 2003; Graziano et al., 2002; space onto the cortex.Thier & Andersen, 1998). The evoked movements It is not yet clear to what extent this etholo-are similar to those evoked from PZ in the motor gical organization of the motor cortex is built incortex. VIP is anatomically connected to PZ (Lewis through evolution, and to what extent it is& Van Essen, 2000; Luppino et al., 1999), and learned during development and adulthood asneurons in VIP respond preferentially to tactile the animal learns its motor repertoire. The dis-stimuli on the face and to visual and auditory tinct connections and anatomical structure ofstimuli near the face, much like neurons in PZ different zones within motor cortex argue for(e.g., Colby et al., 1993; Duhamel et al., 1998; some built-in organization shaped through evo-Schlack et al., 2005). One possibility is that VIP lution to serve the behavioral repertoire. Theand PZ are part of a larger sensory-motor circuit modifications to the map that occur with experi-that contributes to the maintenance of a margin of ence demonstrate that at least some of the cor-safety around the body (Graziano & Cooke, 2006). tical organization reflects learning.SUMMARY REFERENCESThe monkey motor cortex does not appear to Aflalo, T. N., & Graziano, M. S. A. (2006). Possibledecompose movement into the action of sepa- origins of the complex topographicrate muscles. Instead, the control of muscles is organization of motor cortex: Reduction of aextensively overlapped in the cortex, effectively multidimensional space onto a 2-dimensionalallowing for the construction of complex, beha- array. Journal of Neuroscience, 26, 6288–6297. Asanuma, H. (1975). Recent developments in theviorally meaningful movements. Electrical sti- study of the columnar arrangement of neuronsmulation of different zones within cortex tends within the motor cortex. Physiological Reviews,to evoke different categories of movement that 55, 143–156.

468 PRIMATE NEUROETHOLOGYAsanuma, H., & Rosen, I. (1972). Topographical Cooke, D. F., & Graziano, M. S. A. (2004b). Super- organization of cortical efferent zones projecting flinchers and nerves of steel: Defensive to distal forelimb muscles in the monkey. movements altered by chemical manipulation Experimental Brain Research, 14, 243–256. of a cortical motor area. Neuron, 43, 585–593.Asanuma, H., & Sakata, H. (1967) Functional Cooke, D. F., Taylor, C. S. R., Moore, T., & organization of a cortical efferent system Graziano, M. S.A. (2003). Complex movements examined with focal depth stimulation in cats. evoked by microstimulation of the ventral Journal of Neurophysiology, 30, 35–54 intraparietal area. Proceedings of the NationalAsanuma, H., & Ward, J. E. (1971). Pattern of Academy of Sciences, 100, 6163–6168. contraction of distal forelimb muscles Cramer, N. P., & Keller, A. (2006). Cortical control produced by intracortical stimulation in cats. of a whisking central pattern generator. Journal Brain Research, 27, 97–109. of Neurophysiology, 96, 209–217.Beevor, C., & Horsley, V. (1887). A minute analysis Crammond, D. J., & Kalaska, J. F. (1996). Differential (experimental) of the various movements relation of discharge in primary motor cortex and produced by stimulating in the monkey premotor cortex to movements versus actively different regions of the cortical centre for the maintained postures during a reaching task. upper limb, as defined by Professor Ferrier. Experimental Brain Research, 108, 45–61. Philosophical Transactions of the Royal Society Dean, P., Redgrave, P., & Westby, G. W. (1989). of London B, 178, 153–167. Event or emergency? Two response systems inBrecht, M., Schneider, M., Sakmann, B., & Margrie, the mammalian superior colliculus. Trends in T. W. (2004). Whisker movements evoked by Neuroscience, 12, 137–147. stimulation of single pyramidal cells in rat Duhamel, J. R., Colby, C. L., & Goldberg, M. E. motor cortex. Nature, 427, 704–710. (1998). Ventral intraparietal area of theBruce, C. J., Goldberg, M. E., Bushnell, M. C., & macaque: Congruent visual and somatic Stanton, G. B. (1985). Primate frontal eye fields. response properties. Journal of II. Physiological and anatomical correlates of Neurophysiology, 79, 126–136. electrically evoked eye movements. Journal of Dum, R. P., & Strick, P. L. (2002). Motor areas in Neurophysiology, 54, 714–734. the frontal lobe of the primate. Physiology andCaggiula, A. R. & Hoebel, B. G. (1966). Behavior, 77, 677–682. ‘‘Copulation-reward site’’ in the posterior Ethier, C., Brizzi, L., Darling, W. G., & Capaday, C. hypothalamus. Science, 153, 1284–1285. Ethier, C., Brizzi, L., Darling, W. G., & Capaday,Chakrabarty, S., & Martin, J. H. (2000). Postnatal C. (2006). Linear summation of cat motor cortex development of the motor representation in outputs. Journal of Neuroscience, 26, 5574–5581. primary motor cortex. Journal of Ferrier, D. (1874). Experiments on the brain of Neurophysiology, 84, 2582–2594. monkeys – No. 1. Proceedings of the RoyalCheney, P. D., & Fetz, E. E. (1985). Comparable Society of London, 23, 409–430. patterns of muscle facilitation evoked by Finkenstadt, T., & Ewert, J. P. (1983). Visual individual corticomotoneuronal (CM) cells and pattern discrimination through interactions of by single intracortical microstimuli in primates: neural networks: A combined electrical brain Evidence for functional groups of CM cells. stimulation, brain lesion, and extracellular Journal of Neurophysiology, 53, 786–804. recording study in Salamandra salamandra.Colby, C. L., Duhamel, J. R., & Goldberg, M. E. Journal of Comparative Physiology, 153, 99–110. (1993). Ventral intraparietal area of the macaque: Foerster, O. (1936). The motor cortex of man in the Anatomic location and visual response properties. light of Hughlings Jackson’s doctrines. Brain, Journal of Neurophysiology, 69, 902–914. 59, 135–159.Cooke, D. F., & Graziano, M. S. A. (2003). Defensive Fogassi, L., Gallese, V., Fadiga, L., Luppino, G., movements evoked by air puff in monkeys. Matelli, M., & Rizzolatti, G. (1996). Coding of Journal of Neurophysiology, 90, 3317–3329. peripersonal space in inferior premotor cortexCooke, D. F., & Graziano, M. S. A. (2004a). (area F4). Journal of Neurophysiology, 76, 141–157. Sensorimotor integration in the precentral Fritsch, G., & Hitzig, E. (1870). Uber die elektrishe gyrus: Polysensory neurons and defensive Erregbarkeit des Grosshirns. Arch. f. Anat., movements. Journal of Neurophysiology, 91, Physiol und wissenchaftl. Mediz., Leipzig, 1648–1660. 300–332. [On the electrical excitability of the

ETHOLOGICALLY RELEVANT MOVEMENTS MAPPED ON THE MOTOR CORTEX 469 cerebrum. Translated by G. von Bonin. In: W. on the medial surface of the hemisphere. Journal W. Nowinski (Ed.), Some papers on the cerebral of Neuroscience, 15, 3284–3306. cortex (pp. 73–96). Springfield, IL: Thomas.] Hess, W. R. (1957). Functional organization of theGentilucci, M., Fogassi, L., Luppino, G., Matelli, diencephalons. New York: Grune and Stratton. M., Camarda, R., & Rizzolatti, G. (1988). Hocherman, S., & Wise, S. P. (1991). Effects of Functional organization of inferior area 6 in hand movement path on motor cortical the macaque monkey. I. Somatotopy and the activity in awake, behaving rhesus monkeys. control of proximal movements. Experimental Experimental Brain Research, 83, 285–302. Brain Research, 71, 475–490. Hoebel, B. G. (1969). Feeding and self-stimulation.Giszter, S. F., Mussa-Ivaldi, F. A., & Bizzi, E. Annals of the New York Academy of Sciences, 157, (1993). Convergent force fields organized in 758–778. the frog’s spinal cord. Journal of Neuroscience, Huang, C. S., Hiraba, H., Murray, G. M., & Sessle, 13, 467–491. B. J. (1989). Topographical distribution andGraziano, M. S. A., & Aflalo, T. N. (2007). Mapping functional properties of cortically induced behavior repertoire onto the cortex. Neuron, 56, rhythmical jaw movements in the monkey 239–251. (Macaca fascicularis). Journal of Neurophysiology,Graziano, M. S. A., Aflalo, T., & Cooke, D. F. 61, 635–650. (2005). Arm movements evoked by electrical Johnson, P. B., Ferraina, S., Bianchi, L., & Caminiti, stimulation in the motor cortex of monkeys. R. (1996). Cortical networks for visual reaching: Journal of Neurophysiology, 94, 4209–4223. Physiological and anatomical organization ofGraziano, M. S. A., & Cooke, D. F. (2006). Parieto- frontal and parietal lobe arm regions. Cerebral frontal interactions, personal space, and defensive Cortex, 6, 102–119. behavior. Neuropsychologia, 44, 845–859. King, M. B., & Hoebel, B. G. (1968). Killing elicitedGraziano, M. S. A., Cooke, D. F., Taylor, C. S. R., & by brain stimulation in rats. Communications in Moore, T. (2004). Distribution of hand location Behavioral Biology, 2, 173–177 in monkeys during spontaneous behavior. Kohonen, T. (1982). Self-organizing formation of Experimental Brain Research, 155, 30–36. topologically correct feature maps. BiologicalGraziano, M. S. A., & Gandhi, S. (2000). Location Cybernetics, 43, 59–69. of the polysensory zone in the precentral gyrus Kohonen, T. (2001). Self-organizing maps. Berlin: of anesthetized monkeys. Experimental Brain Springer. Research, 135, 259–266. Kwan, H. C., MacKay, W. A., Murphy, J. T., & Wong,Graziano, M. S. A., Hu, X. T., & Gross, C. G. Y. C. (1978). Spatial organization of precentral (1997). Coding the locations of objects in the cortex in awake primates. II. Motor outputs. dark. Science, 277, 239–241. Journal of Neurophysiology, 41, 1120–1131.Graziano, M. S. A., Reiss, L. A. J., & Gross, C. G. Lewis, J. W., & Van Essen, D. C. (2000). (1999). A neuronal representation of the location Corticocortical connections of visual, of nearby sounds. Nature, 397, 428–430. sensorimotor, and multimodal processingGraziano, M. S. A., Taylor, C. S. R., & Moore, T. areas in the parietal lobe of the macaque (2002). Complex movements evoked by monkey. Journal of Comparative Neurology, microstimulation of precentral cortex. Neuron, 428, 112–137. 34, 841–851. Luppino, G., Matelli, M., Camarda, R. M., Gallese,Guitton, D., Crommelinck, M., & Roucoux, A. V., & Rizzolatti, G. (1991). Multiple (1980). Stimulation of the superior colliculus in representations of body movements in mesial the alert cat. I. Eye movements and neck EMG area 6 and the adjacent cingulate cortex: An activity evoked when the head is restrained. intracortical microstimulation study in the Experimental Brain Research, 39, 63–73. macaque monkey. Journal of ComparativeHaiss, F., & Schwarz, C. (2005). Spatial segregation Neurology, 311, 463–482. of different modes of movement control in the Luppino, G., Murata, A., Govoni, P., & Matelli, M. whisker representation of rat primary motor (1999). Largely segregated parietofrontal cortex. Journal of Neuroscience, 25, 1579–1587. connections linking rostral intraparietal cortexHe, S. Q., Dum, R. P., & Strick, P. L. (1995). (areas AIP and VIP) and the ventral premotor Topographic organization of corticospinal cortex (areas F5 and F4). Experimental Brain projections from the frontal lobe: Motor areas Research, 128, 181–187.

470 PRIMATE NEUROETHOLOGYMartin, J. H., Engber, D., & Meng, Z. (2005). Effect Rizzolatti, G., & Luppino, G. (2001). The cortical of forelimb use on postnatal development of the motor system. Neuron, 31, 889–901. forelimb motor representation in primary motor Rizzolatti, G., Scandolara, C., Matelli, M., & cortex of the cat. Journal of Neurophysiology, 93, Gentilucci, M. (1981). Afferent properties of 2822–2831. periarcuate neurons in macaque monkeys. II.Matelli, M., Luppino, G., & Rizzolatti, G. (1985). Visual responses. Behavioral Brain Research, 2, Patterns of cytochrome oxidase activity in the 147–163. frontal agranular cortex of the macaque Robinson, D. A. (1972). Eye movements evoked by monkey. Behavioral Brain Research, 18, 125–136. collicular stimulation in the alert monkey.Matsuzaka, Y., Aizawa, H., & Tanji, J. (1992). A Vision Research, 12, 1795–1808. motor area rostral to the supplementary motor Robinson, D. A., & Fuchs, A. F. (1969). Eye area (presupplementary motor area) in the movements evoked by stimulation of the monkey: Neuronal activity during a learned frontal eye fields. Journal of Neurophysiology, motor task. Journal of Neurophysiology, 68, 32, 637–648. 653–662. Salzman, C. D., Britten, K. H., & Newsome, W. T.Messier, J., & Kalaska, J. F. (2000). Covariation of (1990). Cortical microstimulation influences primate dorsal premotor cell activity with perceptual judgements of motion direction. direction and amplitude during a memorized- Nature, 346, 174–177. delay reaching task. Journal of Neurophysiology, Schiller, P. H., & Stryker, M. (1972). Single-unit 84, 152–165. recording and stimulation in superior colliculusNudo, R. J., Milliken, G. W., Jenkins, W. M., & of the alert rhesus monkey. Journal of Merzenich, M. M. (1996). Use-dependent Neurophysiology, 35, 915–924. alterations of movement representations in Schlack, A., Sterbing, S., Hartung, K., Hoffmann, primary motor cortex of adult squirrel K. P., & Bremmer, F. (2005). Multisensory monkeys. Journal of Neuroscience, 16, 785–807. space representations in the Macaque ventralPark, M. C., Belhaj-Saif, A., Gordon, M., & Cheney, intraparietal area (VIP). Journal of P. D. (2001). Consistent features in the forelimb Neuroscience, 25, 4616–4625. representation of primary motor cortex in Shik, M. L., Severin, F. V., & Orlovsky, G. N. rhesus macaques. Journal of Neuroscience, 21, (1969). Control of walking and running by 2784–2792. means of electrical stimulation of thePenfield, W., & Boldrey, E. (1937). Somatic motor mesencephalon. Electroencephalography and and sensory representation in the cerebral Clinical Neurophysiology, 26, 549. cortex of man as studied by electrical Stepniewska, I., Fang, P. C., & Kaas, J. H. (2005). stimulation. Brain, 60, 389–443. Microstimulation reveals specialized subregionsPenfield, W., & Rasmussen, T. (1950). The cerebral for different complex movements in posterior cortex of man. A clinical study of localization of parietal cortex of prosimian galagos. Proceedings function. New York: Macmillan. of the National Academy of Sciences, 102,Penfield, W., & Welch, K. (1951). The 4878–4883. supplementary motor area of the cerebral Taylor, C. S. R., Cooke, D. F., & Graziano, M. S. A. cortex: A clinical and experimental study. (2002). Complex mapping from precentral cortex American Medical Association Archives of to muscles. Society for Neuroscience Abstracts, 28. Neurology and Psychiatry, 66, 289–317. Thier, P., & Andersen, R. A. (1998). ElectricalPreuss, T. M., Stepniewska, I., & Kaas, J. H. (1996). microstimulation distinguishes distinct saccade- Movement representation in the dorsal and related areas in the posterior parietal cortex. ventral premotor areas of owl monkeys: A Journal of Neurophysiology, 80, 1713–1735. microstimulation study. Journal of Comparative Woolsey, C. N., Settlage, P. H., Meyer, D. R., Neurology, 371, 649–676. Sencer, W., Hamuy, T. P., & Travis, A. M.Ramanathan, D., Conner, J. M., & Tuszynski, M. H. (1952). Pattern of localization in precentral (2006). A form of motor cortical plasticity that and "supplementary" motor areas and their correlates with recovery of function after brain relation to the concept of a premotor area. injury. Proceedings of the National Academy of Association for research in nervous and mental Sciences, 103, 11370–11375. disease, 30, 238–264. New York: Raven Press.

CHAPTER 24Object Recognition: Physiological andComputational InsightsDoris Y. Tsao, Charles F. Cadieu, and Margaret S. LivingstoneINTRODUCTION normally be located) (Maunsell & Van Essen, 1987; Previc, 1990).Visual object recognition is the identification of Understanding the neural computationsa thing in the outside world based on the sense of underlying object recognition is a uniquelyvision. Our eyes are bombarded by a wide variety difficult problem for three reasons. First,of visual forms, from simple shapes like cups and we don’t even know how object recognitionpens, to complicated shapes like keyboards and might be solved computationally. Even thoughsaxophones, to amorphous natural forms like numerous computer models exist, they don’tflowers and bodies. These forms can appear atmultiple poses as they rotate (or as we rotate), come close to rivaling the performance of thecan change size as they move closer or farther, human visual system. Second, a huge expanseand can frequently occlude one another. Yet we of cortical territory is dedicated to objecthave no trouble recognizing them. How does the recognition. Areas TEO and TE, for example,brain transform the unpredictable retinal array are more than 10 times as large as the much-into invariant representations of objects? This studied motion area MT (Felleman & Vanproblem has two aspects: (1) extracting a Essen, 1991). Assuming that multiple levels ofstable, compact, and explicit representation of organization exist within this vast corticalthe forms stimulating our eyes at any moment, expanse, it becomes exceedingly difficult toand (2) comparing the resulting representation map specific computations onto specific sets ofto a stored representation in memory. We will neurons. Thus, single-unit physiologists typi-focus in this chapter on the first aspect: How cally assume that the organization of inferotem-does the primate visual system extract stable poral (IT) cortex is homogeneous and thatrepresentations of the objects in the visual meaningful answers can be obtained from sam-world? pling responses of random populations of IT Visual object recognition is a function of the neurons. Finally, the set of possible objectsinferior temporal lobe, specifically, ventral covers a huge parameter space. While it is clearstream areas V4, TEO, and TE. It has been sug- the brain must use some sort of code that isgested that the specialization of the inferior tem- compact, explicit, and stable to identify objectsporal lobe for object recognition arose during within this space (Connor et al., 2007), it isevolution as an elaboration of the foveal visual unclear what this code might be. To summarize,representation in V1, while the specialization of understanding object recognition is difficult duethe parietal lobe for manipulation of objects in to a lack of effective computer models, the largespace arose as an outgrowth of the lower field number of cortical resources used to solve therepresentation of V1 (where the hands would problem, and a huge parameter space. 471

472 PRIMATE NEUROETHOLOGYFigure 24.1 The challenge of object recognition, exemplified for a face. Over the past three decades, knowledge has different computational approaches to objectgradually accumulated concerning a system in recognition. The central thesis of this chapter isthe temporal lobe that may allow us to overcome that understanding face processing will illumi-these challenges. Both macaques and humans nate the general problem of visual objecthave a specialized system in the temporal lobe recognition.for processing faces that produces an extraor-dinary ability to recognize faces under a variety Functional Architecture of the Inferiorof conditions (Pinsk et al., 2005; Tsao et al., Temporal Lobe2003, 2006, 2008). (Fig. 24.1). This face proces- The functional architecture of the temporal lobesing system provides a spatially limited network sets the stage for the neural processes underlyingwith readily accessible components representing object recognition.a parametrically confined set of objects. Face perception is a microcosm of object recog- General Architecture of the Macaquenition processes, and the solution to the particular Ventral Visual Pathwayproblem of face recognition will undoubtedly yieldinsights into the general problem of object recog- Figure 24.2 shows a schematic of the lateral andnition. The most difficult challenge in object ventral surfaces of the macaque brain. The threerecognition—distinguishing among similar visual major areas of the ventral form processingforms despite substantial changes in appearance pathway are V4, TEO, and TE. Area TE is furtherarising from changes in position, illumination, subdivided into four parts on the basis of anato-occlusion, etc.—is something we can do effort- mical connection criteria: TEpd (dorso-pos-lessly for faces (Fig. 24.1). Although face identifica- terior), TEpv (ventro-posterior), TEad (dorso-tion is often singled out as demanding particular anterior), and TEav (ventro-anterior) (Chengsensitivity to differences between objects sharing a et al., 1997 Saleem & Tanaka, 1996; Yukie &common basic configuration, in fact, such differ- Iwai, 1988). Monkeys with bilateral lesions toences must be represented in the brain for both V4, TEO, and TE show severe and specific defi-faces and nonface objects. It is true that most cits in object recognition tasks (Dean, 1976;humans can easily identify hundreds of faces Gross, 1973). Both V4 and TEO are retinotopi-(Diamond & Carey, 1986), but, even if one cally organized, with TEO containing a coarsecannot recognize a hundred different bottles by but complete representation of the contralateralname, one can certainly distinguish them in pair- visual field (Boussaoud et al., 1991); cells in TEwise discrimination tasks. have large receptive fields centered on the fovea In this chapter, we first discuss the functional (Ito et al., 1995; Kobatake & Tanaka, 1994).architecture of the temporal lobe, with a special The architecture of the inferior temporal lobefocus on the architecture of the system of face- has been studied by classic anatomical tracing.selective areas in macaques and humans. We V4 sends strong projections to TEO from itsthen discuss the physiology of cells in the tem- central visual field representation (Ungerleiderporal lobe, with a focus on the response proper- et al., 2008). TEO in turn sends strong feedfor-ties of face-selective cells. Finally, we discuss ward projections to area TE (Saleem et al., 1993).

OBJECT RECOGNITION: PHYSIOLOGICAL AND COMPUTATIONAL INSIGHTS 473 A V4 Is sts TEO f d STGr TEpd v Lateral view TEad TEav TG amts Ventral view sts TEpd STGr B TEad ots TEpv TEav v4 TFO TF TEav TG 36 TH 28 rs Visual and medial temporal lobe areasFigure 24.2 The ventral visual processing pathway. (A) Lateral view of the right hemisphere of themacaque brain, showing the architectonic subdivisions of inferior temporal cortex. The superiortemporal sulcus (STS) is opened to show the dorsal bank (d), fundus (f), and ventral bank (v).(B) Ventral view of subdivisions of the inferior temporal cortex. ls, lateral sulcus; STGr, superiortemporal gyrus, rostral part; TEad, dorsal subregion of anterior TE; TEav, ventral subregion of anteriorTE; TEpd, dorsal subregion of posterior TE; TEpv, ventral subregion of posterior TE; TG, temporal pole;TH, area TH of parahippocampal cortex; TF, area TF of the parahippocampal cortex; TFO, area TFO of theparahippocampal cortex; 28, entorhrinal cortex; 36, perirhinal cortex. From Saleem, K. S., Kondo, H., &Price, J. L. (2008). Complementary circuits connecting the orbital and medial prefrontal networks with thetemporal, insular, and opercular cortex in the macaque monkey. Journal of Comparative Neurology, 506,659–693. Used with permission.The projection from TEO to TE shows a precise Area TE, the highest purely visual stage of themodularity, with single TEO sites projecting to ventral pathway, sends feedback projections totwo to five clusters in TE (Saleem et al., V4 and TEO, and feedforward projections to1993) (Fig. 24.3A). These clusters are columnar, several polymodal brain sites including the peri-extending across all six cortical layers. Although rhinal cortex, the frontal cortex, the amygdala,the functional properties of the TEO injection and the striatum (Cheng et al., 1997; Saleem &sites and connected TE columns are unknown, it Tanaka, 1996; Suzuki et al., 2000; Webster et al.,seems plausible that this precise anatomical cir- 1991, 1993, 1994). The perirhinal projectionscuitry serves a computational purpose. One from TEav and TEad differ, suggesting thatinteresting possibility is that each network of these areas constitute distinct processing regionsconnected sites in TEO and TE is responsible in TE. Focal TEav injections result in a widelyfor extracting a specific aspect or class of visual distributed labeling in perirhinal cortex, cov-form. ering around half its total extent, whereas

474 PRIMATE NEUROETHOLOGYFigure 24.3 Columnar projections in inferior temporal (IT) cortex. (A) Labeled terminals in TE followinginjections of the anterograde tracer Phaseolus vulgaris leukoagglutinin (PHA) in TEO (indicated by two redspots) form columnar clusters. From Saleem, K. S., Tanaka, K., & Rockland, K. S. (1993). Specific andcolumnar projection from area TEO to TE in the macaque inferotemporal cortex. Cerebral Cortex, 3,454–464. (B) Labeled terminals of intrinsic horizontal axons show patchy columnar arborizationfollowing injection of biocytin into layer 3 in area TE. The injection site is marked by the red asterisk andarrowheads indicate terminal patches. From Fujita, I., & Fujita, T. (1996). Intrinsic connections in themacaque inferior temporal cortex. Journal of Comparative Neurology, 368, 467–486. Used with permission.injections into TEad result in labeling in only a injection site (Fujita & Fujita, 1996) (Fig. 24.3B).small region of lateral perirhinal cortex (Saleem & TEO shows similar, but slightly smaller, columnsTanaka, 1996). Perirhinal cortex has been impli- of connected patches, with the patches slightlycated in encoding long-term memory of objects smaller. Again, it seems plausible that each net-(Meunier et al., 1993) and association between work of locally connected patches forms a systemfamiliar objects (Erickson & Desimone, 1999; specialized to represent a particular aspect of form.Erickson et al., 2000; Miyashita et al., 1996). The Electrophysiology and optical imaging stu-widespread projections from TEav to perirhinal dies also suggest a columnar functional architec-could thus facilitate association between different ture in TE. Neighboring cells are responsive toshapes (e.g., different views of the same object). similar visual features of objects (Fujita et al., In addition to projections to other areas, both 1992). Optical imaging reveals that spots %0.5areas TEO and TE have local connections. mm wide in TE are activated when a monkeyInjections into TE reveal horizontal axons in views a particular object (Tsunoda et al., 2001;layers 2 and 3 terminating in patches 0.5 mm Wang et al., 1996), with different spots activatedwide and cylindrical in shape, spanning layers 1 by different objects. The cortical regions acti-through 3 or even to layers 4 and 5, with the vated by three different objects are shown infarthest patches up to 4 mm distant from the Figure 24.4A. Single-unit recordings confirmed

Figure 24.4 Distributed object representations in macaques and humans. (A) Top: Spots activated by different stimuli (1, 2, and 3, right) as revealed by optical imaging, together with numbered electrode sites. Bottom: Reponses to the three stimuli at each of 17 electrode sites. Scale bar, 1.0 mm. Tsunoda, K., Yamane, Y., Nishizaki, M., & Tanifuji, M. (2001). Complex objects are represented in macaque inferotemporal cortex by the combination of feature columns. Nature Neuroscience, 4, 832–838. (B) Example images from the eight stimulus categories used by Haxby and colleagues (2001) to examine object representations in the human temporal lobe. (C) The pattern of responses to four categories (faces, houses, chairs, shoes) measured separately from data obtained on even-numbered and odd-numbered (two horizontal slices from a single subject are shown). Within-category correlations (0.81, 0.87, 0.45, 0.35) are significantly higher than cross category comparisons (–0.4,475 –0.47, –0.12, –0.1), allowing prediction of the viewed object category. B and C from Haxby, J. V., Gobbini, M. I., Furey, M. L., Ishai, A., Schouten, J. L., & Pietrini, P. (2001). Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science, 293, 2425–2430. Used with permission.

476 PRIMATE NEUROETHOLOGYthe responsiveness of cells within these spots to centimeters (Boynton, 2005; Kamitani & Tong,the particular objects used to activate them (and 2005). Therefore, the distinct fMRI responseabsence of responsiveness outside these spots). patterns to different objects observed by HaxbyFor example, an apple activated spots 1, 2, and 3, and colleagues may instead arise from a coarserand single units in these three sites also scale functional organization, possibly, networksresponded strongly to the apple. Since multiple of connected, clustered columns.spots, distributed over several millimeters ofcortex, were active in response to a single Architecture of the Face Processing Systemimage, and since the size of these spots is similar in Humansto the size of termination zones observed intracer experiments (Fig. 24.3), it is conceivable The existence of a face-selective area in thethat these spots correspond to connected human brain was first suggested by neurolo-columns. gical observations. That there is a specialized area for processing upright faces fits with one of the most striking findings from theDistributed Architecture of the Human neuropsychology literature: Patient C.K., whoVentral Pathway is severely impaired at object recognition,Human functional magnetic resonance imaging including many basic midlevel visual pro-(fMRI) studies provide a much coarser picture cesses, is nonetheless 100% normal at faceof functional architecture of ventral temporal recognition (Moscovitch et al., 1997). C.K.’scortex, but are consistent with results from dissociation is illustrated by his perception ofmacaques. The wide field of view of fMRI the face made up of vegetables by Arcimbaldo—makes it possible to address the question of C.K. sees the face, but not the constituent vege-how distributed the representation of an object tables. His pattern of deficits indicates that faceis. Haxby and colleagues (2001) measured processing is not simply a final stage tackedresponse patterns in ventral temporal cortex onto the end of the nonface object recognitionevoked by viewing of faces, cats, five categories pathway, but rather a different pathway thatof manmade objects, and nonsense pictures branches away from object recognition early in(Fig. 24.4B). They found that object categories the visual hierarchy.are represented by widely distributed and over- The first direct demonstration of face-selectivelapping representations in ventral temporal activation in a human brain area came from posi-cortex. Each category elicited a distinct and tron emission tomography (PET) studies showingreproducible distributed response pattern that activation of the fusiform gyrus in a variety of facecould be used to identify which category was perception tasks (Haxby et al., 1991; Sergent et al.,seen (Fig. 24.4C). 1992) and event-related potential (ERP) studies The distinctiveness of the response patterns is showing a face-selective event-related potential inapparent even at a scale of centimeters (Fig. the fusiform gyrus (Allison et al., 1994; McCarthy24.4C, compare response to chairs with response et al., 1999; Puce et al., 1999). Subsequently fMRIto shoes). This suggests that the structure in the revealed more of the specificity of these corticalresponse patterns reflects a level of functional regions for faces, with demonstrations of fusiformorganization more macroscopic than 0.5-mm regions that responded more strongly to facescolumns. While multivoxel readout techniques than to letter strings and textures (Puce et al.,might be capable of distinguishing response pat- 1996), flowers (McCarthy et al., 1997), everydayterns that differ only at a scale of 0.5-mm col- objects, houses, and hands (Kanwisher et al.,umns (by pooling minute but consistent 1997). Face-specific fMRI activation can be seencategory differences across a large number of in the superior temporal sulcus (STS), in part ofvoxels) (Norman et al., 2006), such functional the occipital lobe (the ‘‘occipital face area,’’ ordistinctions should not be readily apparent using OFA), and most robustly, on the lateral side ofa technique with a resolution on the scale of the right midfusiform gyrus (the ‘‘fusiform face

OBJECT RECOGNITION: PHYSIOLOGICAL AND COMPUTATIONAL INSIGHTS 477Figure 24.5 Comparison of face-selective regions between humans and macaques. (A) Face-selectiveregions shown on a flattened map of the human right hemisphere. (B) Face-selective regions shown on aflattened map of a macaque right hemisphere. By comparison, the human face-selective regions are shiftedventrally away from the STS relative to the macaque face patches. AF, anterior face patch in the STS fundus;AFP1, anterior face patch 1; AL, anterior face patch on the STS lip; AM, anterior face patch on the ventralsurface of IT just lateral and anterior to the AMTS; FFA, fusiform face area; MF, middle face patch in the STSfundus; ML, middle face patch on the STS lip; OFA, occipital face area; PL, posterior face patch; STS-FA,superior temporal sulcus face area. aos, anterior occipital; cas, calcarine; cos, collateral; ios, inferior occipitalsulcus; ips, intraparietal; its, inferior temporal; los, lateral occipital; lots, lateral occipitotemporal; ls, lunatesulcus; ots, occipitotemporal sulcus; pcs, precentral; sf, Sylvian fissure sts, superior temporal; pos, parieto-occipital; tos, transoccipital. Tsao, D. Y., Moeller, S., & Freiwald, W. A. (2008). Comparing face patchsystems in macaques and humans. Biological Sciences – Neuroscience: PNAS 2008 105: 19514–19519;published online November 25, 2008 [DOI: 10.1073/pnas.0809662105]. Used with permission.area,’’ or FFA) (Kanwisher et al., 1997) (Fig. many shape-specific types of columns. However,24.5A). Recent functional imaging provides evi- this view was inconsistent with emerging evi-dence for a fourth face-selective area in the human dence from human neurology and functionalbrain in the anterior temporal lobe, within the imaging that human face processing was localizedcollateral sulcus (Tsao et al., 2008) (Fig. 24.5A). to specific, reproducible regions of the temporal lobe. One possible explanation was that only aArchitecture of the Face Processing System fraction of cells within fMRI-identified face-selec-in Macaques tive areas were actually face selective. AnotherFace-selective cells have been found scattered explanation was that the face areas observed inthroughout the temporal lobe, though they fMRI experiments were unique to humans.tended to be found in clusters (Perrett et al., Finally, it was also possible that macaques have1984). Because other kinds of shape selectivities face areas composed entirely of face cells, butalso tend to be clustered (Desimone et al., 1984; previous single-unit recordings, which sampledFujita et al., 1992; Tanaka et al., 1991; Wang et al., inferior temporal cortex randomly, did not con-1996), it was assumed that within the temporal sistently target these areas.lobe there was a columnar organization for shape, The apparent discrepancy was resolved byin which face columns represented just one of Tsao and colleagues (Tsao et al., 2003, 2006,

478 PRIMATE NEUROETHOLOGY2008), who found that in monkeys, as in the interaural canal, corresponding tohumans, face processing, as revealed by func- posterior TE/anterior TEO). The fact that antional imaging, is localized to six discrete regions area consisting almost entirely of face-selectiveof the temporal lobe (Fig. 24.5B). These six cells exists so early in the ventral stream ispatches are distributed along the anterior-pos- consistent with the idea that the face proces-terior axis of the temporal lobe and organized sing pathway is a completely different pathwayinto one posterior patch on the lateral surface of from the nonface object recognition pathway,TEO (‘‘PL’’, for posterior lateral); two middle and that it branches away early in the visualface patches in posterior TE, one located in the hierarchy. The existence of an area consistingfundus of the STS (‘‘MF,’’ for middle fundus) almost entirely of face-selective cells so early inand one on the lower lip of the STS (‘‘ML,’’ for the ventral stream furthermore implies that themiddle lateral); and three patches in anterior TE, face processing pathway is gated by a faceone located near the fundus of the STS (‘‘AF,’’ detection stage, at which nonface objects arefor anterior fundus), one on the lower lip of the filtered out. As we propose later, the existenceSTS and adjacent gyrus, in TEad (‘‘AL,’’ for of this detection gate may account for the see-anterior lateral), and one more medially on the mingly special ‘‘holistic’’ aspect of faceventral surface, just lateral and anterior to the processing.anterior-medial temporal sulcus, in TEav What is the functional significance of the(‘‘AM,’’ for anterior medial). anatomical localization of face processing? The The face-patch system presents us with a cerebral cortex is functionally parcellated:new kind of functional organization in TE. Neurons concerned with similar things areThe components of this network are coarser organized into areas and columns, each(few millimeters in diameter) than feature col- having extensive interconnections andumns of inferotemporal cortex (%0.5 mm in common inputs and outputs (Mountcastle,diameter; Fujita et al., 1992; Wang et al., 1996), 1997). Face processing, an identifiable and dis-yet finer in scale than the coarse partitioning of crete form of object recognition, appears alsoIT into anatomically defined subregions to be organized into anatomically discrete pro-(Felleman & Van Essen, 1991; Seltzer & cessing centers. Individual neurons connectPandya, 1994; Von Bonin & Bailey, 1947). directly with a small fraction of the rest of theThe face-patch system transgresses area neurons in the brain, usually to nearby cells,boundaries, with face patches located in pos- because longer axons delay neural transmis-terior, middle, and anterior portions of IT sion, are energetically expensive, and take up(Felleman & Van Essen, 1991). space. Furthermore, colocalization of neurons Tsao and colleagues (2006) further showed concerned with face processing enablesthat at least two of the macaque face areas, ML enriched local inhibitory interactions, sinceand MF, consist almost entirely of face-selec- inhibitory neurons are always local (Somogyitive cells (Fig. 24.6). Tsao and colleagues used et al., 1998). Wang and colleagues (2000)functional imaging to localize regions in the recorded responses in anterior IT to a set ofmacaque temporal lobe that were selectively complex stimuli before, during, and afteractivated by faces, compared to nonface applying the g-aminobutyric acid (GABA)objects, and then they recorded almost 500 receptor antagonist bicuculline near thesingle units within ML and MF in two mon- recording electrode. In many cases, for bothkeys. They found a remarkable degree of face face-selective and non–face-selective cells,selectivity within these two regions, with 97% blocking local inhibition revealed responses toof visually responsive cells being face selective, previously nonactivating stimuli, which wereon average showing almost 20-fold larger often activating stimuli for neighboring cells.responses to faces than to nonface objects. This suggests that neighboring cells refineThe region where they recorded was quite pos- each other’s response selectivity by mutualterior in the temporal lobe (6 mm anterior to inhibition.

OBJECT RECOGNITION: PHYSIOLOGICAL AND COMPUTATIONAL INSIGHTS 479Figure 24.6 Mapping face and object selectivity in the monkey brain. (A) Five stimulus categoriesincluded faces, four nonface object categories (hands, gadgets, fruits, and bodies), and grid scrambledpatterns. (B) Responses of 182 neurons from the middle face patch of one monkey to 96 images of faces andnonface objects. (C) Average normalized population response to each image. Tsao, D. Y., Freiwald, W. A.,Tootell, R. B. H., & Livingstone, M. S. (2006). A cortical region consisting entirely of face-selective cells.Science, 311, 670–674. Used with permission. While tracer studies demonstrate connec- scale functional anatomy of one discrete aspecttions between specific nodes in TEO and TE of object recognition.(Saleem et al., 1993), the need to sacrifice the To identify the connectivity of individualanimal to process the tissue prevents assessment face patches, Moeller and colleagues (2008)of the functional properties of connected nodes. used electrical microstimulation combinedIn general, to understand functional architec- with simultaneous fMRI. Stimulation ofture, it is necessary to combine connectivity each of four targeted face patches producedmaps with functional topography. The face pro- strong activation specifically within a subsetcessing system of macaque monkeys, consisting of the other face patches (Fig. 24.7).of six patches that can be identified by fMRI and Stimulation outside the face patches pro-targeted for anatomical experiments, provides duced an activation pattern that spared thean ideal preparation for dissecting the large- face patches. These results indicate that the

480 PRIMATE NEUROETHOLOGYFigure 24.7 Combined functional magnetic resonance imaging and microstimulation demonstrates thatthe six macaque face patches are strongly and specifically interconnected. (A) Face patches from onemacaque shown on a right hemisphere flat map. (B) Regions activated by microstimulation at the regionmarked by an Â compared to blank. Face patches indicated by green outlines. The animal fixated on a blankscreen during the entire experiment. Face region and sulcal abbreviations as in Figure 24.5B. Moeller, S.,Freiwald, W. A., & Tsao, D. Y. (2008). Patches with links: a unified system for processing faces in themacaque temporal lobe. Sebastian Moeller, Winrich A. Freiwald, and Doris Y. Tsao (6 June 2008) Science320(5881), 1355. [DOI: 10.1126/science.1157436]. Used with permission.face patches form a strongly and specifically that the neural machinery supporting these non-interconnected network. face object representations has a precise circuitry. The existence of this network creates unprece- The unit element of this circuitry is a column %0.5dented possibilities for gaining a systematic mm wide. Specific subsets of columns within TEOunderstanding of information flow in IT. For and TE are organized into connected networks.the first time we can access anatomically distinct Alongside the machinery for recognizing nonfacecomponents of a unified object processing net- objects, the temporal lobe of both humans andwork in vivo and intercept the messages being macaques contains a system of strongly face-selec-sent between connected cell pairs within different tive regions. In macaques, this system comprises sixpatches. Thus, in addition to characterizing in patches of cortex extending from TEO to anteriordetail the properties of cells within each patch, TE and, in humans, four areas spanning the lengthwe can sequentially characterize the transforma- of the temporal lobe. The macaque face patches aretions that occur across patches—transformations strongly and specifically connected to each other,that somehow lead to our ability to recognize and at least two of the macaque face patches consistthousands of different faces effortlessly even almost entirely of face-selective cells.under incredible variation (i.e., Fig. 24.1).Interim Summary REPRESENTATIONS IN THE INFERIOR TEMPORAL LOBENonface objects are represented by widely distrib-uted and overlapping representations in ventral In this section we discuss the physiology of cellstemporal cortex. Anatomical tracing, optical ima- in IT that support the remarkable capacity toging, and single-unit studies in macaques suggest recognize objects. We present a review of the

OBJECT RECOGNITION: PHYSIOLOGICAL AND COMPUTATIONAL INSIGHTS 481findings derived from visual stimulation using assumption that experimenter-chosen real-parametric visual features and general objects. world objects will span enough of the codingWe will also present results of electrophysiolo- space in sufficient detail that the approach willgical studies of face representations in visual yield meaningful insights.cortex. Finally, we address the issue of holistic A parts-based representation has intuitivefacial processing and its relation to electrophy- appeal. When asked to describe an object (e.g.,siological findings. a flower), people naturally describe its parts (e.g., petals, leaves, stem). The distributed, combinatorial code created by a parts-based representa-Representation of Objects tion permits an extremely rich set of objects to beTanaka and colleagues pioneered the study of represented. A parts-based representation oftuning properties of single cells in IT (Tanaka, words doubtless in fact underlies your ability to1996; Tanaka et al., 1991). Their work was con- read this page: (1) the geometry of letters mayverged on the concept of a ‘‘critical feature,’’ take advantage of the brain’s hardwired ability todefined as the simplest feature that elicits the distinguish contour parts (Changizi et al., 2006),maximum response in a cell. To identify the through their location (‘‘p’’ vs. ‘‘b’’), curvaturecritical feature of an IT cell, they first showed (‘‘v’’ vs. ‘‘u’’), and connectivity (‘‘t’’ vs. ‘‘T’’),many three-dimensional plant and animal and (2) the possibility of forming infinite mean-models to the cell at different views, then made ings from 26 characters illustrates the represen-pictures of these objects and systematically sim- tational power of a parts-based structural code.plified them until they were able to identify the Connor and colleagues have uncovered evi-minimal feature that elicited maximal activation dence for parts-based coding in V4 (Pasupathy &(Kobatake & Tanaka, 1994). For example, the Connor, 1999, 2001, 2002) and PIT (Brincat &critical feature of a cell preferring a tiger might Connor, 2004, 2006; Haxby et al., 2001) throughbe two round black and white gratings arranged single-unit recordings examining tuning to para-like a snowman (Tanaka, 2003). The main con- metrically defined contour features. Pasupathyclusion was that most cells in TE are selective for and Connor (2001) created a large set of closedmoderately complex feature combinations but contours that could be parameterized by curva-not for whole objects. Such cells are already ture and angular position (Fig. 24.8A), and thenpresent in V4 and TEO, but at a lower propor- analyzed the responses of V4 cells to these shapes.tion, mixed with cells selective for simple fea- They found that most cells in V4 were tuned to atures (such as color or orientation). particular curvature at a particular location Subsequent attempts to decipher the code within the cell’s receptive field (e.g., a sharp con-used by the ventral stream have followed two vexity in the lower left). Furthermore, theygeneral lines of attack: (1) a bottom-up showed that by combining these tuning curves,approach, starting with a model of structural they could reconstruct (approximately) anencoding, and testing the model, and (2) a unknown test shape from the pattern of responsesblack box approach, using responses of a in V4 elicited by the shape (Fig. 24.8B,C)random population of cells to various complex (Pasupathy & Connor, 2002). Extending thisobjects to derive general principles of IT coding. work to IT cortex, Connor and colleagues foundThese two approaches represent two different cells tuned for the presence of multiple partsways of conquering the huge parameter space (specifically, tuning could be described by theoccupied by the set of all possible visual inputs. sum of two to four subunits with GaussianThe structural encoding approach assumes that tuning in a six-dimensional curvature, orienta-the brain represents most objects by decom- tion, and position space) (Brincat & Connor,posing them into parts and part relations. The 2004). Figure 24.8D illustrates a cell tuned to ablack box approach, on the other hand, does not combination of concave contour fragmentsmake any concrete assumptions about how oriented toward the lower right and toward theobjects might be coded, but it does make the lower left, and located in the bottom right of the

Figure 24.8 Encoding of contour curvature in V4 and PIT. (A) The white line represents theboundary curvature as a function of angular position of the test shape in the center. (B) Estimatedpopulation response across the curvature Â position domain to a test shape with the veridicalcurvature function superimposed (white line). The response was estimated by weighting each cell’scurvature Â position tuning function by its response to the test shape. (C) Reconstruction of thetest shape from the population response in B. (D) Response of a posterior inferotemporal cell to itspreferred feature: a combination of concave contour fragments oriented toward the lower right andtoward the lower left (highlighted in green). The cell was highly sensitive to the position of thesecontour elements relative to the object. It responded strongly to concavities at the left of objectcenter (bright green), as exemplified by the average response (gray histogram) to the stimuli in thetop two rows. The light gray shading indicates the 500-ms stimulus-presentation period. The cellresponded only weakly to the same concavities at the right (dark green; bottom two rows). Thistuning for object-relative position was consistent across changes in absolute position of the stimuli.Thus, cells in PIT are coding the curvature of contour fragments in object-centered coordinates. A–Cfrom Pasupathy, A., & Connor, C. E. (2002). Population coding of shape in area V4. NatureNeuroscience, 5, 1332–1338. D from Connor, C. E., Brincat, S. L., & Pasupathy, A. (2007).Transformation of shape information in the ventral pathway. Current Opinions in Neurobiology, 17,140–147. Used with permission. 482

OBJECT RECOGNITION: PHYSIOLOGICAL AND COMPUTATIONAL INSIGHTS 483Figure 24.8 (Continued)object. These experiments suggest that object with complementary contours that preserverecognition relies to a significant extent on a geon structure is just as strong as the effect ofparts-based code implemented by tuning curves priming with identical contours, and signifi-of single neurons. One criticism of this work is cantly stronger than the effect of priming withthat the space of real shapes is much vaster than an abstract object category (Biederman &that tested. But any study of IT must employ an Cooper, 1991) (Fig. 24.9C). Thus, primingexperimenter-limited stimulus set; the finding must be attributed to a representation of thethat cells in both V4 and TEO respond to parts parts of the object (and their interrelations)of objects and not to the whole object (albeit in an and not to the activation of the image featuresobject-centered reference frame) must be or abstract categories.explained by any theory of object representation The defining feature of geons is their viewin the ventral stream. invariance. Metric properties of objects (e.g., A structural encoding scheme for repre- aspect ratio), in contrast, are not viewpointsenting three-dimensional objects was proposed invariant. The geon theory predicts thatby Irving Biederman (Biederman, 1995). In the responses to objects differing in geon structure‘‘geon theory’’ of object recognition, a given should be easier to distinguish, across changes inview of an object is represented as an arrange- view, than responses to objects differing only inment of simple, viewpoint-invariant, volumetric metric properties. To test this, Vogels and col-primitives called ‘‘geons.’’ Five examples of leagues (2001) recorded responses of IT cells togeons are shown in Figure 24.9A. According to six variations of an object: the original objectthe geon theory, an object is represented by the composed of a pair of geons, a ‘‘metric prop-geons it contains (out of 24 total), together with erty’’ variant in which one of the geons waspairwise geon relations (e.g., ‘‘above’’; 81 total) metrically varied (e.g., made wider), and a ‘‘non-and geon attributes (e.g., ‘‘horizontally oriented, accidental property’’ variant in which one of thenarrow relative aspect ratio’’; 15 total). These geons was changed (e.g., a cylinder replaced by alimited parameters could represent 10.5 million block). Rotated versions of these three imagesdifferent two-geon objects. were also constructed. The two variants were The geon theory is supported by experiments equated in terms of low-level image statistics.with partial line drawings. The effect of priming Their results provide partial support of the

484 PRIMATE NEUROETHOLOGYgeon theory. A population multidimensional from Logothetis and colleagues, who trainedscaling (MDS) analysis revealed two dimensions, monkeys to match three-dimensional paperclipsone corresponding to rotation, and one to geon across changes in view. Subsequently, theychange (Fig. 24.10D). The population response recorded in anterior IT and found that manyto the metric variant was very close to that of the cells responded maximally to a particular vieworiginal image (i.e., neurons had trouble distin- of a paperclip, with the response declining gra-guishing them), while the response to the geon dually as the object was rotated away from thisvariant was well removed. This supports the idea preferred view. A very small number of cells werethat cells in IT have a special sensitivity to differ- found that responded in a view-invariantences in geon structure over differences in metric manner.structure. However, the large difference between While a large number of objects can bethe population response to each object and its represented by parts-based parametric schemesrotated counterpart shows that the representa- such as curved contours (Pasupathy & Connor,tion of object shape in IT is not view invariant. 2001) or geons (Biederman, 1987), there are The last finding is consistent with an alter- still many objects that do not fit easily intonative view that the primitives for object recog- such parametric shape schemes, yet may benition in IT are view based (Bulthoff et al., 1995). biologically important to distinguish. Thus, aExperimental evidence for this theory comes second approach to studying IT cortex has A B 1 2 Inner Two tangent Y vertices 3 4 Y vertex (Occluding edge tangent Three Three at vertex to parallel outer discontinuous edge) edges arrow vertices Curved edges Two parallel 5 edges CFigure 24.9 Structural encoding by geons. (A) Examples of fives geons. (B) A block and a cylinder, twoexamples of geons, show particular nonaccidental differences that may be used by the brain to recover theirstructure from line drawings. (C) Examples of the four stimulus types used by Biederman and Cooper todemonstrate the existence of a geon-level encoding in a psychophysical priming experiment.(D) Neurophysiological evidence for representation of nonaccidental properties. Top: Six classes ofimages used by Vogels and colleagues to test for a geon representation in macaque IT. Bottom:Multidimensional scaling of inferotemporal responses to these image classes reveals a dimensionrepresenting the difference between a view-dependent change and a viewpoint-invariant change(dimension 1), and a dimension representing rotation (dimension 2). A–C from Biederman, I. (1995).Visual object recognition. In: S. M. Kosslyn & D. N. Osheron (Eds.), Visual cognition (pp. 121–166).Cambridge, MA: MIT Press. D from Vogels, R., Biederman, I., Bar, M., & Lorincz, A. (2001). Inferiortemporal neurons show greater sensitivity to nonaccidental than to metric shape differences. Journal ofCognitive Neuroscience, 13, 444-453. Used with permission.

OBJECT RECOGNITION: PHYSIOLOGICAL AND COMPUTATIONAL INSIGHTS 485 D Original ∆ Metric Prop. ∆ Nonaccidental Prop. Rotated Versions RO R NAP R MP DIMENSION 2 NAP O MP DIMENSION 1Figure 24.9 (Continued)been to stimulate cells with random sets of Kiani and colleagues (2007) measuredcomplex images and ask, by means of various responses of more than 600 neurons inreadout procedures, what types of information monkey IT cortex to over a thousand differentare available in the population of responses images of natural and manmade objects(e.g., how fast can information be read out, during passive fixation. Separation of thehow many neurons are needed to code certain images into clusters based on response simi-types of information, etc.). Obviously, a null larity across the population of cells showedresult does not say much, since the readout that the neuronal population sorted thealgorithm used may simply be inadequate. On images into intuitive categories: Animate andthe other hand, a positive result from such an inanimate objects created the most distantapproach does reveal what kind of information distinguishable clusters in the populationis present. It should be noted that readout code. The global category of animate objectstechniques can also be used to examine struc- was divided into bodies, hands, and faces.tural encoding hypotheses—for example, one Faces were divided into primate and nonpri-could use classifier techniques to ask whether mate faces, and the primate-face group wasinformation about curvature can be read out divided into human and monkey faces.from cells in a position- invariant way. The Bodies of human, birds, and four-limb ani-main difference lies in whether the stimulus mals clustered together, whereas lower ani-space consists of parametric combinations of mals such as fish, reptiles, and insects madeparts or randomly selected real-world objects. another cluster. Importantly, low-level image

486 PRIMATE NEUROETHOLOGYFigure 24.10 Holistic face detection by an inferotemporal cell. Top: Recording site of a face cell. (A—H)Response selectivity. From Kobatake, E., & Tanaka, K. (1994). Neuronal selectivities to complex objectfeatures in the ventral visual pathway of the macaque cerebral cortex. Journal of Neurophysiology, 71,856–867. Used with permission.characteristics could not account for this cate- Hung and colleagues (2005) used a classifier-gorical classification. Therefore, they con- based readout technique to analyze responses ofcluded that monkey IT specifically extracts 256 IT recording sites to 77 images. They foundcomplex features for the purpose of object that the activity of %100 IT neurons over verycategorization. short time intervals (as small as 12.5 ms

OBJECT RECOGNITION: PHYSIOLOGICAL AND COMPUTATIONAL INSIGHTS 487contained sufficient information to read out not just selective for individual featuresobject "identity" at 72% (chance = 1/77) and (Bruce et al., 1981; Desimone et al., 1984;object ‘‘category’’ at 94% (chance = 1/8) accu- Kobatake & Tanaka, 1994; Leonard et al., 1985;racy. Importantly, this information generalized Oram & Perrett, 1992; Perrett et al., 1982, 1984;over a range of object positions and scales, with Scalaidhe et al., 1999; Tsao et al., 2006). Figureless than 10% reduction in performance. These 24.10 shows nonlinear combinatorial responseresults show that invariant information about properties of a face-selective cell recorded in ITobject category and identity is available in small (most likely in the face patch ML, judging by thepopulations of neurons even during an early recording location) by Kobatake and Tanakaphase of the response. Can object recognition (1994). Out of a large number of three-dimen-be completely solved by means of units with sional objects, this cell responded best to the facethe invariance and tuning properties of ran- of a toy monkey (A), and by testing variousdomly sampled IT cells, as already described in simplified two-dimensional paper stimuli, theystudies such as that of Hung and colleagues? Or determined that the cell would also respond to aare we still waiting to observe a new type of configuration of two black dots over a horizontalinvariance property? Rigorous tests comparing line within a disk (B), but not in the absence ofphysiology to behavior have not been performed either the spots or the line (C and D) or theto test this important question. circular outline (E). The contrast between the inside and the outside of the circle was notRepresentation of Faces critical (G), but the spots and the bar had to be darker than the disk (H). In other words, the cellOne of the biggest differences between face per- only responded when the stimulus looked like aception and general object recognition is that face, no matter how simplified. How face cellsarbitrary objects map to different aspects of con- detect a facial Gestalt is still unknown, and willtour geometry. Faces, on the other hand, share a likely require identifying the inputs to the facecommon template, consisting of eyes, nose, patches.mouth, and face outline (though numerous var- Once a face has been detected, it needs to beiations are possible; e.g., depending on the view identified. Recordings in the face patches MLangle, only one eye may be visible). The vastly and MF have begun to shed light on the neuralreduced template space makes understanding mechanism for distinguishing between differentthe mechanism of face perception more tractable faces. Freiwald and colleagues (2009) used car-compared to understanding the detection and toon faces to study the neural basis of face mea-recognition of arbitrary objects. surement. Dense parametric mapping was used That face cells are truly detecting faces, and to measure responses of cells in the middle facenot some more abstract basis set in which all patch to a cartoon stimulus in which all facepossible shapes are represented by different parameters were independently varied. Cellscells, with some cells tuned to particular para- were found to be tuned to the geometry ofmeters that happen to fit the faces better than facial features, with different cells tuned to dif-any of the other objects tested, was demon- ferent feature subsets. Tuning was strikinglystrated by Foldiak and colleagues (2004). They ramp shaped, with a one-to-one mapping ofpresented 600 to 1,200 stimuli randomly chosen feature magnitude to firing rate (Fig. 24.11).from several image archives to cells recorded These extreme values extended to or even trans-from both the upper and lower bank of the STS gressed the limits of realistic face space. Forand found that the distribution of tuning to example, intereye distances ranged from almostthese images showed bimodality (i.e., cells were cyclopean to abutting the edges of the face, andeither predominantly face selective or not face the most extreme face aspect ratios were outsideselective). those of any known primate. Monotonic tuning Experiments with cartoon faces show that in allows for simple readout (Guigon, 2003) andgeneral face cells require an intact face and are may be a general principle for high-level coding

488 Figure 24.11 Tuning of face cells to a cartoon face space. (A) Six example dimensions (hair width, eyebrow slant, eyebrow height, intereye distance, iris size, mouth top shape) of the 19-dimensional cartoon space. Each row shows example values for one parameter, with all other parameters fixed at their mean. (B) Tuning curves of three example cells to each of the 19 feature dimensions. In gray maximal, minimal, and mean values from shift predictor are shown. Stars mark significant modulation. From Freiwald, W. A., Tsao, D. Y., & Livingstone, M. S. (2008). A face feature space in the macaque temporal lobe. Nature Neuroscience 12, 1187–1196 (2009). Published online: 9 August 2009 [DOI: 10.1038/nn.2363]. Used with permission.

OBJECT RECOGNITION: PHYSIOLOGICAL AND COMPUTATIONAL INSIGHTS 489of visual shapes (Kayaert et al., 2005). It may also face parts when the parts are presented in theaid in emphasizing what makes an individual context of a whole face than in isolation. In theface unique (i.e., separates it from the average composite effect, subjects are slower to identifyface), and could be the neural basis for the power half of a chimeric face aligned with an inconsis-of caricatures. The breadth of tuning under- tent other half-face than if the two half-faces arescores the fact that cells in the middle face misaligned (Young et al., 1987). As with thepatches, ML and MF, encode axes and not indi- part-whole effect, the composite effect indicatesvidual faces. that even when subjects attempt to process only Recordings made in the highest stage of the part of a face, they suffer interference from theform processing hierarchy, the medial temporal other parts, suggesting an inability to accesslobe, reveal the existence of cells that respond to parts of the face and mandatory processing ofspecific familiar individuals in a representation- the whole face.invariant manner (Quiroga et al., 2005), as Holistic face processing could be explainedexpected of a ‘‘grandmother cell.’’ For example, by the existence of an obligatory detection stagesome cells each responded to multiple pictures that uses a coarse upright template to detectof a well-known individual as well as to a letter whole faces (Tsao & Livingstone, 2008). Anstring of their name, but were unresponsive to all aligned chimera would be obligatorily detectedother images. Such individual-specific cells have as a whole face and therefore processed as anot been found in the lateral inferior temporal unit by subsequent measurement and classifica-lobe, where most face cells in monkeys have been tion stages. The key evidence favoring this earlyrecorded, although, as a population, cells in the detection gating hypothesis comes from theanterior inferior temporal gyrus of the macaque finding of six face-selective areas in the macaque(in the general vicinity of AM) can support view- (Tsao et al., 2006, 2008) and the finding that theinvariant identification (Eifuku et al., 2004). middle face patches, located early in this hierarchy, already consist entirely of face-selectiveHolistic Processing cells (Tsao et al., 2006). According to this viewpoint, holistic psychological markers do notSince the thesis of this chapter is that under- indicate that faces are processed by a mysteriousstanding face areas will help us understand and unique mechanism; they simply indicateobject recognition, it is appropriate to discuss that faces are processed by a system that isthe claim that face processing is supported by a employed only for stimuli that are first detectedunique set of mechanisms that do not generalize as faces, but this system may be parts based, andto nonface objects. Face processing obviously is is also likely optimized for analyzing face parts.unique in being housed in a set of specialized An as yet unexplored question is whether othercortical regions, but this doesn’t imply that the object categories are first detected and then ana-underlying mechanisms are unique. lyzed by specialized modules. Other object cate- Specifically, face processing is claimed to be gories that may fit this scheme include bodiesdistinct from nonface object processing in that it and words.is ‘‘holistic’’; that is, faces are represented asnondecomposed wholes, rather than as a com- Interim Summarybination of independently represented compo-nent parts (eyes, nose, mouth) and the relations Perhaps the central theme that has emergedbetween them (Farah et al., 1998). Evidence for from IT research over the past two decades isholistic processing of faces comes from a that apart from biologically special categoriesnumber of behavioral paradigms, of which the (most conspicuously faces), objects are codedtwo most cited are the part-whole effect in IT in terms of their parts and not wholes.(Tanaka & Farah, 1993) and the composite Early experiments studying the responses of ITeffect (Young et al., 1987). In the part-whole cells to decomposed complex objects found thateffect, subjects are better able to identify two most cells are selective for moderately complex

490 PRIMATE NEUROETHOLOGYfeatures but not for whole objects. Consistent recognition. We then turn to a class of modelswith a parts-based code, cells in V4 and PIT from machine vision that produces state-of-theshow tuning to the curvature and location of art recognition abilities. We also examine a dif-contour elements in object-centered coordi- ferent class of models that uses feedback or con-nates. Biederman and colleagues have argued textual information during inference and objectfor explicit structural encoding of view-invariant recognition. Finally, we relate these works to ourvolumetric primitives, but evidence suggests that view of face processing in human and non-the responses of most cells in TEO and TE are human primates.view dependent. Although IT cells do not gen-erally appear to be detectors for complex objects, Theories of Biological Object Recognitionthere are consistently observed populations ofcells selectively responsive to animate objects One of the central problems of object recogni-such as faces, bodies, and hands, suggesting tion is that effective systems must deal with thethat animate objects may be treated differently variations present in the natural world: variationfrom other types of complex patterns. A major in position, size, rotation, illumination, anddistinction between face processing and nonface even nonrigid motion (i.e., movement ofobject processing is that early on (by the middle limbs). This problem materializes when a visualface patches at the latest), the form of a face has system must generalize from its previous experi-been obligatorily detected as a whole. This ence with an object under specific conditions toobservation may explain holistic psychophysical conditions in which the object has never beenphenomena associated with face perception. viewed before. This is a computationally difficultCells in the middle face patch encode facial problem. It is made even more difficult by theidentity through ramp-shaped tuning to fact that the novel object view may differ fromsubsets of facial features, consistent with a previously experienced views in several of theseparts-based code of face identity. At the highest variations at the same time. In order to deal withstages of form processing, in the medial these variations, researchers have sought repre-temporal lobe, cells are found that respond to sentations that will be invariant, or unchanging,specific familiar individuals in a representation- under transformations that produce image var-invariant manner. iation. With respect to this problem, most models of biological object recognition take one of two approaches: models that directlyCOMPUTATIONAL APPROACHES compute invariant responses and those that cor-TO OBJECT RECOGNITION rect for transformations to produce invariantTheories of visual object recognition have responses. See Wiskott (2004) for an additionalreceived attention from both the neuroscience review and analysis of these two approaches.community and the machine vision community. Models that directly compute invariantBoth approaches have contributed to our under- responses take many approaches. However,standing of how visual object recognition might there are strong commonalities among a groupwork in biological systems. We will summarize of models that seek to explain invariant objectkey work in both of these areas. While such a recognition in biological systems. This class ofsurvey does not do justice to the individual con- models can be traced back to the hypothesistributions of each theory or system, it does allow originally proposed by Hubel and Weisel thatus to see the commonalities that might be useful complex cell responses in primary visual cortexfor producing a consensus view of computa- are formed by combining the responses of spa-tional object recognition and for applying this tially shifted simple cell responses (Hubel &interpretation to processing in the primate Wiesel, 1962, 1965). One of the earliest compu-brain, especially in relation to face processing. tational instantiations of this framework was theIn this section we begin by providing a Neocognitron model of Fukushima (Fukushimacursory survey of models of biological object et al., 1988). The Neocognitron is a hierarchical

OBJECT RECOGNITION: PHYSIOLOGICAL AND COMPUTATIONAL INSIGHTS 491model consisting of layers that alternately per- theory of invariant recognition. In recent work,form feature extraction (similar to simple cells) Tenenbaum and Freeman (2000) describe aand layers that build invariance to position or bilinear model that separates the ‘‘content’’ andsmall deformations (similar to complex cells). ‘‘style’’ in various data types. These modelsThe weights in each layer determine the type of extend traditional linear models and are referredcomputation performed. The hierarchy alter- to as bilinear because they are linear when onenately builds up selectivity and invariance until set of the variables is held fixed. This modelat the top layer the responses are selective to allows each set of variables to explicitly representcomplex shapes and invariant to large changes different types of information. For example, inin size and position. Recent theoretical work and text, the character identity, or content, is sepa-simulations have extended this approach in rated from the font, or style. In relation to theseveral directions: VisNet (Wallis & Rolls, dynamic routing theory, the style variables can1997), SeeMore (Mel, 1997), and HMAX be considered as dynamically routing the con-(Riesenhuber & Poggio, 1999). Of note are the tent variables to produce an image with specificextensions and elaboration of the HMAX model, style and content. Therefore, in the context ofwhich have demonstrated correspondence to invariant object recognition, the content couldhuman level object recognition performance be ascribed to the identity of the object and theduring brief presentations (Serre et al., 2007b), style to the visual variations or transformations.explanations of selectivity and invariance in In one promising example, Tenenbaum andinferotemporal cortex (Hung et al., 2005) and Freeman use a bilinear model on a dataset ofvisual area V4 (Cadieu et al., 2007), and models face images containing a number of individualsof biophysical computation (Kouh & Poggio). viewed at a range of poses. The bilinear model In approaches that deal with transformations was able to learn from this dataset to separate theexplicitly, the model tries to correct for, or undo, information into a set of variables indicating thethe transformations so that the image can be identity of a person and a set of variables indi-matched against a stored canonical representa- cating the pose of the person.tion. In the model of Olshausen and colleagues(1993), the visual input is dynamically routed by Computer Vision and Object Recognitioncontrol neurons through the visual hierarchy torescale and shift the image. At the top of the While the models we have addressed so far havehierarchy, the shifted and rescaled image can been directed at explaining biological objectthen be compared to stored templates of cano- recognition or perception, a number of techni-nical object views, thus producing invariant ques from computer vision have interestingrecognition. The control neurons, which expli- implications for biological object recognitioncitly model the transformation, engage in an and face processing in the brain. Importantly,active process to selectively turn on and off con- several computer vision models have demon-nections between layers of the visual hierarchy. strated impressive performance on nontrivialLater work by Arathorn (2005) suggests a com- object recognition problems and have demon-putationally tractable method for determining strated performance comparable to humans forthe appropriate transformations for a given specific tasks. For example, in an impressiveinput image. It is interesting that theories that demonstration, Sivic and Zisserman (2003)deal explicitly with transformations often use developed a system that is capable of searchingfeedback: Memory representations guide the through a Hollywood movie for a specific object,search for the correct transformation, while the- aptly named ‘‘Video Google.’’ Related to theories that directly compute invariant representa- HMAX model we discussed previously, Serretions are constructed in a primarily feedforward and coauthors (2007a) demonstrated that anetwork. feedforward computer vision algorithm per- Bilinear models are a class of mathematical formed at levels comparable to human subjectsmodels closely related to the dynamic routing during a rapid presentation of a visual stimulus

492 PRIMATE NEUROETHOLOGYin which they attempted to determine if an disambiguate low-level information. In theiranimal was present in the image. Furthermore, system, license plates are modeled as a hierarchythe computer vision algorithm and the human of parts: whole license plates, groups of num-subjects exhibited similar patterns of error on bers, individual numbers, and number parts.individual images and image categories, such as When presented with an image, the system pro-far versus close views of animals. duces an interpretation of the visual scene by Many of the most effective computer vision using both bottom-up cues and top-downalgorithms can be described as consisting of two hypotheses from prior knowledge of whatstages: The first stage produces a set of local license plates look like. The authors show thatfeatures, while the second stage performs classi- using their contextual hierarchy greatlyfication or matching to determine the presence improves recognition accuracy in a real-worldof an object or the identity of an exemplar (Berg dataset. Another example of the efficacy of con-et al., 2005; Lowe, 2004; Ranzato et al., 2007; textual information is presented by Torralba andSerre et al., 2007b). In the work of Serre and colleagues (2003). In their system, context iscolleagues (2007b), a feature hierarchy produces used to prime the location of a visual object.a dictionary of visual features that are invariant For example, in an indoor office scene there isto object variation. Standard classification tech- a much higher probability of seeing a computerniques are used on the outputs of the hierarchy monitor than there is in an outdoor forest scene.for a variety of visual recognition tasks, such asidentification and classification. Interestingly, Insights for Biological Face Recognitionthis model is inspired by a theory of biologicalobject recognition (Serre et al., 2005) and falls These results can be used to interpret findingsinto the class of biological object recognition related to face processing in primates, and allowmodels that seek to directly compute invariant us to develop mathematical models that makeresponses. Additional work of Ranzato and col- explicit the architecture of biological face pro-leagues (2007) takes a similar approach and cessing. There are three main connections wedevelops an unsupervised learning algorithm to want to point out. First, a rapid feedforwardspecify the selectivity of the intermediate layers. computation that produces invariant responsesLowe’s influential SIFT algorithm (2004) also may be central to the detection of faces. Second,produces a set of invariant features, but instead complicated facial discriminations and infer-of using a classifier to detect or identify object, ences may be mediated by a more complicatedthe SIFT algorithm determines a geometric cor- series of interconnected processing stages thatrespondence between the features. This increases explicitly model the visual transformations.the accuracy of the algorithm for object recogni- Third, the representations in these later stagestion because it ensures a consistent geometric may represent the independent aspects of facialinterpretation of the features (e.g., the eye fea- appearance and pose.tures, nose features, and mouth features must be Because it is advantageous for facial detectionin the appropriate geometric relationship). to be computed quickly, approaches that directlyRelated work by Berg and colleagues (2005) compute invariant responses may be suitable foruses a different set of features, but also tries to face detection. While direct computation ofestimate the geometrical transformation invariant responses often suffers from the com-between a novel object and stored object binatorial explosion of possible inputs (andrepresentations. therefore requires more processing to overcome Another interesting, and relevant, class of the combinations of variations for a specificmodels uses feedback, or context, to aid in the object), face detection is unique because of itsinterpretation of an image and the recognition ecologically critical role, and because faces con-of objects within a scene. Jin and Geman (2006) stitute a relatively constrained set of visualdescribe a hierarchy of parts for a license inputs compared to all visual objects. The earlyplate detection system that uses context to stages in the ventral stream that exclusively

OBJECT RECOGNITION: PHYSIOLOGICAL AND COMPUTATIONAL INSIGHTS 493respond to faces may serve as the culmination of overcome. Therefore, it seems sensible thatthis rapid process of directly computing invar- explicitly representing these transformationsiant responses to faces. would be advantageous. Could the different We may also gain some insights about biolo- nodes of the interconnected system of facegical face processing from bilinear models, patches be representing different, largely inde-which seek to explicitly and separately represent pendent aspects of facial form and variation?different aspects of the visual input. In the Such a representation would make specificexample examined by Tennenbaum and information independently available for com-Freeman, the pose of a face is represented inde- plex social inferences of ecological value.pendently of the facial identity (Fig. 24.12). In Interestingly, the two main theoreticalface processing it is often the case that the rele- approaches to biological object recognition,vant information is not only the presence or the directly computing invariant responses andidentity of a face but also what it is doing. For explicitly accounting for transformations, mayexample, in complex social interactions it is both be employed for face recognition and pro-important to infer intentions from subtle cues cessing: Direct computation of invariantabout head pose and gaze direction. In these responses may mediate face detection, whilecases the transformations that facial forms computing transformations may mediateundergo are the goal of representation and not further inferences about facial characteristics.just variations in the input that must be The evidence for holistic face processing seemsFigure 24.12 A bilinear model from Tenenbaum and Freeman. In this model, face images are representedby two sets of variables, one encoding pose, ‘‘pose coefficients,’’ and the other encoding identity, ‘‘personcoefficients.’’ The basis images (top left) are weighted by the joint activity of the person coefficients (topright) and the pose coefficients (bottom left) to generate reconstructed images of faces (bottom right). FromTenenbaum, J. B., & Freeman, W. T. (2000). Separating style and content with bilinear models. NeuralComputation, 12, 1247–1283. Used with permission.

494 PRIMATE NEUROETHOLOGY Face-Speciﬁc processing Pose Expre- Identity ssion Face Gate detection Visual representation Visual inputFigure 24.13 A model of face processing. In this model of face recognition a fast, feedforwarddetection stage provides feedback to the face model. The face model, which is only activated uponface detection, computes a variety of parameters specific to faces, such as pose, identity, andexpression (among possibly others, such as gaze direction). The face detection stage may bias theface model to certain regions of the image, and certain sizes of faces from the information it infersfrom the input. Such information aids the face model in inferring more specific face parameters suchas identity or pose.to indicate that face detection gates this further face-specific processing stage, or face model,processing. produces a rich description of the face. The We can make the connections between com- face model may be similar to a bilinear modelputational theories and biological evidence more and mimic the dynamic routing theory of invar-precise by describing a simple model of face iant object recognition.processing in primates. Presented inFigure 24.13, the system incorporates theseinsights. First, a fast, feedforward processing SUMMARYstage computes visual features from the visual Hubel and Wiesel showed that primary visualinput, and faces are detected from these features cortex represents images in a space of localized,in a face detection area. Second, the detection of oriented edges. This was stunning because ita face in the face detection module guides and showed that the brain performs a simple math-gates further face-specific processing. Third, a ematical transformation that condenses the

OBJECT RECOGNITION: PHYSIOLOGICAL AND COMPUTATIONAL INSIGHTS 495information present in the visual world. cognition (pp. 121–166). Cambridge, MA: MITPrimates possess an entire temporal lobe to Press.further condense the visual form of objects. Biederman, I., & Cooper, E. E. (1991). PrimingThe thesis of this chapter is that the system of contour-deleted images: Evidence forface patches in the macaque brain may be a intermediate representations in visual object recognition. Cognitive Psychology, 23, 393–419.‘‘turtle’s underbelly’’ (Medawar, 1981)—if we Boussaoud, D., Desimone, R., & Ungerleider, L. G.can understand it, we will be able to pry open (1991). Visual topography of area TEO in thethe general problem of invariant pattern macaque. Journal of Comparative Neurology,recognition. 306, 554–575. What mathematical transformations are Boynton, G. M. (2005). Imaging orientationbeing performed in the face patches? An selectivity: Decoding conscious perception inunprecedented opportunity now exists to V1. Nature Neuroscience, 8, 541–542.understand the successive stages of face pro- Brincat, S. L., & Connor, C. E. (2004). Underlyingcessing at a mechanistic level. The face-patch principles of visual shape selectivity in posteriorsystem offers a set of dedicated, connected, yet inferotemporal cortex. Nature Neuroscience, 7,anatomically distinct components, inviting us 880–886. Brincat, S. L., & Connor, C. E. (2006). Dynamicto analyze for the first time the messages being shape synthesis in posterior inferotemporalsent within a hierarchical system for high-level cortex. Neuron, 49, 17–24.object recognition. If we can figure out how Bruce, C., Desimone, R., & Gross, C. G. (1981).the brain recognizes faces, then we will have Visual properties of neurons in a polysensorygone a long way toward understanding how area in superior temporal sulcus of thethe brain represents a complex object made of macaque. Journal of Neurophysiology, 46,multiple parts. 369–384. Bulthoff, H. H., Edelman, S. Y., & Tarr, M. J. (1995). How are three-dimensional objectsACKNOWLEDGMENTS represented in the brain? Cerebral Cortex, 5, 247–260.We are grateful to Winrich Freiwald for discus- Cadieu, C., Kouh, M., Pasupathy, A., Connor, C. E.,sions and comments on the manuscript. Riesenhuber, M., & Poggio, T. (2007). A model of V4 shape selectivity and invariance. Journal of Neurophysiology, 98, 1733.REFERENCES Changizi, M. A., Zhang, Q., Ye, H., & Shimojo, S.Allison, T., McCarthy, G., Nobre, A., Puce, A., & (2006). The structures of letters and symbols Belger, A. (1994). Human extrastriate visual throughout human history are selected to cortex and the perception of faces, words, match those found in objects in natural scenes. numbers, and colors. Cerebral Cortex, 4, American Naturalist, 167, E117–139. 544–554. Cheng, K., Saleem, K. S., & Tanaka, K. (1997).Arathorn, D. W. (2005). A cortically-plausible Organization of corticostriatal and inverse problem solving method applied to corticoamygdalar projections arising from the recognizing static and kinematic 3D objects. anterior inferotemporal area TE of the macaque In: Advances in neural information processing monkey: A Phaseolus vulgaris leucoagglutinin systems. Cambridge, MA: MIT Press. study. Journal of Neuroscience, 17, 7902–7925.Berg, A., Berg, T., & Malik, J. (2005). Shape Connor, C. E., Brincat, S. L., & Pasupathy, A. matching and object recognition using low (2007). Transformation of shape information distortion correspondence. Proceedings of in the ventral pathway. Current Opinions in CVPR, 1, 26–33. Neurobiology, 17, 140–147.Biederman, I. (1987). Recognition-by-components: Dean, P. (1976). Effects of inferotemporal lesions A theory of human image understanding. on the behavior of monkeys. Psychological Psychological Review, 94, 115–147. Bulletin, 83, 41–71.Biederman, I. (1995). Visual object recognition. In: Desimone, R., Albright, T. D., Gross, C. G., & S. M. Kosslyn & D. N. Osheron (Eds.), Visual Bruce, C. (1984). Stimulus-selective properties

496 PRIMATE NEUROETHOLOGY of inferior temporal neurons in the macaque. Haxby, J. V., Gobbini, M. I., Furey, M. L., Ishai, A., Journal of Neuroscience, 4, 2051–2062. Schouten, J. L., & Pietrini, P. (2001).Diamond, R., & Carey, S. (1986). Why faces are and Distributed and overlapping representations of are not special: An effect of expertise. Journal of faces and objects in ventral temporal cortex. Experimental Psychology: General, 115, 107–117. Science, 293, 2425–2430.Eifuku, S., De Souza, W. C., Tamura, R., Nishijo, Haxby, J. V., Grady, C. L., Horwitz, B., Ungerleider, H., & Ono, T. (2004). Neuronal correlates of L. G., Mishkin, M., Carson, R. E., et al. (1991). face identification in the monkey anterior Dissociation of object and spatial visual temporal cortical areas. Journal of processing pathways in human extrastriate Neurophysiology, 91, 358–371. cortex. Proceedings of the National Academy ofErickson, C. A., & Desimone, R. (1999). Responses Sciences, 88, 1621–1625. of macaque perirhinal neurons during and after Hubel, D., & Wiesel, T. (1962). Receptive fields, visual stimulus association learning. Journal of binocular interaction and functional Neuroscience, 19, 10404–10416. architecture in the cat’s visual cortex. JournalErickson, C. A., Jagadeesh, B., & Desimone, R. (2000). of Physiology, 160, 106. Clustering of perirhinal neurons with similar Hubel, D., & Wiesel, T. N. (1965). Receptive fields properties following visual experience in adult and functional architecture in two nonstriate monkeys. Nature Neuroscience, 3, 1143–1148. visual areas of the cat. Journal ofFarah, M. J., Wilson, K. D., Drain, M., & Tanaka, J. Neurophysiology, 28, 229–289. N. (1998). What is ‘‘special’’ about face Hung, C. P., Kreiman, G., Poggio, T., & DiCarlo, J. perception? Psychological Review, 105, 482–498. J. (2005). Fast readout of object identity fromFelleman, D. J., & Van Essen, D. C. (1991). macaque inferior temporal cortex. Science, 310, Distributed hierarchical processing in the 863–866. primate cerebral cortex. Cerebral Cortex, 1, 1–47. Ito, M., Tamura, H., Fujita, I., & Tanaka, K. (1995).Foldiak, P., Xiao, D., Keysers, C., Edwards, R., & Size and position invariance of neuronal Perrett, D. I. (2004). Rapid serial visual responses in monkey inferotemporal cortex. presentation for the determination of neural Journal of Neurophysiology, 73, 218–226. selectivity in area STSa. Progress in Brain Jin, Y., & Geman, S. (2006). Context and hierarchy Research, 144, 107–116. in a probabilistic image model. IEEE ConferenceFreiwald, W. A., Tsao, D. Y., & Livingstone, M. S. on Computer Vision and Pattern Recognition, 2. (2009). A face feature space in the macaque Kamitani, Y., & Tong, F. (2005). Decoding the temporal lobe. Nature Neuroscience, Nature visual and subjective contents of the human Neuroscience 12, 1187–1196 (2009). Published brain. Nature Neuroscience, 8, 679–685. online: 9 August 2009 [DOI: 10.1038/nn.2363]. Kanwisher, N., McDermott, J., & Chun, M. (1997).Fujita, I., & Fujita, T. (1996). Intrinsic connections The fusiform face area: A module in human in the macaque inferior temporal cortex. Journal extrastriate cortex specialized for face perception. of Comparative Neurology, 368, 467–486. Journal of Neuroscience, 17, 4302–4311.Fujita, I., Tanaka, K., Ito, M., & Cheng, K. (1992). Kayaert, G., Biederman, I., Op de Beeck, H. P., & Columns for visual features of objects in monkey Vogels, R. (2005). Tuning for shape dimensions inferotemporal cortex. Nature, 360, 343–346. in macaque inferior temporal cortex. EuropeanFukushima, K., Miyake, S., & Ito, T. (1988). Journal of Neuroscience, 22, 212–224. Neocognitron: A neural network model for a Kiani, R., Esteky, H., Mirpour, K., & Tanaka, K. mechanism of visual pattern recognition. In: (2007). Object category structure in response Computer Society Press technology series neural patterns of neuronal population in monkey networks (pp. 136–144). Los Alamitos, CA: IEEE inferior temporal cortex. Journal of Computer Society Press. Neurophysiology, 97, 4296–4309.Gross, C. G. (1973). Visual functions of Kobatake, E., & Tanaka, K. (1994). Neuronal inferotemporal cortex. In: R. Jung (Ed.), selectivities to complex object features in the Handbook of sensory physiology (p. 451–482). ventral visual pathway of the macaque cerebral Berlin: Springer. cortex. Journal of Neurophysiology, 71, 856–867.Guigon, E. (2003). Computing with populations of Kouh, M., & Poggio, T. A canonical neural circuit monotonically tuned neurons. Neural for cortical nonlinear operations. Neural Computation, 15, 2115–2127. Computation, 20, 1427–1451.

OBJECT RECOGNITION: PHYSIOLOGICAL AND COMPUTATIONAL INSIGHTS 497Leonard, C. M., Rolls, E. T., Wilson, F. A., & Baylis, Mountcastle, V. B. (1997). The columnar G. C. (1985). Neurons in the amygdala of the organization of the neocortex. Brain, 120 monkey with responses selective for faces. (Pt 4), 701–722. Behavioral Brain Research, 15, 159–176. Norman, K. A., Polyn, S. M., Detre, G. J., & Haxby,Lowe, D. G. (2004). Distinctive image features J. V. (2006). Beyond mind-reading: Multi-voxel from scale-invariant keypoints. International pattern analysis of fMRI data. Trends in Journal of Computer Vision, 60, 91–110. Cognitive Science, 10, 424–430.Maunsell, J. H., & Van Essen, D. C. (1987). Olshausen, B., Anderson, C., & Van Essen, D. Topographic organization of the middle (1993). A neurobiological model of visual temporal visual area in the macaque monkey: attention and invariant pattern recognition Representational biases and the relationship to based on dynamic routing of information. callosal connections and myeloarchitectonic Journal of Neuroscience, 13, 4700. boundaries. Journal of Comparative Neurology, Oram, M. W., & Perrett, D. I. (1992). Time course 266, 535–555. of neural responses discriminating differentMcCarthy, G., Puce, A., Belger, A., & Allison, T. views of the face and head. Journal of (1999). Electrophysiological studies of human Neurophysiology, 68, 70-84. face perception. II: Response properties of face- Pasupathy, A., & Connor, C. E. (1999). Responses specific potentials generated in occipitotemporal to contour features in macaque area V4. Journal cortex. Cerebral Cortex, 9, 431–444. of Neurophysiology, 82, 2490–2502.McCarthy, G., Puce, A., Gore, J. C., & Allison, T. Pasupathy, A., & Connor, C. E. (2001). Shape (1997). Face-specific processing in the human representation in area V4: Position-specific fusiform gyrus. Journal of Cognitive tuning for boundary conformation. Journal of Neuroscience, 9, 605–610. Neurophysiology, 86, 2505–2519.Medawar, P. B. (1981). Advice to a young scientist. Pasupathy, A., & Connor, C. E. (2002). Population Perseus Publishing. Perseus Books Group coding of shape in area V4. Nature Neuroscience, (Headquarters), 387 Park Avenue South, 12th 5, 1332–1338. Floor, New York, NY 10016. (212) 340–8100. Perrett, D. I., Rolls, E. T., & Caan, W. (1982). VisualMel, B. W. (1997). SEEMORE: Combining color, neurones responsive to faces in the monkey shape, and texture histogramming in a neurally temporal cortex. Experimental Brain Research, inspired approach to visual object recognition. 47, 329–342. Neural Computation, 9, 777–804. Perrett, D. I., Smith, P. A., Potter, D. D., Mistlin, A. J.,Meunier, M., Bachevalier, J., Mishkin, M., & Head, A. S., Milner, A. D., et al. (1984). Neurones Murray, E. A. (1993). Effects on visual responsive to faces in the temporal cortex: Studies recognition of combined and separate of functional organization, sensitivity to identity ablations of the entorhinal and perirhinal and relation to perception. Human Neurobiology, cortex in rhesus monkeys. Journal of 3, 197–208. Neuroscience, 13, 5418–5432. Pinsk, M. A., DeSimone, K., Moore, T., Gross, C. G.,Miyashita, Y., Okuno, H., Tokuyama, W., Ihara, T., & Kastner, S. (2005). Representations of faces & Nakajima, K. (1996). Feedback signal from and body parts in macaque temporal cortex: A medial temporal lobe mediates visual functional MRI study. Proceedings of the National associative mnemonic codes of inferotemporal Academy of Sciences, 102, 6996–7001. neurons. Brain Research: Cognitive Brain Previc, F.(1990). Functionalspecializationinthelower Research, 5, 81–86. and upper visual fields in humans: Its ecologicalMoeller, S., Freiwald, W. A., & Tsao, D. Y. (2008). origins and neurophysiological implications. Patches with links: A unified system for Behavior and Brain Sciences, 13, 519–575. processing faces in the macaque temporal lobe. Puce, A., Allison, T., Asgari, M., Gore, J. C., & Science, 320, 1355–1359. McCarthy, G. (1996). Differential sensitivity ofMoscovitch, M., Winocur, G., & Behrmann, M. human visual cortex to faces, letterstrings, and (1997). What is special about face recognition? textures: A functional magnetic resonance Nineteen experiments on a person with visual imaging study. Journal of Neuroscience, 16, object agnosia and dyslexia but normal face 5205–5215. recognition. Journal of Cognitive Neuroscience, Puce, A., Allison, T., & McCarthy, G. (1999). 9, 555–604. Electrophysiological studies of human face

498 PRIMATE NEUROETHOLOGY perception. III: Effects of top-down processing transactions on pattern analysis and machine on face-specific potentials. Cerebral Cortex, 9, intelligence, 29, (pp. 411–426). IEEE Computer 445–458. Society.Quiroga, R. Q., Reddy, L., Kreiman, G., Koch, C., & Serre, T., Wolf, L., & Poggio, T. (2005). Object Fried, I. (2005). Invariant visual representation recognition with features inspired by visual by single neurons in the human brain. Nature, cortex. Proceedings of IEEE Computer Society 435, 1102–1107. Conference on Computer Vision and PatternRanzato, M., Huang, F., Boureau, Y., & LeCun, Recognition (CVPR). Y. (2007). Unsupervised learning of invariant Sivic, J., & Zisserman, A. (2003). Video Google: feature hierarchies with applications to A text retrieval approach to object matching in object recognition. Proceedings of the videos. Computer vision, 2003. Proceedings of the Computer Vision and Pattern Recognition ninth IEEE international conference Conference (CVPR’07), 1–8. (pp. 1470–1477).Riesenhuber, M., & Poggio, T. (1999). Hierarchical Somogyi, P., Tamas, G., Lujan, R., & Buhl, E. H. models of object recognition in cortex. Nature (1998). Salient features of synaptic organisation Neuroscience, 2, 1019–1025. in the cerebral cortex. Brain Research: BrainSaleem, K. S., Kondo, H., & Price, J. L. (2008). Research Reviews, 26, 113–135. Complementary circuits connecting the orbital Suzuki, W., Saleem, K. S., & Tanaka, K. (2000). and medial prefrontal networks with the Divergent backward projections from the temporal, insular, and opercular cortex in the anterior part of the inferotemporal cortex macaque monkey. Journal of Comparative (area TE) in the macaque. Journal of Neurology, 506, 659–693. Comparative Neurology, 422, 206–228.Saleem, K. S., & Tanaka, K. (1996). Divergent Tanaka, K. (1996). Inferotemporal cortex and projections from the anterior inferotemporal object vision. Annual Review of Neuroscience, area TE to the perirhinal and entorhinal 19, 109–139. cortices in the macaque monkey. Journal of Tanaka, K. (2003). Columns for complex visual Neuroscience, 16, 4757–4775. object features in the inferotemporal cortex:Saleem, K. S., Tanaka, K., & Rockland, K. S. (1993). Clustering of cells with similar but slightly Specific and columnar projection from area different stimulus selectivities. Cerebral Cortex, TEO to TE in the macaque inferotemporal 13, 90–99. cortex. Cerebral Cortex, 3, 454–464. Tanaka, J. W., & Farah, M. J. (1993). Parts andScalaidhe, S. P., Wilson, F. A., & Goldman-Rakic, wholes in face recognition. Quarterly Journal of P. S. (1999). Face-selective neurons during Experimental Psychology A, 46, 225–245. passive viewing and working memory Tanaka, K., Saito, H., Fukada, Y., & Moriya, M. performance of rhesus monkeys: Evidence for (1991). Coding visual images of objects in the intrinsic specialization of neuronal coding. inferotemporal cortex of the macaque monkey. Cerebral Cortex, 9, 459–475. Journal of Neurophysiology, 66, 170–189.Seltzer, B., & Pandya, D. N. (1994). Parietal, Tenenbaum, J. B., & Freeman, W. T. (2000). temporal, and occipital projections to cortex Separating style and content with bilinear of the superior temporal sulcus in the rhesus models. Neural Computation, 12, 1247–1283. monkey: A retrograde tracer study. Journal of Torralba, A., Murphy, K., Freeman, W., & Rubin, Comparative Neurology, 343, 445–463. M. (2003). Context-based vision system forSergent, J., Ohta, S., & MacDonald, B. (1992). place and object recognition. Computer Vision, Functional neuroanatomy of face and object 2003. Proceedings of Ninth IEEE International processing. A positron emission tomography Conference (pp. 273–280). study. Brain, Part 1, 15–36. Tsao, D. Y., Freiwald, W. A., Knutsen, T. A.,Serre, T., Oliva, A., & Poggio, T. (2007a). Mandeville, J. B., & Tootell, R. B. (2003). Faces A feedforward architecture accounts for rapid and objects in macaque cerebral cortex. Nature categorization. Proceedings of the National Neuroscience, 6, 989–995. Academy of Sciences, 104, 6424. Tsao, D. Y., Freiwald, W. A., Tootell, R. B. H., &Serre, T., Wolf, L., Bileschi, S., Riesenhuber, M., & Livingstone, M. S. (2006). A cortical region Poggio, T. (2007b). Robust object recognition consisting entirely of face-selective cells. with cortex-like mechanisms. In: IEEE Science, 311, 670–674.

OBJECT RECOGNITION: PHYSIOLOGICAL AND COMPUTATIONAL INSIGHTS 499Tsao, D. Y., & Livingstone, M. S. (2008). Wang Y, Fujita I, Murayama Y (2000) Neuronal Mechanisms of face perception. Annual Review mechanisms of selectivity for object features of Neuroscience, 31, 411–437. revealed by blocking inhibition inTsao, D. Y., Moeller, S., & Freiwald, W. A. inferotemporal cortex. Nature Neuroscience 3, (2008). Comparing face patch systems in 807–813. macaques and humans. PNAS, 105, 19514– Webster, M. J., Bachevalier, J., & Ungerleider, L. G. 19519. (1993). Subcortical connections of inferiorTsunoda, K., Yamane, Y., Nishizaki, M., & Tanifuji, temporal areas TE and TEO in macaque M. (2001). Complex objects are represented in monkeys. Journal of Comparative Neurology, macaque inferotemporal cortex by the 335, 73–91. combination of feature columns. Nature Webster, M. J., Bachevalier, J., & Ungerleider, L. G. Neuroscience, 4, 832–838. (1994). Connections of inferior temporal areasUngerleider, L. G., Galkin, T. W., Desimone, R., & TEO and TE with parietal and frontal cortex in Gattass, R. (2008). Cortical connections of area macaque monkeys. Cerebral Cortex, 4, 470–483. V4 in the macaque. Cerebral Cortex, 18, Webster, M. J., Ungerleider, L. G., & Bachevalier, J. 477–499. (1991). Connections of inferior temporal areasVogels, R., Biederman, I., Bar, M., & Lorincz, A. TE and TEO with medial temporal-lobe (2001). Inferior temporal neurons show greater structures in infant and adult monkeys. sensitivity to nonaccidental than to metric Journal of Neuroscience, 11, 1095–1116. shape differences. Journal of Cognitive Wiskott, L. (2004). How does our visual system Neuroscience, 13, 444–453. achieve shift and size invariance? In: J. L. vanVon Bonin, G., & Bailey, P. (1947). The neocortex of Hemmen & T. J. Sejnowski (Eds.), 23 Problems Mucoca mulutru. Urbana, IL: University of in systems neuroscience. New York: Oxford Illinois Press. University Press.Wallis, G., & Rolls, E. T. (1997). Invariant face and Young, A. W., Hellawell, D., & Hay, D. C. (1987). object recognition in the visual system. Progress Configurational information in face perception. in Neurobiology, 51, 167–194. Perception, 16, 747–759.Wang, G., Tanaka, K., & Tanifuji, M. (1996). Yukie, M., & Iwai, E. (1988). Direct projections Optical imaging of functional organization in from the ventral TE area of the inferotemporal the monkey inferotemporal cortex. Science, 272, cortex to hippocampal field CA1 in the monkey. 1665–1668. Neuroscience Letters, 88, 6–10.

CHAPTER 25The Primate Frontal and Temporal Lobes andTheir Role in Multisensory Vocal CommunicationLizabeth M. Romanski and Asif A. GhazanfarBy exploring how existing primates use their voca- some nonprimate animals share these capacities,lizations, numerous investigators are building a suggesting that these are general features of therigorous, testable framework for how speech mammalian auditory system that are not specificmight have evolved. In the perceptual domain, to speech. For instance, chinchillas are able toseveral studies compared the auditory psychophy- discriminate human speech sounds as well (Kuhlsical abilities of monkeys and humans discrimi- & Miller, 1975) and sea lions show a left hemi-nating speech sounds. Japanese macaques spheric bias for processing their species-specific(Macaca fuscata) (Sinnott, 1989; Sinnott et al., calls (Boye et al., 2005). Taken together, these1997), vervets (Chlorocebus pygerethrus) (Sinnott, data suggest that the processes underlying the1989), baboons (Heinz & Brady, 1988), and perception of purely auditory components ofchimpanzees (Pan troglodytes) (Kojima, 1990; vocalizations are not definitive in distinguishingKojima & Kiritani, 1989) are able to discriminate between what is unique about speech and speechhuman vowel sounds. Much of speech perception (Trout, 2001).discrimination is thought to be dependent upon One aspect of speech that is overlooked bytemporal cues (Shannon et al., 1995), and both comparative biologists seeking to understand itsrhesus monkeys (Macaca mulatta) (Ghazanfar et origins is the fact that speech is not a purelyal., 2001; Hauser et al., 1998) and cotton-top auditory phenomenon. Human speech is a mul-tamarins (Saguinus oedipus) (Ghazanfar et al., tisensory function and face-to-face communica-2002; Miller & Hauser, 2004) are similarly tion is perceived through both the visual andsensitive to the temporal cues in their own voca- auditory channels. Multisensory speech percep-lizations. Furthermore, this temporal processing tion is evident even at the earliest stages ofof speech seems to be biased toward the left hemi- human cognitive development (Patterson &sphere of the human brain, and some studies of Werker, 2003); its integration across the twomonkeys have also borne out a parallel with this modalities is ubiquitous and automatic (e.g.,aspect of human speech processing. When per- the McGurk effect, McGurk & MacDonald,ceiving their own vocalizations, both Japanese and 1976), and even at the neural level, audiovisualrhesus macaques show a left hemispheric bias speech integration is evident at the ‘‘earliest’’when listening to conspecific vocalizations, and stages of cortical processing (Ghazanfar &in most cases this bias is linked to temporal cues Schroeder, 2006). Indeed, Rosenblum (2005)(Ghazanfar et al., 2001; Hauser et al., 1998; has proposed that multisensory speech is thePetersen et al., 1978). primary mode of speech perception and is not The problem with these studies, in terms of a capacity that is ‘‘piggybacked’’ on to auditoryidentifying the origins of speech, is that at least speech perception. Specifically, he suggests that500

MULTISENSORY VOCAL COMMUNICATION 501the primacy of multisensory speech implies the three-dimensional motion of the face alonethat the perceptual mechanisms, neurophy- (Yehia et al., 1998), as can the motion of thesiology, and evolution of speech perception are tongue (an articulator that is not necessarilybased on primitives that are not tied to a single coupled with the face) (Jiang et al., 2002; Yehiasensory modality. The essence of this idea is et al., 1998). The spatiotemporal behavior of theshared by many other investigators (Fowler, vocal tract articulators involved in sound pro-2004; Liberman & Mattingly, 1985; Meltzoff & duction constrains the shape and time course ofMoore, 1997). visible orofacial movement. Such speech-related Some have suggested that the ability to facial motion, distributed around and beyonddevelop/evolve language depends on the ability the mouth, is what is used in the bimodal inte-to form multisensory associations and imply gration of audiovisual speech signals by humans.that this ability is unique to humans For example, human adults automatically link(Geschwind, 1964). If its multisensory nature is high-pitched sounds to facial postures produ-a fundamental feature of human speech, cing an /i/ sound and low-pitched sounds towhereby visual/facial and vocal signals are inex- faces producing an /a/ sound (Kuhl et al., 1991).tricably linked, then how did such a mechanism In primate vocal production, there is aevolve? This chapter explores this issue by pre- similar link between acoustic output and facialsenting (1) behavioral evidence that nonhuman dynamics. Different rhesus monkey vocaliza-primates (hereafter, primates) integrate face and tions are produced with unique lip configura-voice information, (2) anatomical evidence that tions and mandibular positions, and the motionthe temporal and frontal cortices of primates are of such articulators influences the acoustics ofreciprocally connected and are part of a circuit the signal (Hauser & Ybarra, 1994; Hauser et al.,that subserves the integration of face and vocal 1993). Coo calls, like /u/ in speech, are producedsignals, and (3) physiological evidence that cor- with the lips protruded, while screams, like the /tical areas in the temporal and frontal lobes of i/ in speech, are produced with the lips retractedprimates show integrative responses to com- (Fig. 25.1A). The jaw position and lip configura-bined face/voice stimuli. tion affect the formant frequencies independent of the source frequency (Hauser & Ybarra, 1994; Hauser et al., 1993). Moreover, as in humans,FACIAL MOVEMENT AND VOCAL the articulation of these expressions has visibleACOUSTICS ARE LINKED consequences on facial motion beyond the oralPrimate vocalizations are produced by coordi- region. Grimaces, produced during screamnated movements of the lungs, larynx (vocal vocalizations for instance, cause the skin-foldsfolds), and the supralaryngeal vocal tract (Fitch around the eyes to increase in number. In addi-& Hauser, 1995; Ghazanfar & Rendall, 2008). tion to these production-related facial move-The vocal tract consists of the column of air ments, some vocalizations are associated withderived from the pharynx, mouth, and nasal visual cues that are not directly related to thecavity. The source signals (sounds generated by articulatory movement. Threat vocalizations,the lungs and larynx) travel through the vocal for instance, are produced with intense staring,tract and are filtered according to its shape, eyebrows raised, and ears pulled back (Partan,resulting in vocal tract resonances or formants 2002). Head position and motion (e.g., chindiscernable in the spectra of some vocalizations up vs. chin down vs. neutral position) also(for primates, see Fitch, 1997; Owren et al., 1997; varies according to vocal expression typeRendall et al., 1998). In humans, speech-related (Partan, 2002). Chimpanzees (Pan troglodytes),vocal tract motion results in the predictable though less studied in the domain of multisen-deformation of the face around the oral aperture sory communication, also have a link betweenand other parts of the face (Jiang et al., 2002; facial expression and vocal acoustics. FigureYehia et al., 1998, 2002). In fact, the spectral 25.1B shows the facial expressions producedenvelope of a speech signal can be predicted by for two chimpanzee vocalizations. Bauer (1987)

502 PRIMATE NEUROETHOLOGYanalyzed video and audio tracks of chimpanzee changes in visible articulators such as lip andvocalizations and found that a decline in the teeth opening. Thus, it is likely that many offundamental frequency (F0) occurred when sub- the facial motion cues that humans use formissive screams transitioned into aggressive speech reading are present in at least some apesbarks. These changes in F0 were correlated with and monkeys as well. A 16 Frequency (kHz) 0 0 0.8 16 Frequency (kHz) 0 0 0.26 Time (s) B 5 Frequency (kHz) 0 0 2.1 5 Frequency (kHz) 0 0 3.8 Time (s)Figure 25.1 Exemplars of the facial expressions produced concomitantly with vocalizations. (A) Rhesusmonkey coo and scream calls taken at the midpoint of the expressions with their correspondingspectrograms. (B) Chimpanzee visual components of pant-grunt and scream calls. Next to them arespectrograms of a pant-grunt and scream calls. Chimp expressions reprinted from Izumi, A., & Kojima,S. (2004). Matching vocalizations to vocalizing faces in a chimpanzee (Pan troglodytes). AnimalCognition, 7, 179–184. Sound exemplars taken from the Harvard Cognitive Evolution Lab, www.wjh.harvard.edu/~mnkylab/media/chimpcalls.html.

MULTISENSORY VOCAL COMMUNICATION 503MATCHING FACES TO VOICES correspondence between the heard sound and the appropriate facial posture, then, overall,Given that both humans and other extant pri- they would spend more time looking at themates use both facial and vocal expressions as matching video. The results supported thecommunication signals, it is unlikely that only prediction. The mean percentage of lookinghumans recognize the correspondence between time devoted to the match was 67.3%; thisthe visual and auditory components of vocal was significantly greater than the 50% chancesignals. To test whether other primates do, level (Ghazanfar & Logothetis, 2003).in fact, have this capacity, a preferential looking Furthermore, all 11 subjects looked longer atparadigm was adopted (Ghazanfar & Logothetis, the matching face. When comparing looking2003). In this paradigm, subjects are seated in durations between coos and threats, subjectsfront of two video monitors, one playing the looked longer when the matching face wasfacial dynamics of one expression and the other articulating the coo call than when the matchplaying the facial dynamics of a second expres- face was the threat call (likely because threatssion. In between the two screens, a loudspeaker are aversive); nonetheless, separate analysesplays back the vocalization that matches only revealed that looking preferences were still sig-one of the facial expressions being shown. The nificant for coo calls alone and threat callsdependent measure is the amount of time the alone.subjects spend looking at this matching video as Although these data demonstrate that mon-a proportion of the total looking time to both keys can match faces to voices, in this particularscreens. This paradigm was developed by cogni- experiment they may have been doing so usingtive scientists to test intermodal speech percep- low-level stimulus cues. The coo and threat callstion in prelinguistic infants (Kuhl & Meltzoff, differ dramatically in their duration—the threat1984; Patterson & Werker, 2003) and thus it is is almost invariably shorter than the coo. Thus, itperfect for testing the natural capacities of pri- is possible that the monkeys may be using a cuemates without training or reward. that is redundant between the two modalities— Using this procedure, rhesus monkeys were matching the duration of mouth movements totested on their ability to recognize the audi- the duration of the auditory componenttory-visual correspondences between their (Lewkowicz, 2002). This type of matching does‘‘coo’’ and ‘‘threat’’ calls (Ghazanfar & not necessitate the identification of the parti-Logothetis, 2003). These are among the most cular facial expression (i.e., the morphology offrequently produced calls in the rhesus the mouth). A subsequent study with capuchinmonkey repertoire, both in the wild and in monkeys (Cebus apella) avoided this confoundcaptivity. Coo calls are tonal signals of rela- by using stimuli that were better controlled fortively long duration and are produced in many duration and included vocal expressions fromaffiliative contexts, including group move- not only conspecifics but heterospecifics asments, separation, and feeding; threat calls, well (rhesus monkey and human vocalizations)in contrast, are noisy, short-duration pulsatile (Evans et al., 2005). Without duration cues, thecalls produced during aggressive encounters capuchins could still match the appropriate(Hauser & Marler, 1993; Rowell & Hinde, facial expression to the heard vocalization1962). Each of these calls is associated with a across all stimulus conditions. However, it wasunique facial posture. The hypothesis was that not clear whether they can make this match forthe vocalization heard from the central each species’ set of calls or for only the capuchinspeaker would systematically influence the calls. Another way to avoid the temporal cuesduration of subjects’ visual fixations on the confound is to use a match-to-sample beha-two screens—one that displayed the video of vioral paradigm that requires training subjects.a coo expression and the other that displayed Chimpanzees can match conspecific faces andthe video of the threat expression. Specifically, voices under these conditions (Izumi &if rhesus monkeys recognized the Kojima, 2004; Parr, 2004).

504 PRIMATE NEUROETHOLOGYMATCHING MULTIPLE FACES TO This capability is on par with the perceptualVOICES REVEALS ACOUSTIC separation of voices by humans via pitch differ-STREAM SEGREGATION ences and harmonicity (Brokx & Nooteboom, 1982; Summerfield et al., 1992). This is notableGiven that monkeys can readily match faces to because a previous study found that highlyvoices, can they do something with these signals trained monkeys could discriminate con-that is slightly more complicated and relevant current sequences of artificial sounds onlyto their everyday socio-ecological tasks? One when their frequency ranges did not overlapproblem that they might confront is segregating (Izumi, 2002).competing voices in a chorus, much as humansmight do in a cocktail party scenario. Rhesusmonkeys produce a chorus of coos and othervocalizations upon the anticipation or dis- MATCHING ACOUSTIC SIZE WITHcovery of food. Using the preferential looking VISUAL SIZEparadigm, monkeys were tested on their ability Vowels and consonants, the essential phoneticto segregate the competing voices of two or elements of all human speech, differ in theirthree unfamiliar individuals in the auditory resonances, or formants, produced by the vocaldomain and to then attend to the video screen tract—the nasal and oral cavities above the vocalthat showed the correct number of individuals folds (Ghazanfar & Rendall, 2008). As describedthat they heard (Jordan et al., 2005). In this previously, during vocal production in humansexperiment, two versus three coo calls were and other primates, pulses of air generated byused that were equal in duration and tempo- the rapid movement of the vocal folds producerally coincident. The video displays depicted an acoustic signal. As signals pass through thetwo synchronously cooing individuals on one vocal tract, they excite resonances in the vocalside and three individuals on the other side. It tract resulting in the enhancement of particularwas hypothesized that if they could segregate frequency bands; these are the formants. Thethe voices and assess their number, then they ability to create and perceive a wide variety ofwould preferentially attend to the dynamic different formant patterns is a prerequisite forvisual displays featuring the number of conspe- human speech. This fact raises the following twocifics they simultaneously heard vocalizing. questions. First, how did the ability to perceiveUnder these conditions, monkeys spent a differences between formant patterns arise?greater proportion of time looking at the dis- Second, which role, if any, did formants play inplay that numerically matched the number of prelinguistic primates? The answers to these ques-vocalizers they heard compared to the non- tions may lay not so much in the fact that formantmatching display (Jordan et al., 2005). patterns are important phonetic elements of This spontaneous, multisensory representa- speech, but that they also carry important infor-tion of multiple individuals in animals is an mation related to physical characteristics of theimportant parallel to similar representations in individual speaker (Fitch, 1997; Smith &humans (Barth et al., 2003). Using a nearly iden- Patterson, 2005). Specifically, the length of thetical stimulus paradigm but with human voices vocal tract determines the frequencies of the for-(Jordan & Brannon, 2006), human infants (aged mants. Large individuals with long vocal tracts7 months) performed nearly identically to have lower-frequency formants than do smallerrhesus monkeys. This suggests that perhaps individuals who have smaller vocal tracts.there are homologous underlying mechanisms Previous behavioral studies demonstrated(see Zangehenpour et al., 2009, for important that trained baboons (Heinz & Brady, 1988)caveats). These results also indicate that rhesus and macaques (Le Prell et al., 2001; Sinnott,monkeys can segregate simultaneously pre- 1989; Sommers et al., 1992) can discriminatesented coo vocalizations, even though the different human vowel sounds presumablypower spectra of the calls are highly overlapping. based on formant frequency differences. Fitch

MULTISENSORY VOCAL COMMUNICATION 505and Fritz (2006) significantly extended these to the physical characteristics of the vocalizerfindings by showing that rhesus monkeys can, (Fitch, 1997; Owren et al., 1997; Rendall et al.,without training, discriminate differences in the 1998). The results further suggest that the use offormant structure of their own conspecific calls. formant cues in the perception of vowel soundsCan rhesus monkeys use formants as acoustic by humans in a linguistic context emerged gra-cues to assess age-related body size differences dually, perhaps for other functional reasons,among conspecifics? The potential role of for- over the course of human evolution.mants as indexical cues in rhesus monkey voca- Perception of indexical cues, such as age-relatedlizations was tested using the preferential body size, via formants in vocalizations may belooking method (Ghazanfar et al., 2007). When one functional link between the vocalizations ofpresented with two videos, one depicting an human and other primates.older, larger macaque and the other showing ayoung, small monkey (Fig. 25.2A), subjects NEURAL CIRCUITS MEDIATINGspent more time looking at the larger monkey MULTISENSORY VOCALwhen they heard a coo produced with a large COMMUNICATIONmonkey’s vocal tract and vice versa when theyheard a coo produced with a small monkey’s In spite of much evidence to the contrary (pre-vocal tract (Fig. 25.2B–D). In other words, mon- sented here for bimodal vocal communicationkeys spent more time looking at the video with and elsewhere for cognitive and social functions;the monkey that matched in size what they heard see Seyfarth & Cheney, this volume), there arein the audio track (Ghazanfar et al., 2007). some who would argue that the vocal commu- These data are the first direct evidence for the nication of primates is of a different kind alto-hypothesis that formants embedded in the gether from that of human speech (Arbib, 2005;acoustic structure of primate calls provide cues Pinker, 1994). Some of these arguments are theA B C D 11 VTL = 10 cm VTL = 10 cm Volts frequency (kHz) –10 60 Percentage of total looking time Amplitude –30 50 0 1 –50 40 –1 0 350 0 2 4 6 8 10 VTL = 5.5 cm VTL = 5.5 cm 30 11 –10 Volts frequency (kHz) Amplitude –20 20 –30 0 1 –40 10 –50 –1 0 350 0 2 4 6 8 10 Time (ms) Matching Head Size Frequency (kHz)Figure 25.2 (A) Still frames extracted from the videos used in the preferential looking experiments. Thetop shows a frame from the large monkey. Videos were synchronized and edited so that they appeared to besynchronously producing the coo vocalization shown in B. (B) Resynthesized coo vocalizations based onone of the two coo exemplars used in both the habituation-discrimination and preferential lookingparadigms. Diagram shows the spectrograms and waveforms of a coo vocalization resynthesized with twodifferent vocal tract lengths. The arrow in the spectrogram indicates the position of an individual formant,which increases in frequency as the apparent vocal tract length decreases. Only the coos based on 5.5-cm and10-cm vocal tract lengths were used as stimuli. (C) Power spectra (black line) and linear predictive codingspectra (gray lines) for the long vocal tract length (10 cm, top panel) and short vocal tract length (5.5 cm,bottom panel) used in the experiment and seen in B. (D) The mean percentage of total looking time spentlooking at the matching video display; the dotted line indicates chance expectation. Error bar is SEM.

506 PRIMATE NEUROETHOLOGYresult of an overeager interpretation of a subset receive inputs from a number of sensory areas.of monkey neurobiological data (e.g., mirror Consistent with this idea, the prefrontal cortexneurons) that are potentially related to the evo- (PFC) and the dorsal (or upper) bank of thelution of human speech. Others simply argue superior temporal sulcus (STS) both receive athat the brains of monkeys and humans are wealth of afferents from sensory areas. In parti-fundamentally different when it comes to circular, these association cortices receive projec-cuits involved in communication. Indeed, tions from auditory and visual cortical areas,Pinker (1994) has suggested the following: some of which are also connected with each‘‘Language could have arisen, and probably did other.arise. . .by a revamping of primate brain circuitsthat originally had no role in primate vocal com- Visual Connections of Superior Temporalmunication’’ (p. 350). For the most part, these Sulcus and Prefrontal Cortexarguments ignore much of what we know aboutthe processing of vocalizations in the primate Tracer injections into areas within the STS andbrain (Ghazanfar & Hauser, 1999, 2001) and the PFC indicate that each receives projectionsthe numerous behavioral similarities in the use from both dorsal and ventral stream visual areas.of vocalizations by our primate cousins (some of Traditionally, the dorsal bank of the rostral STS,which were described previously; see also chap- which includes area TPO, was thought to beters by Miller & Cohen and Seyfarth & Cheney in connected with the parietal, prefrontal, andthis volume). Certainly most, if not all, accounts superior temporal auditory regions (Seltzer &of the evolution of primate vocal communica- Pandya, 1978, 1991, 1994). This implied thattion ignore that fact that it is integrated across TPO might not receive connections carryingmultiple sense modalities. object, form, or face processing information The compounding evidence that monkeys, directly from visual areas like the inferotemporallike humans, combine auditory and visual signals cortex. However, injections of anterogradesuggests two things. First, monkeys are an acces- tracer into distinct dorso-anterior (TEad) andsible model system for investigating the sensor- ventro-anterior (TEav) subdivisions of the infer-imotor integration of a natural communicative otemporal cortex showed a dense distribution ofbehavior. Second, monkeys and humans likely terminals and cell bodies in TPO following TEadshare some homologous neural substrates to injections, whereas labeling was confined to themediate such multisensory behavior. For lower bank and fundus of the rostral STS fol-instance, although such behavior likely involves lowing tracer injections into TEav (Saleem et al.,many disparate regions in the brain, there is a 2000). Thus, rostral TPO is reciprocally con-robust, three-node cortical circuit that may be nected with ventral stream visual processingmore heavily weighted for processing facial- areas in inferotemporal cortex. Middle andvocal signals. The nodes include the prefrontal caudal TPO also receive projections from visualcortex, the superior temporal sulcus, and the processing areas (Seltzer & Pandya, 1994). Whenauditory cortex. These three regions are densely different tracers were injected into both pos-interconnected. The following sections will terior parietal and inferotemporal cortex of thereview the anatomical circuits and neurophysio- same hemisphere, overlapping anterogradelogical processes that are thought to mediate (at fibers were found near the fundus of the STSleast in part) multisensory communication in (Baizer et al., 1991), supporting the idea thatprimates. this area receives converging inputs from the dorsal and ventral visual streams. With regard to the frontal lobe, early anato-FRONTAL AND TEMPORAL LOBE mical studies by Barbas, Pandya, and their col-CIRCUITRY leagues (Barbas, 1988; Barbas & Mesulam, 1981;One principle of anatomical connections in the Barbas & Pandya, 1989; Chavis & Pandya, 1976)neocortex is that ‘‘high-level’’ association areas noted a segregation of visual inputs to discrete

MULTISENSORY VOCAL COMMUNICATION 507regions of the prefrontal cortex. For example, caudal STG and caudal PFC are reciprocallyBarbas (1988, p. 313) wrote ‘‘. . .the basoventral connected (Chavis & Pandya, 1976; Pandya &prefrontal cortices are connected with ventral Kuypers, 1969; Petrides & Pandya, 1988, 2002;visual areas implicated in pattern recognition Romanski et al., 1999a,b), while the rostral STGand discrimination, whereas the mediodorsal is reciprocally connected with rostral prefrontalcortices are connected with medial and dorso- areas around the principal sulcus (areas 46 andlateral occipital and parietal areas associated 10) and orbito-frontal areas (areas 11 and 12)with visuospatial functions.’’ This dissociation (Chavis & Pandya, 1976; Pandya & Kuypers,was echoed by Bullier and colleagues (1996), 1969; Pandya et al., 1969). In the last decade,who found some segregation of inputs to PFC studies carefully characterized temporo-pre-when paired injections of tracers were placed frontal connections utilizing a better under-into temporal and parietal visual processing standing of auditory cortical organization.regions. In their study, visual temporal cortex These studies refined the rostral-caudal topo-projected mainly to area 45, located ventrolater- graphy that was previously noted by deter-ally in the PFC, while parietal cortex sent projec- mining that the frontal pole and anteriortions to both ventrolateral PFC (area 45) and principal sulcus are densely connected withdorsolateral PFC (areas 8a and 46) (Bullier et al., anterior belt and parabelt regions of the auditory1996). The specific connectivity of ventrolateral cortex (Hackett et al., 1999; Romanski et al.,PFC areas 12 and 45, which contain object- and 1999a), while the anterior third of the temporalface-selective neurons (O’Scalaidhe et al., 1997, lobe, designated STGr (Hackett et al., 1999), has1999; Wilson et al., 1993), with inferotemporal strong reciprocal connections with orbitofrontalareas TE and TEO was specifically documented areas (Hackett et al., 1999; Romanski et al.,by Webster and colleagues (1994). Comparison 1999a). Moreover, the caudal parabelt and beltof TE and TEO connectivity revealed a number of auditory cortex are reciprocally connectedof important differences, including the finding with the dorsal prearcuate and caudal principalthat it is mainly area TE that projects to ventro- sulcus.lateral PFC and orbitofrontal areas 11, 12, and Direct evidence of auditory cortex innerva-13. These orbital regions also contain object- tion of the PFC was shown by placing tracerand face-responsive cells (Thorpe et al., 1983). injections into frequency matched locations in tonotopically organized areas of the lateral auditory belt—areas AL, ML, and CL (Romanski etAuditory Connections of Superior Temporal al., 1999b). The connections between the frontalSulcus and Prefrontal Cortex cortex and the auditory belt/parabelt were topo-Auditory regions in the superior temporal gyrus graphically organized such that projections from(STG) and plane send projections that terminate AL typically involved the frontal pole (area 10),in rostral and middle portions of the STS. the rostral principal sulcus (area 46), the inferiorPolysensory area TPO and area TAa both receive convexity (areas 12vl and 45) and the laterala dense projection from portions of the rostral orbital cortex (areas 11, 12o). In contrast, pro-and middle STG (Seltzer & Pandya, 1994; Seltzer jections from area CL targeted the dorsal peri-et al., 1996). More direct evidence revealed that arcuate cortex (area 8a, frontal eye fields) andthe parabelt and belt auditory regions, whose the caudal principal sulcus (area 46) as well asneurons are responsive to complex sounds the caudal inferior convexity (areas 12vl and 45)(Rauschecker et al., 1995; Romanski et al., and, in two cases, premotor cortex (area 6d).1999a), also project to areas TPO and TAa Thus, projections from the auditory association(Hackett et al., 1999). cortex to the prefrontal cortex are topographic. The connections of the PFC with superior Furthermore, while small topographic projec-temporal auditory areas have been studied in tions target the PFC from the lateral belt, para-great detail. Early anatomical studies indicated belt projections are more dense (Hackett et al.,that a rostro-caudal topography exists such that 1999; Romanski et al., 1999a) (Fig. 25.3).

508 PRIMATE NEUROETHOLOGY cs asd 8a 9 46 AI CL 12 45 asv 10 ML ls AL stsFigure 25.3 Connections between the auditory belt and parabelt are summarized on the lateral brainschematic of the macaque monkey. Combined physiology and anatomy studies demonstrated topographicconnections between the anterior belt (AL and anterior ML) and the rostral and ventral prefrontal cortex, areas10, rostral 46, 12vl, and 12o. In contrast, the caudal belt (areas CL and caudal ML) project to caudal and dorsalprefrontal cortex including areas 8a (the frontal eye fields), caudal 46, and the dorsal portion of areas 45 and 12.Larger connections originate in the parabelt and echo this same rostral and caudal topography. asd, dorsalarcuate sulcus; asv, ventral arcuate sulcus; cs, central sulcus; ls, lateral sulcus; sts, superior temporal sulcus.ENCODING AND INTEGRATION integration of simple artificial stimuli (see Stein,OF FACES AND VOICES IN THE 1998, for review). Although many principles ofTEMPORAL AND FRONTAL LOBES multisensory integration have been identified for neurons in the superior colliculus, it is unknownThe anatomical data strongly suggest an interac- to what extent these principles apply to complex,tion between multiple cortical areas in mediating naturalistic stimuli in the primate neocortex.auditory-visual integration. This is supported by To bridge this epistemic gap between what wehuman neuroimaging and EEG studies of audio- know about human and monkey neural processes,visual speech. The imaging data consistently recent studies have investigated dynamic face/show that the superior temporal sulcus, auditory voice integration in the superior temporal sulcuscortex (including primary auditory cortex), and (Barraclough et al., 2005; Chandrasekaran &prefrontal cortex play important roles in the Ghazanfar, 2009), auditory cortex (Ghazanfarintegration process (Calvert, 2001). Human et al., 2005, 2008), and ventrolateral prefrontalneuroimaging and EEG data of audiovisual cortex (Sugihara et al., 2006) of the macaquespeech have added much to our knowledge of monkey brain using the species’ natural commu-its underlying cortical circuitry and have gener- nication signals. Unlike pairings of artificial sti-ated several interesting hypotheses; however, the muli, audiovisual vocalizations are ethologicallynature of these signals precludes a direct under- relevant and thus may tap into specialized neuralstanding of how this network functions and mechanisms (Ghazanfar & Santos, 2004) or, mini-interacts. Problems inherent in these imaging mally, integrative mechanisms for socially learnedand event-related potential studies include the audiovisual associations.inability to distinguish between inhibitoryversus excitatory responses, poor source localiza- THE SUPERIOR TEMPORALtion, and/or poor temporal resolution. For many SULCUSdecades, animal models of multisensory integra-tion have largely focused on the role of the The superior temporal sulcus has long beensuperior colliculus in the spatial and temporal known to have neurons responsive to visual,

MULTISENSORY VOCAL COMMUNICATION 509auditory, and/or somatosensory stimuli when it did, the responses could not be explained(Benevento et al., 1977; Bruce et al., 1981; by the TTV variable. In contrast, the alpha bandHikosaka et al., 1988). Few attempts, however, showed enhancement of power when presentedwere made to intensively investigate whether with face/voice combinations and this enhance-these polysensory neurons integrated stimuli ment was dependent on the TTV—short TTVsfrom different modalities. These early studies elicited enhancement, while long TTVs elicited noreported anecdotal evidence that, in a few neurons, integration. The gamma band revealed a third,auditory stimuli attenuated visual stimuli distinct integrative capacity. The gamma band(Benevento et al., 1977; Bruce et al., 1981). A invariably showed enhanced responses to face-recent study systematically investigated the inte- voice pairings and these responses were completelygrative properties of single neurons in STS using independent of the TTV parameterbiologically relevant dynamic stimuli such as rip- (Chandrasekaran & Ghazanfar, 2009).ping paper, chewing, and vocalizations The finding that different neural frequency(Barraclough et al., 2005). Twenty-three percent bands show independence in their face/voiceof neurons responsive to the sight of biological integration strategies is supported by findingsmotion could be significantly modulated by the in other domains. An increasing number of stu-corresponding auditory component. The sound of dies are revealing that different LFP frequencythe action either enhanced or suppressed the ranges are best correlated with different sensoryvisually evoked response for an equal number of or motor parameters (Liu & Newsome,neurons. In neurons where the sound enhanced 2006; Rickert et al., 2005; Wilke et al., 2006).neural responses, the enhancement was dependent Together, these data suggest that differentupon the auditory signal being congruent with the neural frequency bands reflect distinct neuronalvisual one. For example, audiovisual human ‘‘lip- processes and are generated by distinct sources.smacking’’ elicited a stronger response from a One attractive hypothesis is that the frequencygiven neuron than visual lip-smacking alone; how- is inversely proportional to the scale of theever, when visual lip-smacking was paired with cortical network underlying that frequencyother auditory signals (like monkey calls), the (Buzsaki, 2006). Gamma band activity isneuron did not show any enhanced activity. thought to reflect activity in a local corticalConversely, in neurons whose responses to visual microcolumn, whereas alpha and theta bandstimuli were suppressed by sound stimuli, the activity could putatively represent processessuppression was not dependent upon the congru- whose sources may include multiple corticalence of the auditory and visual stimulus. areas acting in a coordinated fashion. In the Beyond single units, the integration of faces and case of face/voice integration, this would sug-voices in the STS was tested both as a function of gest that the areas with which the STS connectsdifferent frequency bands of the local field potential would have similar behavior in the lower fre-(LFP) responses and as a function of the natural quencies but divergent behavior in higher fre-time delay between the initial facial movement and quencies. Putative sources of such inputs intothe onset of the voice component (the ‘‘time-to- STS are reviewed previously. Recordings fromvoice,’’ or TTV) (Chandrasekaran & Ghazanfar, multiple structures simultaneously and joint2009). Faces and voices elicited distinct and con- analysis of these signals (see later; Ghazanfarcurrent activity in different frequency ranges of the et al., 2008; Maier et al., 2008) will help shedSTS LFP: theta (4 to 8 Hz), alpha (8 to 14 Hz), and light onto the mechanisms underlying activitygamma (>40 Hz). Remarkably, these different in these different frequency ranges and theirneural frequency bands integrated faces and role in multisensory integration.voices differently. The theta band showed no con-sistent relationship between the multisensory AUDITORY CORTEXresponse and the time-to-voice; that is, the thetaband did not, in general, show enhancement or Presumptive unimodal sensory areas, such as audi-suppression to face/voice combinations, and tory cortex, are now known to be multisensory as

510 PRIMATE NEUROETHOLOGYwell (Ghazanfar & Schroeder, 2006). The auditory been confirmed at the single-neuron level as wellcortex, in particular, has been the focus of much (Ghazanfar et al., 2008) (Fig. 25.4).attention (Ghazanfar, 2009). Numerous lines of The vast majority of LFP responses wereboth physiological and anatomical evidence specific to face/voice integration and suchdemonstrate that at least some regions of the audi- integration could take the form of either enhance-tory cortex respond to touch as well as sound (Fu ment or suppression, although enhancedet al., 2003; Kayser et al., 2005; Hackett et al., responses were more common (Ghazanfar et al.,2007a,b; Lakatos et al., 2007; Schroeder & Foxe, 2005). The specificity of these integrative responses2002; Smiley et al., 2007). The potential roles of was tested by replacing the dynamic faces withsuch auditory-somatosensory interactions in vocal dynamic discs that mimicked the aperture andcommunication are not known (Ghazanfar, 2009), displacement of the mouth. In human psychophy-but one possibility is that they mediate sensory sical experiments, such artificial dynamic stimulifeedback to guide vocal production (Ghazanfar can still lead to enhanced speech detection, but not& Turesson, 2008; Nasir & Ostry, 2006). In the to the same degree as a real face (Bernstein et al.,auditory-visual domain, the anatomical data 2004; Schwartz et al., 2004). When cortical sites orsuggest (reviewed previously) that the auditory single units were tested with dynamic discs, far lesscortex has many potential sources of visual integration was seen when compared to the realinputs. This is supported by an increasing monkey faces (Ghazanfar et al., 2005, 2008) (Fig.number of functional and physiological studies 25.4). This was true primarily for the lateral beltin monkeys (Ghazanfar et al., 2005, 2008; auditory cortex and was observed to a lesser extentKayser et al., 2007, 2008). Here we focus on in the primary auditory cortex.those auditory cortical studies investigating Coos and grunts are both produced in aface/voice integration. variety of affiliative contexts (Hauser & In order to investigate the integrative properties Marler, 1993). In light of this, another inter-of the auditory cortex, monkey subjects viewed esting finding that came out of this study is thatunimodal and bimodal versions of two different grunt vocalizations were overrepresented rela-species-typical vocalizations (‘‘coos’’ and tive to coos in terms of enhanced multisensory‘‘grunts’’) while performing a fixation task LFP responses. One possibility is that this isrequiring them to maintain their eye fixations because grunts are spectrally noisy calls andwithin the video frame. Recordings of the mean they thus evoked responses in more neuronsextracellular field potential (i.e., unit and subthres- due to their broad frequency spectrum; how-hold neural activity) were made using intracranial ever, the frequency spectra of coos overlappedelectrodes placed in the core auditory cortex considerably with grunt spectra. Another pos-(which includes primary and primary-like audi- sibility is that this differential representationtory areas) and in the lateral belt auditory cortex (a may reflect a behaviorally relevant distinction,higher-order auditory area) (Ghazanfar et al., as coos and grunts differ in their direction of2005). The stimulus conditions were Face+Voice, expression and range of communication. CoosFace alone, and Voice alone. Consistent with are generally contact calls rarely directedevoked potential studies in humans (Besle et al., toward any particular individual. In contrast,2004; van Wassenhove et al., 2005), the combina- grunts are often directed toward individuals intion of faces and voices led to integrative responses one-on-one situations, often during social(significantly different from unimodal responses) approaches (Cheney & Seyfarth, 1982;in the vast majority of auditory cortical sites— Palombit et al., 1999). Given their productionboth in primary auditory cortex and the lateral at close range and context, grunts may producebelt auditory cortex. The data demonstrated that a stronger face/voice association than coo calls.raw LFP signals and multiunit activity in the This distinction appeared to be reflected inauditory cortex are capable of multisensory the pattern of significant multisensoryintegration of facial and vocal signals in monkeys responses in auditory cortex; that is, this bias(Ghazanfar et al., 2005) and this has subsequently toward grunt calls may be related to the fact

MULTISENSORY VOCAL COMMUNICATION 511 Face+Voice Pr Grunt Gr Grunt Voice 100 80 Face Disc+Voice 0 0spikes/s –400 0 400 800 1200 –400 0 400 800 1200 He Coo Kz Grunt 100 60 0 0 –400 0 400 800 1200 –400 0 400 800 1200 Time (ms) Time (ms)Figure 25.4 Examples of multisensory integration of Face+Voice stimuli compared to Disc+Voicestimuli in single auditory neurons recorded from the lateral belt region. Left panels show enhancedresponses when voices are coupled with faces, but no similar modulation when coupled with discs.Right panels show similar effects for suppressed responses. The insets show frames from theFace+Voice stimulus and the temporally corresponding Disc+Voice stimulus. X-axes show timealigned to onset of the face (solid line). Dashed lines indicate the onset and offset of the voicesignal. Y-axes depict the firing rate of the neuron in spikes per second. Shaded regions denote theSEM of the mean.that grunts (relative to coos) are often pro- following reasons. First, there are reciprocalduced during intimate, one-on-one social connections between the STS and the lateralinteractions. belt and other parts of auditory cortex (see previous discussion; Barnes & Pandya, 1992; Seltzer & Pandya, 1994). Second, neurons in the STS are sensitive to both faces and biologicalINTERACTIONS BETWEEN THE motion (Harries & Perrett, 1991; Oram &SUPERIOR TEMPORAL SULCUS Perrett, 1994). Finally, the STS is known to beAND THE LATERAL BELT multisensory (Bruce et al., 1981; Schroeder &AUDITORY CORTEX MEDIATE Foxe, 2002). One mechanism for establishingINTEGRATION OF FACES AND whether auditory cortex and the STS interact atVOICES the functional level is to measure their temporalThe face-specific visual influence on the lateral correlations as a function stimulus condition.belt auditory cortex begs the question as to its Thus, concurrent recordings of LFPs andanatomical source. Although there are multiple single neurons in the lateral belt of auditorypossible sources of visual input to auditory cortex and the dorsal bank of the STS werecortex (Ghazanfar & Schroeder, 2006), the STS made (Ghazanfar et al., 2008). The functionalis likely to be a prominent one, particularly interactions, in the form of gamma band corre-for integrating faces and voices, for the lations, between these two regions increased

512 PRIMATE NEUROETHOLOGYin strength during presentations of faces no overt responses to faces alone). Second, theseand voices together relative to the unimodal con- integrative responses were often ‘‘face specific’’ditions (Fig. 25.5A). Furthermore, these interac- and had a wide distribution of latencies, whichtions were not solely modulations of response suggested that the face signal was an ongoingstrength, because the phase relationships were signal that influenced auditory responsessignificantly less variable (tighter) in the multi- (Ghazanfar et al., 2008). Finally, this hypothesissensory conditions (Varela et al., 2001). A control for an ongoing signal is supported by the sus-condition, in which the face was replaced with a tained gamma band activity between auditorydynamic disc that mimicked mouth movements cortex and STS (Fig. 25.5A) and by a spike-(as in Ghazanfar et al., 2005), revealed that inter- field coherence analysis of the relationshipareal interactions in general were significantly less between auditory cortical spiking activity androbust (Fig. 25.5A). gamma band oscillations from the STS With regard to spiking activity, not only do (Ghazanfar et al., 2008) (Fig. 25.5B).single auditory cortical neurons integrate faces It should be noted, however, that func-and voices (Fig. 25.4), but this spiking output tional interactions between STS and auditoryalso seems to be modulated (not driven) by cortex are not likely to occur solely during theongoing activity arising from the STS. Three presentation of faces with voices. Other con-lines of evidence suggest this scenario. First, gruent, behaviorally salient audiovisual eventsvisual influences on single neurons were most such as looming signals (Maier et al., 2004,robust when in the form of dynamic faces and 2008) or other temporally coincident signalswere only apparent when neurons had a signifi- may elicit similar functional interactionscant response to a vocalization (i.e., there were (Noesselt et al., 2007). A Face + Voice Face Voice Disc + Voice 198 2.0 Mean normalized amplitude 164 Frequency (Hz) 136 1.8 113 1.6 94 78 1.4 64 54 1.2 44 37 1.0 −400 −200 0 200 400 600 −400 −200 0 200 400 600 −400 −200 0 200 400 600 −400 −200 0 200 400 600 Time (ms) Time (ms) Time (ms) Time (ms) B Face + Voice Face Voice Mean normalized power 198 198 198 Frequency (Hz) 1.3 139 139 139 1.2 97 97 97 1.1 68 68 68 1.0 48 48 48 0.9 −200 −150 −100 −50 0 50 100 −200 −150 −100 −50 0 50 100 −200 −150 −100 −50 0 50 100 Time (ms) Time (ms) Time (ms)Figure 25.5 (A) Time-frequency plots (cross-spectrograms) illustrate the modulation of functionalinteractions (as a function of stimulus condition) between the lateral belt auditory cortex and thesuperior temporal sulcus (STS) for a population of cortical sites. X-axes depict the time in milliseconds asa function of onset of the auditory signal (solid black line). Y-axes depict the frequency of the oscillations inHertz. Color bar indicates the amplitude of these signals normalized by the baseline mean. (B) Spike-fieldcross-spectrogram illustrates the relationship between the spiking activity of auditory cortical neurons andthe STS local field potential across the population of cortical sites. X-axes depict time in milliseconds as afunction of the onset of the multisensory response in the auditory neuron (solid black line). Y-axes depictthe frequency in Hertz. Color bar denotes the cross-spectral power normalized by the baseline mean fordifferent frequencies.

MULTISENSORY VOCAL COMMUNICATION 513VENTROLATERAL PREFRONTAL the same region), or do they simply encodeCORTEX complex sound features in a manner similar to, but at higher level, than auditory cortexWhile much research has focused on under- proper?standing the electrophysiological correlates of Positron emission tomography (PET) andthe prefrontal cortex’s involvement in visual functional magnetic resonance imaging (fMRI)working memory, decision making utilizing studies have suggested that the human inferiorvisual cues, and spatial visual perception, few frontal gyrus, or ventral frontal lobe, plays a rolestudies addressed the cellular mechanisms of in semantic processing (Demb et al., 1995;auditory or multisensory processes. This is Poldrack et al., 1999). In neurophysiologicaldespite the wealth of evidence that the PFC experiments with rhesus monkeys, this questionreceives input from all sensory modalities (see has engendered some controversy (see alsoprevious discussion). Only a handful of studies Miller & Cohen, this volume). Studies fromdemonstrated that neurons in the PFC respond Cohen and colleagues, in which vocalizationsto auditory stimuli or are active during auditory were presented in the context of an oddball pre-tasks in primates (Bodner et al., 1996; Ito, 1982; sentation paradigm, suggested that vocalizationsTanila et al., 1992; Vaadia et al., 1986; Watanabe, with similar meaning (‘‘food calls’’) elicited1992). Weakly responsive auditory neurons were similar prefrontal neuronal responses under par-found sporadically and distributed across a wide ticular conditions (Gifford et al., 2005).region of the PFC including portions of the Romanski and colleagues (2005) tested neuronsventral PFC, principal sulcus, arcuate sulcus, with exemplars from 10 different vocalizationand premotor cortex. None of these previous categories, which included food and nonfoodstudies used species-specific communication calls, and found that prefrontal neurons show acalls as stimuli. In the visual domain, a series of selectivity that is similar to neurons in the lateralstudies from the laboratory of Goldman-Rakic belt area of auditory cortex—the majority ofdemonstrated that neurons within particular neurons were selective for approximately tworegions within the ventrolateral PFC (VLPFC) vocalizations when tested with 10 call typeswere selectively responsive to faces including (Fig. 25.6A,B). Importantly, the types of vocali-species-specific faces (O’Scalaidhe et al., 1997, zations that elicited similar response magnitudes1999). This finding coincided with the demon- shared similar acoustic features, suggesting thatstration that these same regions of VLPFC prefrontal neurons might be encoding complexreceived afferents from the lateral and parabelt acoustics rather than semantics (Averbeck &regions of auditory cortex as well as the dorsal Romanski, 2004, 2006; Romanski et al., 2005)bank of the STS (Hackett et al., 1999; Romanski (Fig. 25.7A–C).et al., 1999a,b). An important point to note is that the rhesus Soon after the localization of face cells in the monkey vocal repertoire is assembled by humanVLPFC, neurons responsive to acoustic stimuli, listeners and is based solely on behavioral con-including species-specific vocalizations, were texts (defined by the human observer) anddiscovered. Unlike auditory cortex, VLPFC neu- acoustic features. Thus, it is not surprising, androns did not readily respond to simple acoustic indeed highly likely, that prefrontal neurons arestimuli such as pure tones (Romanski & capable of categorical discrimination of vocali-Goldman-Rakic, 2002), but were robustly zations and are using complex feature analysis todriven by complex sounds (Averbeck & encode category membership when testedRomanski, 2006; Cohen et al., 2007; Gifford during simple presentation and not within theet al., 2005; Romanski et al., 2005). The ques- confines of a cognitive task. A Hidden-Markovtion then arose, Would these higher-order model analysis of the probability that prefrontalauditory neurons encode the ‘‘semantics’’ of neural activity categorizes vocalizations revealedvocalizations, perhaps independent of sensory that prefrontal neurons can be described asmodality (given the presence of face neurons in linear functions of the probabilities that

514 PRIMATE NEUROETHOLOGY A B VLPFC ANTERIOR BELT Percent of cells 40 40 30 30 20 20 10 10 0 0 1 2 3 4 5 6 7+ 1 2 3 4 5 6 Number of vocalizationsFigure 25.6 (A) Selectivity of single-unit responses to vocalizations in the ventrolateral prefrontal cortex(VLPFC). Single neurons in the VLPFC responded to vocalizations when presented to awake fixatingmonkeys with varying selectivity. A large percentage of cells (49% of 124 cells) responded to just one ortwo vocalization types when tested with exemplars from 10 vocalization categories. This is similar to theselectivity in AL of the auditory belt shown in (B). The majority of neurons in AL also responded to only oneto two vocalizations when tested with six different vocalization types. Modified from Romanski, L. M.,Averbeck, B. B., & Diltz, M. (2005). Neural representation of vocalizations in the primate ventrolateralprefrontal cortex. Journal of Neurophysiology, 93, 734–747; and Tian, B., Reser, D., Durham, A., Kustov, A.,& Rauschecker, J. P. (2001). Functional specialization in rhesus monkey auditory cortex. Science, 292(5515),290–293.individual calls belong to a particular acoustic combination of complex acoustic featurescategory (Averbeck & Romanski, 2006); that is, and relevant behavioral or referential factorsprefrontal neurons respond in a similar manner simultaneously. This is the case for other puta-to vocalizations from the same or acoustically tive behavioral functions of the prefrontalsimilar categories. cortex (see, for example Nieder, this volume) While data has emerged that is suggestive of and for other association cortices that are con-semantic encoding by the VLPFC at the single nected to prefrontal cortex.neuron level (Gifford et al., 2005), it seems The anatomical and physiological evidenceimportant to distinguish semantic encoding for convergence of auditory, visual, and multi-from responses to acoustic features. Thus, sensory responses in areas such as the STS andneurons that respond well to vocalizations auditory cortex naturally leads to the idea thatfrom a similar semantic context category the PFC should be multisensory as well. Earliershould not respond to vocalizations or syn- neurophysiological studies noted multisensorythetic complex sounds that have a similar responses to simple stimuli in restricted partsacoustic morphology but differ in semantic of dorsolateral prefrontal cortex in highlycontext. Such an experiment has not yet been trained animals (Bodner et al., 1996; Fusterperformed, but would be the key to deter- et al., 2000) and in the orbitofrontal cortex ofmining the role of VLPFC in semantic proces- anesthetized monkeys (Benevento et al., 1977).sing at the single neuron level. In considering In the VLPFC, studies revealed complex audi-all of the anatomical data showing the cascade tory (Romanski & Goldman-Rakic, 2002), visualof acoustic afferents reaching this region, (Hoshi et al., 2000; Pigarev et al., 1979;which include multiple regions of auditory O’Scalaidhe et al., 1997, 1999; Rosenkilde et al.,cortex, and the physiological and imaging 1981; Wilson et al., 1993), and somatosensorydata that demonstrate a role in semantic pro- (Romo et al., 1999) responsive neurons. Thecessing for the prefrontal region, it is likely integration of faces and voices may be morethat prefrontal auditory neurons encode a likely to occur in the VLPFC where there are

MULTISENSORY VOCAL COMMUNICATION 515A Submissive scream Copulation scream Gecker Bark Girney33.0 sp/s 0 1000 0 1000 0 1000 0 1000 0 1000 Harmonic arch Warble Coo Grunt Shrill bark 0 1000 0 1000 0 1000 0 1000 0 1000B C Submissive scream Copulation scream 8 25K 25K Frequency (Hz) 6 4 2 0 0 0 0 0.580 0 0.250 Coo Bark Gecker Girney Harmonic arch Grunt Warble Shrill bark Cop scream Scream Time (s)Figure 25.7 Response of a ventrolateral prefrontal cortex (VLPFC) single neuron to species-specificvocalizations. In (A), the neural activity of a single neuron in the VLPFC is displayed in a raster/spike-density plot for each one of 10 vocalization exemplars. The cell shown has a best response to two screamvocalizations (first two panels in A). The waveforms for each vocalization are shown below the rasterresponse. In (B), the spectrograms for two of these sounds show that they can share some similar acousticfeatures including F0 at %4,000 Hz. A hierarchical cluster analysis of the mean firing rate to each of thevocalizations resulted in the dendrogram shown in (C), where similar responses cluster together. For thisand other neurons in VLPFC, sounds with similar complex features evoked similar neuronal responses inVLPFC and clustered together. Lines are drawn around the dominant groupings in the dendrogram and thesame line patterns are drawn below the neuronal response for these vocalizations.overlapping domains for both auditory stimuli movies were separated into audio and video(particularly, vocalizations; Romanski & streams and neural response to the unimodalGoldman-Rakic, 2002) and visual stimuli (par- stimuli were compared to combined audiovisualticularly, faces; O’Scalaidhe et al., 1997) responses. This experimental paradigm is similar(Fig. 25.8). To test this idea, rhesus monkeys to that used in the studies of STS and auditorywere presented with movies of familiar monkeys cortex (Barraclough et al., 2005; Ghazanfar et al.,vocalizing while single neurons were recorded 2005, 2008). Approximately half of thefrom the VLPFC (Sugihara et al., 2006). These neurons recorded in the VLPFC were bimodal

516 PRIMATE NEUROETHOLOGY AS 8a Opt PS CM PG 9 46 CL Tpt MT A1 RM 45 ML O 10 R TE CP M 12 RT AL RT 12 o CiS FST P RTL TO RP a PG TAa Auditory-responsive LS IOS region IT a Visual/face- TE a IP responsive region STSFigure 25.8 A color-coded schematic diagram of the temporal-prefrontal connectivity in the macaquebrain. Auditory cortical areas in the temporal lobe are colored yellow and the core, belt, and parabeltareas are labeled. The visually responsive areas within the inferotemporal cortex are colored orange,while multisensory regions within the superior temporal sulcus (STS) are colored red to indicate bothauditory and visual responses. Auditory, visual, and multisensory cortical regions send projections tothe ventral prefrontal cortex where they combine to form the basis of multisensory and unimodalprocessing. Studies have previously described auditory and visual/face responses in areas 12 and 45,respectfully. Areas 8a, 9, 10, 12, 45, and 46 are cytoarchitectonic subdivisions of the prefrontal cortex(Preuss & Goldman-Rakic, 1991). A1, primary auditory cortex; AS, arcuate sulcus; CiS, circular sulcus;CL, caudal lateral belt; CM, caudal medial belt; CP, caudal parabelt; IT, inferotemporal cortex; IOS,intermediate orbital sulcus; LS, lateral sulcus; ML, middle part of the lateral belt; PGa, parietalassociated area in the STS; R, rostral core auditory cortex; RM, rostral medial belt auditory cortex;RP, rostral parabelt auditory cortex; RT, rostral temporal auditory cortex; RTL, rostral temporal lateralbelt auditory cortex; RTM, rostral temporal medial belt auditory cortex; STS, superior temporal sulcus;TAa, temporal lobe; Tpt, temporal-parietal association auditory cortex; TAa, anterior temporal area inSTS; TPO, temporal parieto-occipital area in STS.in the sense that they responded to both unim- responses more frequently than nonface/non-odal auditory and visual stimuli or responded voice combinations, as in auditory cortexdifferently to bimodal stimuli than to either (Ghazanfar et al., 2008) and in the STSunimodal stimuli (Sugihara et al., 2006). As in (Barraclough et al., 2005). This adds support tothe STS and auditory cortex, prefrontal neurons the notion that VLPFC is part of a circuit thatexhibited enhancement or suppression, and, like may be specialized for integrating face andthe STS but unlike the auditory cortex, suppres- voices. The selectivity of some neurons in thission (73% of neurons) was more commonly prefrontal region is in accord with fMRI studiesobserved than enhancement (27% of neurons) indicating that a putatively homologous region(Fig. 25.9). of the human brain, area 47 (pars orbitalis), is Responses varied according to the stimulus more activated by human vocal sounds com-exemplar used so that a given cell might show pared with animal and nonvocal soundsmultisensory suppression with one pair of con- (Fecteau et al., 2005). It is unclear, however, togruent faces and voices and an enhancement what extent the VLPFC, the STS, and/or thewith a different pair. It was also interesting that auditory cortex are ‘‘specialized’’ for processingface/voice stimuli evoked multisensory faces and voices. For example, the STS appears to

MULTISENSORY VOCAL COMMUNICATION 517 A Multisensory enhancement Aud Vis AV 120 30 Spikes/s 0 –250 0 1000 1250 0 A V AV Multisensory suppression B Aud Vis AV 35 15 Spikes/s 0 –250 0 1000 1250 0 A V AV C as ps ls Auditory-responsive sts region Visual/face- responsive regionFigure 25.9 Multisensory integration in single neurons of the ventrolateral prefrontal cortex (VLPFC).(A) An example of a neuron that exhibited multisensory enhancement, where the response to the combinedaudiovisual stimuli is greater than the response to either unimodal stimuli. (B) A cell that showedmultisensory suppression is shown. Here the response to the combined audiovisual stimulus is less thanthe response to the unimodal stimuli. The neural response is shown as rasters/spike-density plots on the leftand as the average spike rate in the bar graph on the right in A and B. A schematic of the macaque brainshowing the location of audiovisual responsive cells in the VLPFC is shown in (C). The cells were scatteredthroughout previously identified auditory- and visual-responsive regions.be sensitive to biological motion in general, and CONCLUSIONSnot just dynamic faces (Barraclough et al., 2005;Oram & Perrett, 1994), and the multisensory Much of this line of comparative work is in itsresponses to faces and voices in the auditory infancy, and we are limited in our overview bycortex may be a product of cortical inputs from the paucity of studies across different species—the prefrontal cortex and the STS (Ghazanfar et most of this work has used macaque monkeys asal., 2005, 2008). Thus, while each cortical node model systems together with a few behavioralin a sensory integration network may contribute studies with chimpanzees. The neural processesuniquely to the processing of multisensory com- underlying audiovisual integration of commu-munication stimuli, the overall process is likely nication signals appear to be mediated by net-to be an emergent property involving multiple works embedded within the frontal andnodes. temporal lobes, possibly homologous with

518 PRIMATE NEUROETHOLOGYthose networks mediating audiovisual speech difficult to apply with primates using tradi-perception in humans. It will be important to tional methodologies. Use of modified pre-extend these findings to other species to know ferred looking paradigms and other methodsfor certain. There are other possible comparative may assist in the training of these difficult tasksstudies of great interest. For example, it may be and allow for the recording of neural signalsthat arboreal monkeys or apes, who spend much during the process of behavioral-level audio-of their time out of visual contact with conspe- visual integration. Another important goal iscifics, will exhibit a lesser degree of multisensory creating more realistic environments for bothface/voice processing. It is not known whether behavioral and neurophysiological experi-face/voice multisensory processes are exhibited ments. Communication is an action thatby prosimians, which do not have an extensive involves, at a minimum, a pair of individuals.suite of facial expressions and rely on olfactory This is not captured by the paradigms cur-cues to a greater degree than other primates. A rently in use in laboratory environments.related question would be the degree to which Alternative approaches may come in the formprimates can match the correspondences of interactive playbacks, the use of telemetrybetween heterospecific faces and voices and the (Grohrock et al., 1997; Obeid et al., 2004a,b),degree to which this ability is dependent upon and simulating social exchanges using artificialexperience. For example, a recent study in agents, much like the talking-head animationshuman infants suggest that species-typical used by speech researchers (Massaro &experience narrows the perceptual ability to Bosseler, 2006).match faces and voices to the subjects’ own spe-cies (Lewkowicz & Ghazanfar, 2006); this devel-opmental shift in face/voice perception is not seen REFERENCESin infant vervet monkeys (Zangehenpour et al.,2009). One aspect of multisensory processing that Arbib, M. A. (2005). From monkey-like actionis absent from the behavioral studies reported in recognition to human language: An evolutionarythis review is an abundance of evidence for inte- framework for neurolinguistics. Behavioral andgration. To date, there is no evidence that pri- Brain Sciences, 28, 105–167.mates have anything like the ‘‘McGurk effect,’’ Averbeck, B. B., & Romanski, L. M. (2004). Principal and independent components ofand only a few studies have demonstrated an macaque vocalizations: Constructing stimulienhancement in behavioral performance with to probe high-level sensory processing. Journalbimodal versus unimodal communication signals of Neurophysiology, 91, 2897–2909.(see increased accuracy in chimpanzees: Parr, Averbeck. B. B., & Romanski, L. M. (2006).2004). Thus, there is a strong need to demonstrate Probabilistic encoding of vocalizations in maca-that, beyond just matching, primates use multi- que ventral lateral prefrontal cortex. Journal ofsensory cues to enhance detection, discrimina- Neuroscience, 26, 11023–11033.tion, and/or learning of communication signals. Baizer, J. S., Ungerleider, L. G., & Desimone, R. In concert with these field studies is the need (1991). Organization of visual inputs to theto use more sophisticated behavioral paradigms inferior temporal and posterior parietal cortexduring neurophysiology experiments. Studies of in macaques. Journal of Neuroscience, 11, 168–190.the auditory cortex, the STS, and the prefrontal Barbas, H. (1988). Anatomic organization ofcortex reveal that neurons in these structures basoventral and mediodorsal visual recipientenhance or suppress their firing in responses to prefrontal regions in the rhesus-monkey.audiovisual stimuli versus unimodal presenta- Journal of Comparative Neurology, 276,tions, but how these responses relate to behavior 313–342.is unknown, and we are left only with the sugges- Barbas, H., & Mesulam, M. M. (1981). Organizationtion that there may be a connection to behavior. of afferent input to subdivisions of area-8 in theUnfortunately, tasks that require behavioral rhesus-monkey. Journal of Comparativeintegration of auditory and visual signals are Neurology, 200, 407–431.

MULTISENSORY VOCAL COMMUNICATION 519Barbas, H., & Pandya, D. N. (1989). Architecture Buzsaki, G. (2006). Rhythms of the brain. New York: and intrinsic connections of the prefrontal Oxford University Press. cortex in the rhesus-monkey. Journal of Calvert, G. A. (2001). Crossmodal processing in the Comparative Neurology, 286, 353–375. human brain: Insights from functionalBarnes, C. L., & Pandya, D. N. (1992). Efferent neuroimaging studies. Cerebral Cortex, 11, cortical connections of multimodal cortex of 1110–1123. the superior temporal sulcus in the rhesus- Chandrasekaran, C., & Ghazanfar, A. A. (2009). monkey. Journal of Comparative Neurology, Different neural frequency bands integrate 318, 222–244. faces and voices differently in the superiorBarraclough, N. E., Xiao, D. K., Baker, C. I., Oram, temporal sulcus. Journal of Neurophysiology, M. W., & Perrett, D. I (2005). Integration of 101, 773–788. visual and auditory information by superior Chavis, D. A., & Pandya, D. N. (1976). Further temporal sulcus neurons responsive to the observations on corticofrontal connections in sight of actions. Journal of Cognitive rhesus-monkey. Brain Research, 117, 369–386. Neuroscience, 17, 377–391. Cheney, D. L., & Seyfarth, R. M. (1982). HowBarth, H., Kanwisher, N., & Spelke, E. S. (2003). The vervet monkeys perceive their grunts - Field construction of large number representations in playback experiments. Animal Behaviour, 30, adults. Cognition, 86,201–221. 739–751.Bauer, H. R. (1987). Frequency code: Orofacial Cohen, Y. E., Theunissen, F., Russ, B. E., & Gill, P. correlates of fundamental frequency. Phonetica, (2007). Acoustic features of rhesus vocalizations 44, 173–191. and their representation in the ventrolateralBenevento, L. A., Fallon, J., Davis, B. J., & Rezak, M. prefrontal cortex. Journal of Neurophysiology, (1977). Auditory-visual interactions in single 97, 1470–1484. cells in the cortex of the superior temporal Demb, J. B., Desmond, J. E., Wagner, A. D., sulcus and the orbital frontal cortex of the Vaidya, C. J., Glover, G. H., & Gabrieli, J. D. macaque monkey. Experimental Neurology, 57, E. (1995). Semantic encoding and retrieval in 849–872. the left inferior prefrontal cortex - A functionalBernstein, L. E., Auer, E. T., & Takayanagi, S. (2004). MRI study of task-difficulty and process Auditory speech detection in noise enhanced by specificity. Journal of Neuroscience, 15, lipreading. Speech Communication, 44, 5–18. 5870–5878.Besle, J., Fort, A., Delpuech, C., & Giard, M. H. Evans, T. A., Howell, S., & Westergaard, G. C. (2005). (2004). Bimodal speech: Early suppressive Auditory-visual cross-modal perception of visual effects in human auditory cortex. communicative stimuli in tufted capuchin European Journal of Neuroscience, 20, 2225–2234. monkeys (Cebus apella). Journal of Experi-Bodner, M., Kroger, J., & Fuster, J. M. (1996). mental Psychology: Animal Behavior Processes, 31, Auditory memory cells in dorsolateral 399–406. prefrontal cortex. Neuroreport, 7, 1905–1908. Fecteau, S., Armony, J. L., Joanette, Y., & Belin, P.Boye, M., Gunturkun, O., & Vauclair, J. (2005). (2005). Sensitivity to voice in human prefrontal Right ear advantage for conspecific calls in cortex. Journal of Neurophysiology, 94, adults and subadults, but not infants, California 2251–2254. sea lions (Zalophus californianus): Hemispheric Fitch, W. T. (1997). Vocal tract length and formant specialization for communication? European frequency dispersion correlate with body size in Journal of Neuroscience, 21, 1727–1732. rhesus macaques. Journal of the AcousticalBrokx, J. P., & Nooteboom, S. G. (1982). Intonation Society of America, 102, 1213–1222. and the perceptual separation of simultaneous Fitch, W. T., & Hauser, M. D. (1995). Vocal voices. Journal of Phonetics, 10, 23–36. production in nonhuman-primates - Acoustics,Bruce, C., Desimone, R., & Gross, C. G. (1981). physiology, and functional constraints on honest Visual properties of neurons in a polysensory advertisement. American Journal of Primatology, area in superior temporal sulcus of the macaque. 37, 191–219. Journal of Neurophysiology, 46, 369–384. Fitch, W. T., & Fritz, J. B. (2006). Rhesus macaquesBullier, J., Schall, J. D., & Morel, A. (1996). Functional spontaneously perceive formants in conspecific streams in occipito-frontal connections in the vocalizations. Journal of the Acoustical Society of monkey. Behavioural Brain Research, 76, 89–97. America, 120, 2132–2141.

520 PRIMATE NEUROETHOLOGYFowler, C. A. (2004). Speech as a supramodal Ghazanfar, A. A., & Schroeder, C. E. (2006). Is or amodal phenomenon. In: G. A. Calvert, neocortex essentially multisensory? Trends in C. Spence, & B. E. Stein (Eds.), The handbook of Cognitive Sciences, 10, 278–285. multisensory processes (pp. 189–201). Cambridge, Ghazanfar, A. A., Smith-Rohrberg, D., & Hauser, MA: MIT Press. M. D (2001). The role of temporal cues inFu, K. M. G., Johnston, T. A., Shah, A. S., Arnold, rhesus monkey vocal recognition: Orienting L., Smiley, J., Hackett, T. A., et al. (2003). asymmetries to reversed calls. Brain, Behavior, Auditory cortical neurons respond to and Evolution, 58, 163–172. somatosensory stimulation. Journal of Ghazanfar, A. A., Smith-Rohrberg, D., Pollen, A. A., Neuroscience, 23, 7510–7515. & Hauser, M. D. (2002). Temporal cues in theFuster, J. M., Bodner, M., & Kroger, J. K. (2000). antiphonal long-calling behaviour of cottontop Cross-modal and cross-temporal association tamarins. Animal Behaviour, 64, 427–438. in neurons of frontal cortex. Nature, 405, Ghazanfar, A. A., & Turesson, H. K. (2008). Speech 347–351. production: How does a word feel? CurrentGeschwind, N. (1964). The development of the Biology, 18, R1142–R1144. brain and the evolution of language. In: C. I. J. Ghazanfar, A. A., Turesson, H. K., Maier, J. X., van M. Stuart (Ed.), Report of the 15th Annual Dinther, R., Patterson, R. D., & Logothetis, N. Round Table Meeting on Languages and K. (2007). Vocal tract resonances as indexical Linguistics (pp. 155–169). Washington, DC: cues in rhesus monkeys. Current Biology, 17, Georgetown. 425–430.Ghazanfar, A. A. (2009). The multisensory roles Gifford, G. W., MacLean, K. A., Hauser, M. D., & for auditory cortex in primate vocal comm- Cohen, Y. E. (2005). The neurophysiology of unication. Hearing Research, doi:10.1016/ functionally meaningful categories: Macaque j.heares.2009.04.003. ventrolateral prefrontal cortex plays a criticalGhazanfar, A. A., Chandrasekaran, C., & role in spontaneous categorization of species- Logothetis, N. K. (2008). Interactions between specific vocalizations. Journal of Cognitive the superior temporal sulcus and auditory Neuroscience, 17, 1471–1482. cortex mediate dynamic face/voice integration Grohrock, P., Hausler, U., & Jurgens, U. (1997). in rhesus monkeys. Journal of Neuroscience, 28, Dual-channel telemetry system for recording 4457–4469. vocalization-correlated neuronal activity inGhazanfar, A. A., & Hauser, M. D. (1999). The freely moving squirrel monkeys. Journal of neuroethology of primate vocal communication: Neuroscience Methods, 76, 7–13. Substrates for the evolution of speech. Trends in Hackett, T. A., De La Mothe, L. A., Ulbert, I., Cognitive Sciences, 3, 377–384. Karmos, G., Smiley, J., & Schroeder, C. E.Ghazanfar, A. A., & Hauser, M. D. (2001). The (2007a). Multisensory convergence in auditory auditory behaviour of primates: A cortex, II. Thalamocortical connections of the neuroethological perspective. Current Opinion caudal superior temporal plane. Journal of in Neurobiology, 11, 712–720. Comparative Neurology, 502, 924–952.Ghazanfar, A. A., & Logothetis, N. K. (2003). Facial Hackett, T. A., Smiley, J. F., Ulbert, I., Karmos, G., expressions linked to monkey calls. Nature, 423, Lakatos, P., de la Mothe, L. A., et al. (2007b). 937–938. Sources of somatosensory input to the caudalGhazanfar, A. A., Maier, J. X., Hoffman, K. L., & belt areas of auditory cortex. Perception, 36, Logothetis, N. K. (2005). Multisensory integ- 1419–1430. ration of dynamic faces and voices in rhesus Hackett, T. A., Stepniewska, I., & Kaas, J. H. (1999). monkey auditory cortex. Journal of Neuro- Prefrontal connections of the parabelt auditory science, 25, 5004–5012. cortex in macaque monkeys. Brain Research,Ghazanfar, A. A., & Rendall, D. (2008). Evolution 817, 45–58. of human vocal production. Current Biology, 18, Harries, M. H., & Perrett, D. I. (1991). Visual R457–R460. processing of faces in temporal cortex -Ghazanfar, A. A., & Santos, L. R. (2004). Primate Physiological evidence for a modular brains in the wild: The sensory bases for social organization and possible anatomical interactions. Nature Reviews Neuroscience, 5, correlates. Journal of Cognitive Neuroscience, 3, 603–616. 9–24.

MULTISENSORY VOCAL COMMUNICATION 521Hauser, M. D., Agnetta, B., & Perez, C. (1998). of faces they see. Current Biology, 15, Orienting asymmetries in rhesus monkeys: 1034–1038. The effect of time-domain changes on acoustic Kayser, C., Petkov, C. I., Augath, M., & Logothetis, perception. Animal Behaviour, 56, 41–47. N. K. (2005). Integration of touch and sound inHauser, M. D., Evans, C. S., & Marler, P. (1993). auditory cortex. Neuron, 48, 373–384. The role of articulation in the production of Kayser, C., Petkov, C. I., Augath, M., & Logothetis, rhesus-monkey, Macaca-mulatta, vocaliz- N. K. (2007). Functional imaging reveals visual ations. Animal Behaviour, 45, 423–433. modulation of specific fields in auditory cortex.Hauser, M. D., & Marler, P. (1993). Food- Journal of Neuroscience, 27, 1824–1835. associated calls in rhesus macaques (Macaca- Kayser, C., Petkov, C. I., & Logothetis, N. K. (2008). mulatta). 1. Socioecological Factors. Behavioral Visual modulation of neurons in auditory Ecology, 4, 194–205. cortex. Cerebral Cortex, 18, 1560–1574.Hauser, M. D., & Ybarra, M. S. (1994). The role of Kojima, S. (1990). Comparison of auditory lip configuration in monkey vocalizations - functions in chimpanzee and human. Folia Experiments using xylocaine as a nerve block. Primatologica, 55, 62–72. Brain and Language, 46, 232–244. Kojima, S., & Kiritani, S. (1989). Vocal-auditoryHeinz, R. D., & Brady, J. V. (1988). The acquisition functions in the chimpanzee: Vowel perception. of vowel discriminations by nonhuman International Journal of Primatology, 10, primates. Journal of the Acoustical Society of 199–213. America, 84, 186–194. Kuhl, P. K., & Meltzoff, A. N. (1984). TheHikosaka, K., Iwai, E., Saito, H. A., & Tanaka, K. intermodal representation of speech in infants. (1988). Polysensory properties of neurons in Infant Behavior and Development, 7, 361–381. the anterior bank of the caudal superior Kuhl, P. K., & Miller, J. D. (1975). Speech- temporal sulcus of the macaque monkey. perception by chinchilla - Voiced-voiceless Journal of Neurophysiology, 60, 1615–1637. distinction in alveolar plosive consonants.Hoshi, E., Shima, K., & Tanji J (2000). Neuronal Science, 190, 69–72. activity in the primate prefrontal cortex in the Kuhl, P. K., Williams, K. A., & Meltzoff, A. N. process of motor selection based on two (1991). Cross-modal speech perception in behavioral rules. Journal of Neurophysiology, adults and infants using nonspeech auditory 83, 2355–2373. stimuli. Journal of Experimental Psychology:Ito, S. I. (1982). Prefrontal unit-activity of macaque Human Perception and Performance, 17, monkeys during auditory and visual reaction- 829–840. time tasks. Brain Research, 247, 39–47. Lakatos, P., Chen, C-M., O’Connell, M. N., Mills,Izumi, A. (2002). Auditory stream segregation in A., & Schroeder, C. E. (2007). Neuronal Japanese monkeys. Cognition, 82, B113–B122. oscillations and multisensory interaction inIzumi, A., & Kojima, S. (2004). Matching primary auditory cortex. Neuron, 53, 279–292. vocalizations to vocalizing faces in a Le Prell, C. G., Niemiec, A. J., & Moody, D. B. chimpanzee (Pan troglodytes). Animal (2001). Macaque thresholds for detecting Cognition, 7, 179–184. increases in intensity: Effects of formantJiang, J. T., Alwan, A., Keating, P. A., Auer, E. T., structure. Hearing Research, 162, 29–42. & Bernstein, L. E. (2002). On the relationship Lewkowicz, D. J. (2002). Heterogeneity and between face movements, tongue heterochrony in the development of movements, and speech acoustics. Eurasip intersensory perception. Cognitive Brain Journal on Applied Signal Processing, 2002, Research, 14, 41–63. 1174–1188. Lewkowicz, D. J., & Ghazanfar, A. A. (2006). TheJordan, K. E., & Brannon, E. M. (2006). The decline of cross-species intersensory perception multisensory representation of number in in human infants. Proceedings of the National infancy. Proceedings of the National Academy of Academy of Sciences, 103, 6771–6774. Sciences, 103, 3486–3489. Liberman, A. M., & Mattingly, I. (1985). The motorJordan, K. E., Brannon, E. M., Logothetis, N. K., & theory revised. Cognition, 21, 1–36. Ghazanfar, A. A. (2005). Monkeys match the Liu, J., & Newsome, W. T. (2006). Local field number of voices they hear with the number potential in cortical area MT: Stimulus tuning

522 PRIMATE NEUROETHOLOGY and behavioral correlations. Journal of Oram, M. W., & Perrett, D. I. (1994). Responses of Neuroscience, 26, 7779–7790. anterior superior temporal polysensory (Stpa)Maier, J. X., Chandrasekaran, C., & Ghazanfar, A. neurons to biological motion stimuli. Journal of A. (2008). Integration of bimodal looming Cognitive Neuroscience, 6, 99–116. signals through neuronal coherence in the Owren, M. J., Seyfarth, R. M., & Cheney, D. L. temporal lobe. Current Biology, 18, 963–968. (1997). The acoustic features of vowel-likeMaier, J. X., Neuhoff, J. G., Logothetis, N. K., & grunt calls in chacma baboons (Papio Ghazanfar, A. A. (2004). Multisensory cyncephalus ursinus): Implications for integration of looming signals by Rhesus production processes and functions. Journal of monkeys. Neuron, 43, 177–181. the Acoustical Society of America, 101, 2951–2963.Massaro, D. W., & Bosseler, A. (2006). Read my lips Palombit, R. A., Cheney, D. L., & Seyfarth, R. M. - The importance of the face in a computer- (1999). Male grunts as mediators of social animated tutor for vocabulary learning by interaction with females in wild chacma children with autism. Autism, 10, 495–510. baboons (Papio cynocephalus ursinus).McGurk, H., & MacDonald, J. (1976). Hearing lips Behaviour, 136, 221–242. and seeing voices. Nature, 264, 229–239. Pandya, D. N., Hallett, M., & Mukherje, S. (1969).Meltzoff, A. N., & Moore, M. (1997). Explaining Intra-hemispheric and interhemispheric facial imitation: A theoretical model. Early connections of neocortical auditory system in Development and Parenting, 6, 179–192. rhesus monkey. Brain Research, 14, 49.Miller, C. T., & Hauser, M. D. (2004). Multiple Pandya, D. N., & Kuypers, H. G. J. (1969). Cortico- acoustic features underlie vocal signal cortical connections in rhesus monkey. Brain recognition in tamarins: Antiphonal calling Research, 13, 13. experiments. Journal of Comparative Physiology Parr, L. A. (2004). Perceptual biases for multimodal A: Neuroethology Sensory Neural and Behavioral cues in chimpanzee (Pan troglodytes) affect Physiology, 190, 7–19. recognition. Animal Cognition, 7, 171–178.Nasir, S. M., & Ostry, D. J. (2006). Somatosensory Partan, S. R. (2002). Single and multichannel signal precision in speech production. Current composition: Facial expressions and Biology, 16, 1918–1923. vocalizations of rhesus macaques (MacacaNoesselt, T., Rieger, J. W., Schoenfeld, M. A., mulatta). Behaviour, 139, 993–1027. Kanowski, M., Hinrichs, H., Heinze, H-J., Patterson, M. L., & Werker, J. F. (2003). Two- et al. (2007). Audiovisual temporal corres- month-old infants match phonetic pondence modulates human multisensory information in lips and voice. Developmental superior temporal sulcus plus primary sensory Science, 6, 191–196. cortices. Journal of Neuroscience, 27, 11431– Petersen, M. R., Beecher, M. D., Zoloth, S. R., 11441. Moody, D. B., & Stebbins, W. C. (1978).O’Scalaidhe, S. P., Wilson, F. A. W., & Goldman- Neural lateralization of species-specific Rakic, P. S. (1997). Areal segregation of face- vocalizations by Japanese macaques (Macaca- processing neurons in prefrontal cortex. fuscata). Science, 202, 324–327. Science, 278, 1135–1138. Petrides, M., & Pandya, D. N. (1988). AssociationO’Scalaidhe, S. P., Wilson, F. A. W., & Goldman- fiber pathways to the frontal-cortex from the Rakic, P. S. (1999). Face-selective neurons superior temporal region in the rhesus- during passive viewing and working memory monkey. Journal of Comparative Neurology, performance of Rhesus monkeys: Evidence for 273, 52–66. intrinsic specialization of neuronal coding. Petrides, M., & Pandya, D. N. (2002). Comparative Cerebral Cortex, 9, 459–475. cytoarchitectonic analysis of the human and theObeid, I., Nicolelis, M. A. L., & Wolf, P. D. (2004a). macaque ventrolateral prefrontal cortex and A multichannel telemetry system for single unit corticocortical connection patterns in the neural recordings. Journal of Neuroscience monkey. European Journal of Neuroscience, 16, Methods, 133, 33–38. 291–310.Obeid, I., Nicolelis, M. A. L., & Wolf, P. D. (2004b). Pigarev, I. N., Rizzolatti, G., & Scandolara, C. A low power multichannel analog front end for (1979). Neurons rsponding to visual stimuli in portable neural signal recordings. Journal of the frontal lobe of macaque monkeys. Neuroscience Methods, 133, 27–32. Neuroscience Letters, 12, 207–212.

MULTISENSORY VOCAL COMMUNICATION 523Pinker, S. (1994). The language instinct. New York: Rowell, T. E., & Hinde, R. A. (1962). Vocal William Morrow and Company, Inc. communication by the rhesus monkeyPoldrack, R. A., Wagner, A. D., Prull, M. W., (Macaca mulatta). Proceedings of the Zoological Desmond, J. E., Glover, G. H., & Gabrieli, J. D. Society London, 138, 279–294. E. (1999). Functional specialization for Saleem, K. S., Suzuki, W., Tanaka, K., & semantic and phonological processing in the Hashikawa, T. (2000). Connections between left inferior prefrontal cortex. Neuroimage, 10, anterior inferotemporal cortex and superior 15–35. temporal sulcus regions in the macaqueRauschecker, J. P., Tian, B., & Hauser, M. (1995). monkey. Journal of Neuroscience, 20, Processing of complex sounds in the macaque 5083–5101. nonprimary auditory cortex. Science, 268, Schroeder, C. E., & Foxe, J. J. (2002). The timing 111–114. and laminar profile of converging inputs toRendall, D., Owren, M. J., & Rodman, P. S. (1998). multisensory areas of the macaque neocortex. The role of vocal tract filtering in identity Cognitive Brain Research, 14, 187–198. cueing in rhesus monkey (Macaca mulatta). Schwartz, J-L., Berthommier, F., & Savariaux, C. vocalizations. Journal of the Acoustical Society (2004). Seeing to hear better: Evidence for early of America, 103, 602–614. audio-visual interactions in speechRickert, J., de Oliveira, S. C., Vaadia, E., Aertsen, A., identification. Cognition, 93, B69–B78. Rotter, S., & Mehring, C. (2005). Encoding of Seltzer, B., Cola, M. G., Gutierrez, C., Massee, M., movement direction in different frequency Weldon, C., & Cusick, C. G. (1996). ranges of motor cortical local field potentials. Overlapping and nonoverlapping cortical Journal of Neuroscience, 25, 8815–8824. projections to cortex of the superior temporalRomanski, L. M., Averbeck, B. B., & Diltz, M. sulcus in the rhesus monkey: Double (2005). Neural representation of vocalizations anterograde tracer studies. Journal of in the primate ventrolateral prefrontal cortex. Comparative Neurology, 370, 173–190. Journal of Neurophysiology, 93, 734–747. Seltzer, B., & Pandya, D. N. (1978). AfferentRomanski, L. M., Bates, J. F., & Goldman-Rakic, P. cortical connections and architectonics of S. (1999a). Auditory belt and parabelt superior temporal sulcus and surrounding projections to the prefrontal cortex in the cortex in rhesus-monkey. Brain Research, 149, rhesus monkey. Journal of Comparative 1–24. Neurology, 403, 141–157. Seltzer, B., & Pandya, D. N. (1991). Post-rolandicRomanski, L. M., & Goldman-Rakic, P. S. (2002). cortical projections of the superior temporal An auditory domain in primate prefrontal sulcus in the rhesus-monkey. Journal of cortex. Nature Neuroscience, 5, 15–16. Comparative Neurology, 312, 625–640.Romanski, L. M., Tian, B., Fritz, J., Mishkin, M., Seltzer, B., & Pandya, D. N. (1994). Parietal, Goldman-Rakic, P. S., & Rauschecker, J. P. temporal, and occipital projections to cortex (1999b). Dual streams of auditory afferents of the superior temporal sulcus in the rhesus target multiple domains in the primate monkey: A retrograde tracer study. Journal of prefrontal cortex. Nature Neuroscience, 2, Comparative Neurology, 343, 445–463. 1131–1136. Shannon, R. V., Zeng, G. F., Kamath, V., Wygonski,Romo, R., Brody, C. D., Hernandez, A., & Lemus, J., & Ekelid, M. (1995). Speech recognition with L. (1999). Neuronal correlates of parametric primarily temporal cues. Science, 270, 303–304. working memory in the prefrontal cortex. Sinnott, J. M. (1989). Detection and discrimination Nature, 399, 470–473. of synthetic English vowels by old-worldRosenblum, L. D. (2005). Primacy of multimodal monkeys (Cercopithecus, Macaca) and speech perception. In: D. B. Pisoni & R. E. humans. Journal of the Acoustical Society of Remez (Eds.), Handbook of speech perception America, 86, 557–565. (pp. 51–78). Malden, MA: Blackwell. Sinnott, J. M., Brown, C. H., Malik, W. T., &Rosenkilde, C. E., Bauer, R. H., & Fuster, J. M. Kressley, R. A. (1997). A multidimensional (1981). Single cell activity in ventral prefrontal scaling analysis of vowel discrimination in cortex of behaving monkeys. Brain Research, humans and monkeys. Perception and 209, 375–394. Psychophysics, 59, 1214–1224.

524 PRIMATE NEUROETHOLOGYSmiley, J. F., Hackett, T. A., Ulbert, I., Karmas, G., Vaadia, E., Benson, D. A., Hienz, R. D., & Lakatos, P., Javitt, D. C., et al. (2007). Goldstein, M. H. (1986). Unit study of Multisensory convergence in auditory cortex, monkey frontal fortex - Active localization of I. Cortical connections of the caudal superior auditory and of visual stimuli. Journal of temporal plane in macaque monkeys. Journal of Neurophysiology, 56, 934–952. Comparative Neurology, 502, 894–923. van Wassenhove, V., Grant, K. W., & Poeppel, D.Smith, D. R. R., & Patterson, R. D. (2005). The (2005). Visual speech speeds up the neural interaction of glottal-pulse rate and vocal-tract processing of auditory speech. Proceedings of length in judgement of speaker size, sex and age. the National Academy of Sciences, 102, Journal of the Acoustical Society of America, 118, 1181–1186. 3177–3186. Watanabe, M. (1992). Frontal units of the monkeySommers, M. S., Moody, D. B., Prosen, C. A., & coding the associative significance of visual and Stebbins, W. C. (1992). Formant frequency auditory stimuli. Experimental Brain Research, discrimination by Japanese macaques 89, 233–247. (Macaca-fuscata). Journal of the Acoustical Webster, M. J., Bachevalier, J., & Ungerleider, L. Society of America, 91, 3499–3510. G. (1994). Connections of inferior temporalStein, B. E. (1998). Neural mechanisms for areas TEO and TE with parietal and frontal synthesizing sensory information and cortex in macaque monkeys. Cerebral Cortex, producing adaptive behaviors. Experimental 4, 470–483. Brain Research, 123, 124–135. Wilke, M., Logothetis, N. K., & Leopold, D. A.Sugihara, T., Diltz, M., Averbeck, B. B., & Romanski, (2006). Local field potential reflects perceptual L. M. (2006). Integration of auditory and visual suppression in monkey visual cortex. communication information in the primate Proceedings of the National Academy of ventrolateral prefrontal cortex. Journal of Sciences, 103, 17507–17512. Neuroscience, 26, 11138–11147. Wilson, F. A. W., Scalaidhe, S. P. O., &Summerfield, Q., Culling, J. F., & Fourcin, A. J. Goldmanrakic, P. S. (1993). Dissociation of (1992). Auditory segregation of competing object and spatial processing domains in voices: Absence of effects of FM or AM primate prefrontal cortex. Science, 260, coherence. Philosophical Transactions of the 1955–1958. Royal Society of London Series B: Biological Yehia, H., Rubin, P., & Vatikiotis-Bateson, E. Sciences, 336, 357–366. (1998). Quantitative association of vocal-tractTanila, H., Carlson, S., Linnankoski, I., Lindroos, F., & and facial behavior. Speech Communication, 26, Kahila, H. (1992). Functional properties of 23–43. dorsolateral prefrontal cortical neurons in awake Yehia, H. C., Kuratate, T., & Vatikiotis-Bateson, E. monkey. Behavioural Brain Research, 47, 169–180. (2002). Linking facial animation, head motionThorpe, S.J., Rolls, E.T., & Maddison, S. (1983). and speech acoustics. Journal of Phonetics, 30, The orbitofrontal cortex: neuronal activity in 555–568. the behaving monkey. Experimental Brain Zangehenpour, S., Ghazanfar, A. A., Research, 49, 93–115. Lewkowicz, D. J., & Zatorre, R. J. (2009).Trout, J. D. (2001). The biological basis of speech: Heterochrony and cross-species intersensory What to infer from talking to the animals. matching by infant vervet monkeys. PLoS Psychological Review, 108, 523–549. ONE, 4, e4302.

CHAPTER 26Neuroethology of Attention in PrimatesStephen V. Shepherd and Michael L. Platt social cognition have great difficulty navigating Expert Commission Decides That the Horse the overwhelmingly complex societies in which Actually Reasons. we live, and by which we thrive. As Clever Hans BERLIN, Sept. 13. — The remarkable horse demonstrated, the cognitive mechanisms under- called ‘‘Clever Hans’’ has just been examined by lying our shared humanity appear to also be a special commission of experts, in order that a shared with many other species. It thus seems decision might be arrived at whether it is a horse possessed of extraordinary brain power . . . likely that advances in the evolution of cognition derive, at least in part, from the need to predict —From The London Standard, in The New York the actions of others. These social cognitive abil- Times, October 2, 1904 ities are built on a long genetic and developmental history, but are founded on something so simpleThe case of ‘‘Clever Hans’’ is perhaps the most and fundamental that we often fail to notice it. Tofamous con job in the history of psychology. learn how to make sense of another’s actions—Both experts and public were taken in, believing whether human, monkey, or horse—we have tothat human-like perceptual and reasoning abil- attend to them. In the following pages, we exploreities could be exhibited by a horse trained by this social orienting behavior, describe its appa-Wilhelm van Osten. After 3 years and a great rent adaptive functions, and attempt to under-deal of public scrutiny, however, Oskar Pfungst stand how orienting decisions are made withindetermined that the horse could only answer the brain.questions to which the experimenters alreadyknew the answer—in short, the horse did nothave these human abilities, and was merely PSYCHOLOGICAL,responding to subtle social cues provided by ETHOLOGICAL, ANDhis audience. Thus, the horse’s exceptional NEUROETHOLOGICALbrainpower was dismissed. APPROACHES TO ATTENTION But this is the wrong lesson of this story. Afterall, the con artist was not Wilhelm van Osten—he All mobile animals orient to salient features ofwas taken in as well—but his horse. A better lesson their environment. This can occur overtly, byis that the horse, like many animals, is exquisitely shifting gaze, or covertly, by deploying attentioncapable of learning to recognize subtle cues to the without eye movements. Psychophysical, electro-emotions, intentions, expectations, and knowl- physiological, and neuroimaging studies con-edge not only of his own species, but even ducted in the laboratory have extensively probedhumans. These abilities are no less impressive orienting in both human and nonhuman pri-because they are social, rather than abstract. mates. Generally, subjects have been trained toIndeed, humans with selective disruption of discriminate simple stimuli whose salience or 525

526 PRIMATE NEUROETHOLOGYbehavioral significance has been arbitrarily assig- ethological and laboratory approaches willned through explicit instruction or association foster the development of a unified evolutiona-with rewards (e.g., Posner, 1980). Such studies rily motivated theory of attention, which willsuggest the operation of two distinct systems for have broad impact on our understanding oforienting attention (James, 1890; Jonides, 1981; brain systems. For many animals, in particularPosner & Cohen, 1984), one fast and reflexive primates like ourselves, one of the most impor-(exogenous) and the other slow and voluntary tant variables influencing attentional deploy-(endogenous), mediated by partially overlapping ment is the current behavioral state of nearbyneural circuits (Corbetta & Shulman, 2002; Egeth individuals—the current social context. In the& Yantis, 1997; Mangun, 1995). following sections, we consider the impact of Outside the laboratory, observational studies social context on attention, outlining some ofin natural settings indicate that social stimuli are what has been learned from both laboratory andintrinsically salient and strongly attract attention field studies. In particular, we describe our own(Caine & Marra, 1988; Keverne et al., 1978; efforts to bridge these approaches by studyingMcNelis & Boatright-Horowitz, 1998). More- the neuroethology of social attention in humanover, recent laboratory studies have suggested and nonhuman primates.that social cues, such as the direction of gaze inan observed individual, access a privileged infor- EVOLUTION OF VISUALmation channel capable of directing attention SPECIALIZATIONS IN PRIMATES(Deaner & Platt, 2003; Driver et al., 1999;Friesen & Kingstone, 1998). These studies imply Primates are unusual among mammals in theirthat the neural mechanisms that mediate atten- strong reliance on vision (Allman, 1999).tion have evolved sensitivity to cues predicting Initially, visual specializations probably evolvedthe goals and intentions of other individuals— in primates to support movement throughbut the precise nature of these social cues, and the upper tree branches (Robert Martin’s ‘‘fine-specific neural systems by which they are pro- branch niche hypothesis,’’ Martin, 1990), tocessed, remain obscure. Indeed, current evidence facilitate hunting for insects (Matt Cartmill’ssuggests that social attention maps poorly onto ‘‘visual predator’’ hypothesis, Cartmill, 1972),existing models of attention, which emphasize or both. Primates might thus be expected todichotomous exogenous and endogenous use vision primarily for locomotion and foodorienting systems. acquisition while retaining common mamma- The neuroethological approach is an alterna- lian visual functions such as predator avoidance.tive paradigm that works to resolve these issues. Over the course of primate evolution, how-We contend that complete understanding of the ever, visual processing appears to have becomebiology of attention must account not only for increasingly specialized for guiding social inter-gross patterns of orienting in natural environ- action. Many primates make extensive use ofments but also for the fine spatiotemporal details vision to localize, monitor, and interact withof orienting measured in controlled laboratory other individuals, and likewise devote a largesettings. These ethological and psychophysical portion of their brains to visual processing.goals are often approached separately, using dif- Notably, the expansion of the primate brainferent animal models and highly divergent tech- has been accompanied by a correspondingniques, reflecting in part the fact that the increase in the flexibility and complexity of pri-demands of naturalistic observation generally mate social groups (Allman, 1999; Barton &preclude precise measurements of orienting. Dunbar, 1997). While prosimian primates relyLikewise, psychophysical experiments have typi- heavily on olfactory and pheromone-mediatedcally failed to simulate the behavioral contexts in modes of communication, these ancestral sen-which orienting behavior normally operates. sory modalities have been supplanted in more Nonetheless, we contend that these divisi- derived primates by visually mediated signalsons are surmountable, and that combining such as coloration, poise, gesture, facial

NEUROETHOLOGY OF ATTENTION IN PRIMATES 527expression, and gaze (de Waal, 2003; note alsoGilad et al., 2004), as well as affective and refer-ential vocalizations (Cheney & Seyfarth, 1990;Seyfarth & Cheney, 2003). While scientists havelong recognized the importance of studying pri-mate visual attention in the laboratory, we haveall too often neglected the role of attention in thenatural social ecology.BEHAVIORAL GOALS DRIVEORIENTING IN NATURALISTICSETTINGSThe first studies of naturalistic orienting in pri-mates were conducted in humans by the Russianpsychologist Alfred Yarbus in the 1950s and1960s (Yarbus, 1967). He measured overt visualorienting behavior by recording visual fixationpatterns during free and instructed scanning ofpictures with light-reflecting mirrors suction-cupped to the eyes. Yarbus demonstrated thatsocial stimuli are intrinsically salient and Figure 26.1 Social context and behavioral goalsstrongly attract attention. Despite this strong alter fixation patterns during free viewing. Panelsbias, current behavioral goals also regulate A–H show the different gaze patterns of viewersvisual attention. For example, when shown the when asked different questions about thepainting ‘‘An Unexpected Visitor,’’ subjects con- illustration, Rjepin’s ‘‘Unexpected Visitor,’’ shownsistently oriented attention toward the faces of at upper left. Viewers scanned the photographs inpeople in the scene (Fig. 26.1). When asked to very different ways when asked, e.g., to estimate thedetermine the wealth of the family in the picture, family’s wealth (C), estimate their ages (D),however, subjects shifted their gaze to the memorize the position of people and objects (G),clothing worn by the figures in order to extract or estimate how long the ‘‘unexpected visitor’’ had been away (H). After Yarbus, A. (1967). Eyethe information requested by the experimenter. movements during perception of complex objects. Recently, Land and Hayhoe (2001) reported In: Eye movements and vision (pp. 171–211).similar context dependence using noninvasive New York: Plenum Press. Used with permission.video gaze tracking in human subjects. Theyshowed that attention was almost completelydetermined by task demands during simple niche supplied by ethology, but also the rigorousactions such as making a sandwich or preparing mathematical tool set for understanding deci-tea. By contrast, subjects almost never attended to sion making provided by behavioral economics.task-irrelevant regions of space. These data sug- The marriage of the latter fields with electrophy-gest that attentional priorities not only are shaped siology and functional imaging has produced anby evolutionary pressures but also can serve as explosive change in our approach to humanexternal indicators of the shifting internal goals decisions through the field of neuroeconomics,governing an individual’s moment-to-moment and neuroethology holds similar promise forbehavior (Shepherd & Platt, 2008). extending our mechanistic understanding of To make sense of natural orienting patterns, behavioral evolution. A central message of boththen, requires not just the sophisticated under- these disciplines, however, has been that neuralstanding of behavioral repertoire and ecological processes are strongly influenced by social

528 PRIMATE NEUROETHOLOGYvariables, ranging from the framing effects McNelis & Boatright-Horowitz, 1998). Suchinvoked by word choice in task instructions data have generally been limited, however, to(De Martino et al., 2006; Tversky & Kahneman, distal observations in natural settings or, in the1981) to the intrinsic reward of watching other laboratory, to qualitative analysis of fixation pat-individuals (Aharon et al., 2001; Hayden et al., terns within still photographs.2007). Given the various limitations of previous studies, one goal of our research has been to quantitatively measure primate visual attention inSOCIAL ATTENTION IN NATURAL naturalistic environments and species-typicalSETTINGS social groups. To do this, we recorded gazeObservational data support the idea that visual behavior in socially housed, freely moving ring-attention in nonhuman primates is also biased tailed lemurs (Lemur catta) interacting in largetoward social stimuli (Caine & Marra, 1988; three-dimensional environments. We used aKeverne et al., 1978; McNelis & Boatright- lightweight telemetric optical gaze-trackingHorowitz, 1998). Furthermore, these biases are device (Fig. 26.2; see Shepherd & Platt, 2006)not uniform; some social stimuli attract more operating at 0.22 degrees Â 33 ms resolution—attention than others. For example, monkeys a degree of precision comparable to eye-trackingspend more time looking at pictures of faces methods used in the laboratory. Our approachgazing toward them than faces with averted differed, however, in that we did not provide anygaze (Keating & Keating, 1982), and look pre- task or instruction, but instead inferred the goalsferentially toward the regions around the eyes guiding visual attention in natural contexts fromand mouth (Guo et al., 2003; Keating & Keating, the observed patterns of typical behavior1982; Kyes & Candland, 1987). They also look (Shepherd & Platt, 2008). Ring-tailed lemurs,more often toward higher-ranking animals than prosimian primates that diverged from thelower-ranking animals (Keverne et al., 1978; ancestors of ‘‘higher’’ primates some 60 millionFigure 26.2 Equipment for tracking gaze during the natural behavior of freely moving animals. Wetracked gaze during spontaneous and natural interactions with cohabitant conspecifics (A) using atelemetric optical gaze-tracking system developed by Iscan, Inc. The system (B) was composed of aninfrared camera and LED (a) imaging the lemur’s right eye through a dichroic mirror (b), an opticalcamera (c) viewing the scene in front the lemur’s head, and a telemetry system housed in a primate vest (d),which broadcast to a remote monitoring station where the subject’s recorded gaze direction was analyzedand projected onto locations in the recorded visual scene. After Shepherd, S. V., & Platt, M. L. (2006).Noninvasive telemetric gaze tracking in freely moving socially housed prosimian primates. Methods, 38,185–194. Used with permission.

NEUROETHOLOGY OF ATTENTION IN PRIMATES 529Figure 26.3 Fixation priorities in stationary and moving lemurs. (A) Lemurs fixated humans (h), lemurs(l), food (f), and high-contrast environmental features (e) significantly more than chance expectation;furthermore, they fixated humans and lemurs significantly more than food rewards, and all threesignificantly more than environmental features. (B) Lemurs fixated environmental features significantlymore often when moving than when stationary, seemingly at the cost of fixations toward animals andrewards. ***, P <0.0005; **, P <0.001; *, P <0.01. After Shepherd, S. V., & Platt, M. L. (2008). Spontaneoussocial orienting and gaze following in ringtailed lemurs (Lemur catta). Animal Cognition, 11, 13–20. Usedwith permission.years ago, were chosen as subjects based on their periods of active locomotion (Fig. 26.3B). Whiletolerance of handling and their availability at the moving, lemurs attended to environmentalDuke Lemur Center. Ring-tailed lemur social features that served as potential movementgroups are similar to those of many higher pri- substrates along the path toward their destina-mates, comprising 10 to 20 individuals of both tion. At the same time, other visual priorities,sexes organized in well-defined social hierar- such as monitoring other lemurs and foraging,chies, and communicating through auditory, were temporarily but systematically diminished.olfactory, and visual modalities (Jolly, 1966; Together with earlier research (Land & Hayhoe,Sauther et al., 1999). 2001; Yarbus, 1967), these findings validate the We found that male ring-tailed lemurs fixated use of quantitative gaze measurements as antheir human handlers—as would be expected externally observable indicator of otherwisegiven we had just suited them, temporarily, into unobservable mental states (e.g., the currentrecording equipment. More important, they behavioral goals of an animal) and furtherfixated their social companions, and did so reveal that the typical behavioral context for amore often than they fixated small food rewards lemur involves not only monitoring threats,(Fig. 26.3A). Each of these three a priori cate- such as predators, and rewards, such as food,gories—human handlers, conspecifics, and food but also guiding movement and maintainingrewards—were fixated significantly more often observational contact with other members ofthan chance and significantly more often than the social group.high-contrast environmental features, stimuli weexpected to attract attention based on low-levelvisual salience (Carmi & Itti, 2006; Peters et al., DOMINANCE, SEX, AND SOCIAL2005). These data suggest that animals, especially SALIENCEconspecifics, and rewards, such as potential food Our ongoing field studies of orienting in ring-sources, were effectively identified, localized, and tailed lemurs support the idea that ancestralattended during natural visual behavior. primates possessed neural specializations for These social attention biases were not orienting toward and extracting relevant infor-inflexible, however, and in fact changed during mation from other animals (Tomasello et al.,

530 PRIMATE NEUROETHOLOGY2005). The diversity of stimuli and complexity We know, however, that in the wild, monkeysof behavioral contexts that typify the field set- visually monitor one another (Caine & Marra,ting, however, has challenged our ability to 1988; Keverne et al., 1978; McNelis & Boatright-draw definitive conclusions regarding the spe- Horowitz, 1998), and in the laboratory, theycific stimuli that guide visual attention during preferentially seek out visual stimuli with socialnatural social behavior—an endeavor that is content (Butler, 1954; Sackett, 1966). To pre-ongoing in our laboratory. Moreover, despite cisely quantify how rhesus monkeys prioritizethe evident similarity between human visual attending to specific classes of social stimuli, weattention priorities and those observed in developed a choice task designed to balance fluidlemurs, the genomes, brains, behavioral reper- rewards against the chance to glimpse photos oftoires, and social systems of our species differ other monkeys. Specifically, monkeys chosedramatically. Unfortunately, little is known between orienting to either of two targets, oneabout brain function in lemurs. associated with a juice reward and another asso- To address these limitations, we have con- ciated with an alternative juice reward and aducted parallel investigations of the visual picture of a familiar monkey. By determiningorienting behavior of another primate, whose the differential juice reward at which monkeysvisual abilities, social structure, environmental were equally likely to choose to view the image,niche, and physiology more closely mirror our we were able to quantify the reward value ofown. Rhesus macaques (Macaca mulatta) are an attending to different classes of social stimulioft-studied anthropoid primate with relatively (Deaner et al., 2005).well-explored biology, and like humans, they Using this ‘‘pay-per-view’’ paradigm, welive in large, hierarchical social groups with found that male monkeys consistently ‘‘overva-extensive multisensory behavioral interactions. lued’’ seeing potential mating cues (femaleAlthough rhesus monkeys have been widely used hindquarters) and faces of dominant males, butto study visual attention, most past studies have ‘‘undervalued’’ seeing the faces of low-rankingused arbitrary stimuli with little or no intrinsic males (Fig. 26.4). The attraction of attention tobehavioral relevance. high-ranking males is somewhat counterintuitive,Figure 26.4 Monkeys sacrifice juice to view important social stimuli. When monkeys were offereddifferent juice rewards to fixate two targets, only one of which also yielded an image reward, they choseeach option equally when the intrinsic value of viewing an image offset the amount of juice sacrificed (A).Monkeys paid the highest amount of juice to see female perinea (P) and a lower amount to see high-rankingmale faces (HF), but required extra juice to look at low-ranking male faces (LF) or, to a lesser extent, uniformgray squares (GS). A similar pattern is evident in the amount of time per presentation that monkeys fixatedeach category of image (B). This measure differs, however, in that monkeys dwell for similar lengths of timeon low- and high-status faces. After Deaner, R. O., Khera, A. V., & Platt, M. L. (2005). Monkeys pay per view:adaptive valuation of social images by rhesus macaques. Current Biology, 15, 543–548. Used with permission.

NEUROETHOLOGY OF ATTENTION IN PRIMATES 531since under natural conditions direct staring coordination of movements to regulate spacingserves as a threat gesture in many primate species between each individual and its cohort. For this(van Hoof, 1967). Analysis of dwell times—the reason, Chance and Jolly (1970: 171) suggestedduration of glances toward particular social sti- that ‘‘The social attention of individuals within amuli—provides a potential explanation for this cohort . . . must be directed exclusively at theparadox: Sexual cues consistently evoked pro- other members of it,’’ going on to note thatlonged stares, whereas faces generally evoked fixa- ‘‘even when they are an integral part of thetions of shorter duration. Frequent, furtive complete society, the distinct coherence of aglances toward high-ranked males may serve to cohort. . .may depend on their maintaining amaximize acquisition of important social infor- predominant degree of attention toward them-mation while simultaneously minimizing risk of selves.’’ Chance and Jolly proposed that the keyconflict. mechanism of dominance is not the threat of violence from the strongest member of the troop, but rather the ability of these individualsEVOLUTIONARY BIOLOGY OF to command the attention of other group mem-SOCIAL ATTENTION bers. In short, Chance and Jolly argued thatThus, both for freely moving lemurs and for primate societies are bound together by centri-macaques performing tasks in the laboratory, petal attention, specifically, in hierarchical socie-an important goal of visual attention is to main- ties, by attention toward high-status animals.tain observational contact with conspecifics. Although dominance may be structuredEthological studies of primate behavior suggest by the threat of violence and by the need forthis behavioral bias may serve at least two com- coalitional defense against these threatsplementary biological functions. Vision has long (Cheney & Seyfarth, 1990; Keverne et al.,been known to play a role in hunting and fora- 1978), status-based saliency seems to be positiveging, affecting both predators (e.g., carnivores), in valence and largely prosocial, in that it pro-where selection pressures favor focused bino- motes proximity to the group. For example,cular fields of view, and prey (e.g., ungulates), Chance and Jolly (1970) described a behaviorwhere selection favors broad, monocular visual called ‘‘reflected escape’’ in which a subordinatefields (Allman, 1999; Cartmill, 1972). Primates, animal, threatened, runs in a looping arc, firsttoo, have focused, binocular visual fields away from the challenger and then back toward(Allman, 1999; Martin, 1990), but this does not the central members of the group—even if thesefree us from the need to be vigilant for predators same dominant individuals had initiated theor hostile competitors. Indeed, primate societies threat! These ideas seem to be supported byare characterized by certain baseline levels of findings that gaze (Keverne et al., 1978), likeaggression, and primates must be wary of allegiance and grooming (Cheney & Seyfarth,social threats both external and internal to their 1990), is allocated preferentially to dominantown social group. Thus, many primates must individuals but independently of their aggres-actively balance surveillance against external siveness, and also by our own finding that maca-predation or rival social groups with surveillance ques sacrifice more juice to view dominantagainst aggression from within the social group animals than subordinate animals.(e.g., Caine & Marra, 1988). Protection against Just as prosocial reward may drive attention,social threat is a key need driving visual however, there is evidence that enhanced sal-attention. ience may itself drive reward. Specifically, the Centripetal surveillance, however, implies mere act of attending to a stimulus appears tothat there is a social group in the first place. enhance its desirability. Zajonc first describedFrom this we infer a second, more subtle role these effects in 1968 when he found that briefof social attention, first articulated by Chance presentation of unfamiliar visual stimuli causedand Jolly (1970). Cohesion of social groups people to subsequently rate those stimuli morerequires, as a principal element, the esthetically pleasing, even when they could not

532 PRIMATE NEUROETHOLOGYrecall having seen them (Zajonc, 1968, reviewed the ventral striatum or amygdala. We tend toin Bornstein, 1989). More recently, two studies estimate the qualities of subjective experienceshave generalized this effect from ‘‘mere expo- by analogy: To understand animals whose beha-sure’’ to attentional state. Raymond and collea- vior differs broadly from our own, we may pro-gues (2003) found that stimuli that were ceed most safely when we can supplement ourpresented, but ignored, accrued negative asso- analogical reasoning with mechanistic, as well asciations in a variety of task conditions. Shimojo behavioral, data endorsing homology. Cases likeand colleagues (2003) made a complementary this suggest that a neuroethological approachdiscovery, using simple preference judgments. has the potential to clarify both our mechanisticThey found that prior to selecting the more understanding and our theoretical interpreta-attractive of two faces, subjects looked increas- tion of adaptive behavior.ingly long and often at the face they subsequentlychose. Importantly, when subjects were forced to ATTENDING TO INTENTIONSlook at a particular face, they were also morelikely to subsequently prefer it. Together, these Ralph Waldo Emerson (1876: 118) wrote, ‘‘Thefindings suggest that attention may drive eyes of men converse as much as their tongues,changes in affective judgments. By extension, with the advantage, [sic] that the ocular dialect‘‘mere exposure’’ may mediate social cohesion needs no dictionary, but is understood all thein primates by encouraging approach toward world over.’’ As Emerson intimated, where wepreviously attended members of the social look often betrays our deepest interests, inten-group, even if the attention was initially gar- tions, and desires. We use visual orienting notnered through such negatively valenced ago- only to localize other individuals but also tonistic interactions as direct competition or interpret their relationships, attitudes, andthreat. In this way, social saliency could play a intentions. Nonhuman primates also appear tocritical role in patterning the spacing behavior of use orienting by others to infer the location ofanimals in a group, making the most often important stimuli and events, to predict beha-fixated animals the most desirable for approach. vior, and perhaps even to interpret social rela- This complex relationship between attention, tionships among others (Cheney & Seyfarth,approach, and hedonic value suggests that 1990; Emery, 2000; Tomasello et al., 2005).simple approach/avoid, pleasure/pain dichoto- Subtler still, humans (and perhaps other pri-mies may serve us poorly in studying the neu- mates, particularly apes; de Waal, 2003) useroscience of attention. Instead, it may be more and recognize a number of deictic gestures,useful to consider attention in terms of ‘‘motiva- from a quick flick of the eyes to sustainedtional salience,’’ the predicted marginal beha- pointing, which signal important threats andvioral utility of information, and in terms of opportunities within our shared environment.the specific neural systems through motivational Furthermore, we use these signals in competitivesalience that govern behavioral orienting. contexts to read intent and predict action It seems reasonable that attention to high- (watching someone’s eyes during chess), andvalue social targets is promoted by the reward even to confound such predictions by otherssystems of the brain, and this idea is endorsed by (the ‘‘no-look pass,’’ a feint in which a playerongoing studies in our lab (Klein & Platt, 2008). looks toward a different teammate than the oneThe motivators that drive attention, however, to whom he or she intends to pass the ball).are not always entirely appetitive; prolonged Despite the obvious importance of social cuesstaring at dominant males, for example, is risky for guiding attention in natural behavior, thisand unlikely to be hedonically pleasing (conse- process has remained, until recently, relativelyquentially, fixation durations are quite short). In unexplored by psychologists or neurobiologists.the end, it is much harder to answer the question A typical laboratory approach to visual attentionof whether behaviors are mediated by ‘‘pleasure’’ asks subjects to stare at a fixation point, followedor ‘‘fear’’ than whether they are mediated by, say, by either a central cue or peripheral stimulus

NEUROETHOLOGY OF ATTENTION IN PRIMATES 533directing attention toward a peripheral location, with observed gaze appear to be categoricallyfollowed by a behavioral measure of attentional distinct from attentional responses to explicitdeployment at the cued or uncued location (e.g., cues such as predictive arrows (Friesen et al.,Posner 1980). Studies using this technique have 2004) or abstract spatial associations (Galfanorevealed that central cues that validly predict the et al., 2006), suggesting that social orienting doeslocation of a future peripheral target cause sub- not neatly fit within classical models of exo-jects to shift attention in a voluntary (‘‘endogenous or endogenous attention. These resultsgenous’’ or ‘‘top-down’’) manner, whereas appear to support the idea that humans evolvedabrupt peripheral cues, even when they have no a dedicated gaze-following module specializedpredictive value, automatically attract attention for rapid and reflexive sharing of attention in(‘‘exogenous,’’ ‘‘reflexive,’’ ‘‘bottom-up’’ atten- social groups (Baron-Cohen, 1994; Perrett &tion). These attention shifts are evident in Emery, 1994).changes in sensory discrimination performance To test this hypothesis, we contrasted theand reaction time, and have distinct time socially cued orienting responses of rhesuscourses: Exogenous attention operates more macaques and humans (Deaner & Platt, 2003).quickly and generates a subsequent orienting Surprisingly, we found that macaques anddeficit (‘‘inhibition of return’’), while endo- humans both responded more quickly to angenous attention is slower and more sustained unpredictable target when it appeared where a(Jonides, 1981; Muller & Rabbitt, 1989). Despite monkey had just been seen to look. Further-the obvious utility of this paradigm for under- more, eye position during fixation in bothstanding basic aspects of attention, its etholo- species drifted in the direction of observedgical relevance has long been limited by a failure gaze, likely reflecting an incompletely sup-to apply these laboratory techniques to the kind pressed drive to overtly orient in the sameof social stimuli that pervasively guide orienting direction (Engbert & Kliegl, 2003; Hafed &by primates in the natural world. Clark, 2002). The temporal dynamics with Friesen and Kingstone (1998) addressed this which attention followed observed gaze weregap by modifying the Posner paradigm to inves- highly similar in the two species (Fig. 26.5),tigate how social cues influence attention. They suggesting shared neural mechanisms.discovered that viewing a face with averted Our results strongly support the conclusiongaze rapidly and reflexively shifts attention in that gaze following is not unique to humans, andthe same direction, even when gaze direction may, in fact, rely on neural substrates that aredoes not predict the eventual location of the widespread among primates and possibly othertarget. In their experiments, human subjects animals known to follow gaze. Though gaze fol-were instructed to fixate a central point, where a lowing by other animals may differ, both inface briefly appeared with its eyes cast either right- strength and kind, from that evinced by anthro-ward or leftward. A split second later, a peripheral poid primates (Okamoto-Barth et al., 2007;target appeared, randomly in the direction of gaze Tomonaga, 2007), it appears that many animalsor in the opposite direction. Subjects were faster are able to shift attention in response to observedto respond to targets appearing in the direction of social cues. Consistent with this argument,observed gaze, even for cue-to-target delays as Tomasello and colleagues, along with a numberbrief as 105 ms (termed ‘‘stimulus onset asyn- of other research groups, have amassed a largechrony,’’ or SOA). body of work showing that many animals, Subsequent studies determined that these including apes (Brauer et al., 2005), dogseffects were both general and involuntary—a (Agnetta et al., 2000), monkeys (Tomasello etturned head shifted attention as easily as averted al., 1998), goats (Kaminski et al., 2005), dolphinseyes (Langton & Bruce, 1999), and social cuing (Tschudin et al., 2001), and ravens (Bugnyarpersisted even when the target was 80% likely to et al., 2004), can use social gaze cues to findappear in the direction opposite viewed gaze hidden food or retrieve objects (reviewed in(Driver et al., 1999). Attention shifts associated Emery, 2000, and Itakura, 2004).

534 PRIMATE NEUROETHOLOGYFigure 26.5 Gaze following by monkeys and humans shares psychophysical features. Monkeys andhumans show similar magnitude and time course of gaze following in response to nonpredictive monkeygaze cues presented continuously for 100, 200, 400, or 800 ms prior to target presentation. These attentionshifts were evident both by decreases in normalized reaction times to congruent (dashed) versusincongruent (solid) stimuli (A) and by microsaccades in the direction of observed gaze during cuepresentation (B). **, P <0.001; *, P <0.05. After Deaner, R. O., & Platt, M. L. (2003). Reflexive socialattention in monkeys and humans. Current Biology, 13, 1609–1613. Used with permission. In most of those studies, human experimen- infrared telemetric gaze-tracking device whileters, rather than conspecifics, provided the social they spontaneously interacted with otherattention stimuli. This technical limitation lemurs. We found that lemurs tended to orientsimultaneously limits the ease with which we their eyes in the same direction that other lemurscan generalize results to naturally occurring oriented their bodies and heads (Fig. 26.6A,B).social interaction, and poses the fascinating Such gaze alignment, however, could reflectquestion of how heterospecific and conspecific coincidental orienting to salient events in asocial perceptions interrelate. As the neural shared environment (e.g., a loud sound) rathersystems mediating predator avoidance, prey than active use of social gaze cues. To addresscapture, and ‘‘pure’’ social interaction remain this confound, we examined the temporalvirtually unknown, the many fascinating evolu- sequence of gaze alignments around the timetionary and computational links between them the subject lemur oriented to an observedremain almost largely unexplored. lemur. We found that, prior to fixating the The potential ubiquity of gaze following in observed lemur, there was no alignment betweenprimates is supported by our recent studies of the two animals’ gaze. After fixating the observedvisual orienting by freely moving, socially lemur, however, gaze alignment increased sig-housed lemurs. In those studies, we quantita- nificantly (Fig. 26.6C). The temporal sequencetively and precisely monitored orienting beha- of gaze alignment supports the conclusion thatvior of two male ring-tail lemurs with an lemurs actively follow the gaze of other

NEUROETHOLOGY OF ATTENTION IN PRIMATES 535Figure 26.6 Spontaneous gaze following in lemurs. Lemurs spontaneously follow the gaze direction oftheir conspecifics in natural interaction. Lemurs not only coorient with the body (A) and head (B) axes ofobserved lemurs but also selectively increase gaze alignment with those individuals they have recentlyattended (C). In panels A and B, red outward lines are gaze offsets that are overrepresented with respect tochance, while blue inward lines are gaze offsets that are underrepresented. In panel C, tick marks occur atmean gaze offsets recorded in §-second periods prior to fixation, in the period during which the lemur isfixated, and for § second periods after fixation. Shaded regions in panel C reflect the dispersion of gazealignments. Starred intervals are significantly aligned with gaze (chi2 test P <0.05). After Shepherd, S. V., &Platt, M. L. (2008). Spontaneous social orienting and gaze following in ringtailed lemurs (Lemur catta).Animal Cognition, 11, 13–20. Used with permission.individuals (Shepherd & Platt, 2008). Our when such cues fail to predict the location of aresults stand in sharp contrast to at least two behavioral goal, it has been argued that gazeprior observational studies (Anderson & following is a strictly reflexive behavior mediatedMitchell, 1999; Itakura, 1996) that concluded by a dedicated neural module (Deaner & Platt,that prosimian primates do not follow the gaze 2003; Driver et al., 1999). Recent studies, how-of human observers. ever, challenge the notion that gaze cuing is purely reflexive, and instead indicate that social context can influence gaze-following behaviorSOCIAL CONTEXT INFLUENCES both in humans and monkeys. Specifically, sev-GAZE FOLLOWING eral lines of evidence suggest that neural systemsBecause both monkeys and humans shift their contributing to gaze following are regulatedattention in response to social gaze cues, even both by internal factors and online, by social

536 PRIMATE NEUROETHOLOGYcontext. For example, human females respond monkeys. Importantly, each animal was desig-much more strongly to social gaze cues than do nated dominant or subordinate based on themales (Fig. 26.7) (Bayliss et al., 2005; Deaner direction and frequency of threat and submis-et al., 2007); moreover, our lab has found that sion gestures during controlled pair-wise con-gaze following in females, but not males, is influ- frontations (see Deaner et al., 2005; Shepherd etenced by the familiarity of the observed male cue al., 2006). We found that subordinate monkeys(Deaner et al., 2007). These observations suggest rapidly and automatically followed the gaze of allthe possibility that sex hormones may play an other monkeys (Fig. 26.8A), while dominantimportant role in regulating social attention. monkeys followed gaze later, and then onlyThese observations also indicate that gaze-fol- in response to other dominant monkeyslowing mechanisms are not strictly reflexive and (Fig. 26.8B). These differences in gaze-followinginformationally encapsulated, but instead are behavior were weakly correlated with differencessensitive to subtle changes in social variables in testosterone production (Shepherd et al.,such as the familiarity of the observed face. 2006), as inferred from measurements of testis In parallel, we have discovered that both volume (Bercovitch & Ziegler, 2002). We inter-social context and biological factors regulate pret these data to indicate that both internal andgaze following in rhesus macaques (Shepherd external factors govern macaque gaze following,et al., 2006). Specifically, we probed gaze- comprising both biological variables like testos-following behavior by seven male rhesus maca- terone and ecological variables like relative socialques in response to four rightward- and four status.leftward-looking photos of each of four familiar One way that familiar or dominant individuals might evoke stronger gaze following is by virtue of their intrinsic salience. Under some conditions, human gaze following appears to be modulated by emotional content in faces. Specifically, patients with anxiety disorders show heightened following of fearful gaze relative to other emotional expressions (Holmes et al., 2006; Hori et al., 2005; Mathews et al., 2003) (note also Hietanen & Leppanen, 2003, and Putman et al., 2006). This interaction probably reflects a tendency for patients with anxiety to more strongly attend to negatively valenced social stimuli, while healthy subjects dwell less on them (e.g., Bar-Haim et al., 2005; Bradley et al., 1997). These studies implyFigure 26.7 Sex differences in gaze following in that attention to another individual spreads auto-humans. Human females exhibit stronger gaze matically to other objects with which that indivi-following than males, and furthermore dual is engaged.discriminate between familiar and unfamiliar Together, these results demonstrate that theindividuals when following gaze. Females (solid neural substrates of gaze following are deeplylines) have greater reaction time savings for gaze- integrated with the larger social information pro-congruent than gaze-incongruent targets when cessing stream. That gaze following is an inherentgaze cues were from familiar (A) rather than component of face perception is suggested by theunfamiliar (B) individuals (at 200 ms, P < 0.003).Males (dashed lines) did not distinguish fact that heightened attention to faces spreads insignificantly between these conditions (at 200 ms, the direction of gaze, both in the case of femalesP > 0.4). After Deaner, R. O., Shepherd, S. V., & viewing familiar faces and anxious patients seeingPlatt, M. L. (2007). Familiarity accentuates gaze faces with negatively valenced emotional content.cuing in women but not men. Biology Letters, 3, At the same time, however, the fact that gaze64–67. Used with permission. following is modulated by factors like familiarity

NEUROETHOLOGY OF ATTENTION IN PRIMATES 537Figure 26.8 Social context influences gaze following in macaques. Even at the briefest cue durations,subject social status appears to influence gaze-following behavior (A, P <0.005). Specifically, low social statusmakes a monkey more likely to follow gaze within 100 ms of seeing the cue, and also more likely to havestrong inhibition of return at the latest time point—a temporal profile consistent with a reflexive attentionshift, possibly due to increased anxiety or to the modulatory effects of sex- and status-linked hormones liketestosterone on social processing circuitry in the brain. Cue social status also plays an important role (B, P<0.01), leading to prolonged attention in the direction of gaze of a high-status cue and inhibited attention inthe direction of gaze of a low-status cue, particularly by high-status subjects. Reaction times for congruenttrials are shown in red lines and for incongruent trials are shown in blue lines. After Shepherd, S. V., Deaner,R. O, & Platt, M. L. (2006). Social status gates social attention in monkeys. Current Biology, 16, R119–120.Used with permission.and social dominance suggests that it is not GAZE AS A STRATEGIC SOCIALmediated by an isolated module sequestered SIGNALfrom other aspects of face processing and socialknowledge. Finally, sex differences in humans and As described previously, social saliency maysocial rank differences in monkeys both hint at a play a role not only in guiding attention butpossible role for sex hormones in shaping social also in shaping the physical spacing of groupattention systems in the brain. Together these members and the affective tenor of theirfindings strongly support the idea that social interactions. It may also serve as a startingattention is a fundamental feature of natural pri- point for the development of much moremate behavior and cognition, and provides a advanced cognitive behaviors. David Perrettfundamental challenge to the traditional endo- and Simon Baron-Cohen have argued thatgenous/exogenous model of attentional control. detection of eyes and interpretation of gaze

538 PRIMATE NEUROETHOLOGYare foundational to building a theory of structuring social behavior through gaze. Likemind, by which we intuitively mirror the humans, many animal species are capable ofattentional and perhaps even intentional following gaze. Chimpanzees are even reportedstates of others. We do this so instinctively to use deictic gestures (de Waal, 2003), thoughthat we frequently anthropomorphize even these signals may hold little behavioral currencyalien and impersonal phenomena, perhaps due to the ubiquity of competitive and paucityallowing us to understand complex and of cooperative interactions in these animalsdynamic patterns by analogy to human beha- (Hare & Tomasello, 2004). The importance ofvioral goals (e.g., ‘‘the electrons don’t like to social gaze for facilitating increased cooperationbe near one another and are instead attracted among human ancestors may even have lead toto the positive core of an atom, causing them somatic adaptations that increase the saliencyto settle sequentially into the centermost and specificity of gaze cues, for example, byuncrowded orbitals’’). Typically developing enhancing the visibility of gaze through increasedhumans have an intuitive expertise at com- contrast of the pupil versus sclera (Kobayashi &municating affect and attentional state, in Koshima, 2001; Tomasello et al., 2006). At thepart, perhaps, because they have an intrinsic same time, however, the continued importance ofdrive to learn to do so: From a young age, competitive interactions for early human ances-typically developing humans take pleasure in tors may have led to a compensatory enhance-successfully directing another’s attention ment of covert attention mechanisms relativetoward stimuli that we, too, have perceived to those possessed by most nonhuman primates(Tomasello et al., 2005). and, especially, other mammals. These considerations naturally lead us to This manipulative role of gaze is perhapsconsider overt eye movements as an active sig- the least understood aspect of visual orientingnaling mechanism shaping primate social inter- behavior; virtually nothing is known aboutactions. We have mentioned the role of eye how the demands of social signaling bring theirmovements in initiating conflict, but primates influence to bear on the gaze control system ofmake far more sophisticated use of gaze. For the brain. These considerations challenge theexample, eye contact can signal not only aggres- conventional division between ‘‘reflexive’’sion (van Hoof, 1967) but also sexual interest and ‘‘reward-mediated’’ processes in social(Dixson, 1998) or solicitation for coalition for- orienting. Given that current evidence arguesmation in agonistic interactions versus third against informational encapsulation in socialparties (de Waal, 2003). Likewise, humans use perception and cognition, it seems likely thateye contact as a key aspect of affiliation, court- the fastest orienting processes reflect the needship (Hrdy & Whitten, 1987), and intimidation to quickly acquire behaviorally relevant infor-(Argyle & Cook, 1976), and also during coordi- mation, and that slower processes reflect morenation of attention (‘‘triadic’’ or ‘‘joint atten- nuanced contextual constraints—whether thattion’’; Emery, 2000). Moreover, gaze acts to information is likely to have predictive value instructure both verbal and nonverbal human the present situation, or whether the orientingsocial interactions. To signal rank relationships, behavior might expose the subject to risk offor example, people look preferentially toward agonistic interaction or being ‘‘scooped’’ bythe most high-ranking person, and when con- competitors in pursuit of resources. Forversing, gaze is used to emphasize spoken argu- example, monkeys in our studies initiallyments, to conclude statements, to emphasize looked toward other individuals, generally fol-nonverbal reactions to heard statements, and lowing their gaze, but at later intervals diverseto coordinate turn-taking in conversation contextual considerations were brought to(Argyle & Cook, 1976). bear—abbreviating risky glances toward With the evolution of greater visual and higher-ranked individuals (Deaner et al., 2005;social complexity, some primates appear to Haude et al., 1976), extinguishing gaze followinghave evolved ever more sophisticated means of of lower-ranking animals (Shepherd et al.,

NEUROETHOLOGY OF ATTENTION IN PRIMATES 5392006), and prolonging male fixation on female seeking to share enjoyment, interests, or achieve-hindquarters (Deaner et al., 2005). Thus, ments with other people,’’ or to reciprocatealthough much work remains to be done on when these experiences and emotions arethe neurobiology of orienting responses, it shared by others (American Psychologicalseems clear that neither a strictly reflexive nor a Association, 1994). Outside the laboratory,strictly cognitive approach will explain all even high-functioning autistic individuals, unaf-aspects of social orienting. fected by physical problems such as seizures or repetitive movements, are nonetheless challenged in responding to the constant exchangeSOCIAL ATTENTION AND of social cues that structures human interaction.AUTISM: FROM THE LAB TO Temple Grandin, an associate professor ofTHE FIELD Animal Science at Colorado State UniversityData from syndromes that disrupt social beha- who has autism, reports that she functions invior, notably autism, suggest a gulf between social situations ‘‘solely by intellect and visuali-behavioral responses in the laboratory and zation skills’’ (Grandin, 1999, http://www. autis-spontaneous use of social cues in the real m.org/temple/social.html). She says (ibid) ‘‘I didworld. In a comparison of visual orienting by not know that eye movements had meaningautistic subjects with that of typically devel- until I read Mind Blindness by Simon Baron-oping children, van der Geest and colleagues Cohen. I had no idea that people communicated(2002) found that the fixation patterns of the feelings with their eyes. I also did not know thattwo groups could not be distinguished when people get all kinds of little emotional signalsthey viewed simple cartoons including human which transmit feelings. My understanding offigures. In contrast, Pelphrey and colleagues this became clearer after I read Descartes’ Error(2002) found substantial differences between by Antonio Damasio.’’ It may be that the com-autistic and typically developing control sub- plement of processes evoked by social stimuli injects’ inspecting of photographs of real faces. typically developing individuals is disrupted inSimilarly, although autistic subjects often show autism spectrum disorders, and that withoutnormal gaze following in the social variant of the these foundational elements, more sophisticatedPosner attention task (Chawarska et al., 2003; forms of empathy and social reasoning cannotKylliainen & Hietanen, 2004; Swettenham et al., develop.2003; but see Bayliss et al., 2005, and Ristic et al., It is interesting to note that both autism2005), they consistently show severe disruptions (Wassink et al., 2007) and social anxiety dis-in social orienting in more natural contexts. order (Skuse, 2006) have been associated withWhen Klin and colleagues (2002a,b) measured dysfunction in the serotonin signaling system.gaze behavior in autistic individuals watching Serotonin has likewise been linked to domi-the movie Who’s Afraid of Virginia Woolf, they nance status, affiliative social interaction, andfound that gaze toward social stimuli was dis- decreases in antagonistic and impulsive socialordered—for example, with normal fixations interactions (Edwards & Kravitz, 1997; Raleightoward the eye regions replaced by fixations et al., 1991), suggesting that this neuromodula-toward the mouth. Moreover, socially cued tory system may also contribute to differenceslocations were severely neglected, as shown by in social attention between dominant and sub-a marked lack of fixations toward gaze- and ordinate macaques. Together, these findingsgesture-cued regions of space. Furthermore, hint at a role for serotonin in regulating socialthe authors found that the degree of abnorm- attention in both human and nonhuman pri-ality in the fixation pattern of individual autistic mates. Determining the impact of biologicalsubjects in this task was strongly predictive of factors, such as serotonin and testosterone, onreal-life social impairment. social attention may point to possible interven- This is hardly surprising. After all, autism is tions to improve social functions in commondefined by a marked ‘‘lack of spontaneous psychopathologies like autism.

540 PRIMATE NEUROETHOLOGYTOWARD A NEUROETHOLOGY ventral visual processing stream (Ungerleider &OF ATTENTION IN PRIMATES Mishkin, 1982) to quickly identify images containing animals. It is important to note, however,If we were to develop a biologically plausible, that this model explicitly fails to localize animalsethologically motivated model of attention in within images. The processes that link objectprimates, what features must it have? We feel recognition by the ventral visual processingstrongly that the bottom-up component of stream to target localization within the dorsalthese models must not only reflect what we stream thus remain largely unknown, despiteknow about the primate visual system but must the fact that it is the dorsal stream that selectsalso consider the role vision plays in guiding the parts of the visual field for further processing. Inbehavior of primates in species-typical ecolo- fact, Serre and colleagues note that their modelgical and social contexts. For example, Laurent ‘‘cannot account for our everyday vision whichItti and colleagues, among others, have used involves eye movements and top-down effects,’’visual filters, inspired by the physiology of the (p. 6426) and that an extension of the modelprimate visual system, to predict human visual requiring ‘‘top-down signals from higher toattention. Such models estimate saliency by fil- lower areas . . . limit[ing] visual processing to atering images through a series of low-level fea- ‘spotlight of attention’ centered around theture maps (Carmi & Itti, 2006; Peters et al., animal target’’ results in ‘‘significant improve-2005). Each map tracks the extent to which a ment in the classification performance’’ (p. 6428).region ‘‘pops out’’ from its surroundings along a Serre’s study thus illustrates the benefits ofparticular visual dimension, such as brightness, considering the natural goals of orienting inorientation, texture, motion, or color, and these social contexts, and likewise of considering evi-maps can be combined to successfully model dence from functional imaging and neurophy-many aspects of bottom-up attention. siological recording studies. Recent functional While these models can accurately identify magnetic resonance imaging (fMRI) studies insalient regions of still images and video, they humans have identified brain areas that areoften fail to highlight social stimuli such as involved in visual analysis of body position andfaces, or rely heavily on image motion to assign identity (Downing et al., 2001), identification ofsaliency to humans and animals. Without under- faces (Haxby et al., 1994), and interpretation ofvaluing either these accomplishments or the actions and facial expressions (Allison et al.,importance of motion as a predictor of animacy, 2000); other studies suggest that homologouswe nevertheless note that demands of both soci- areas operate in macaques (Logothetis et al.,ality and predator avoidance require accurate and 1999; Tsao et al., 2003). The general conserva-fast discrimination of animals, even when those tion of cortical organization across primateanimals are stationary or when dynamic environ- species, together with these recent findings, sug-ments (e.g., running water, blowing leaves) pro- gests that visual areas specialized for processingduce irrelevant image motion. Moreover, while it social stimuli may be part of the primordialis true that identification and tracking of animate visual cortex that was present in stem primatesobjects has proved a challenge for computer (Rosa & Tweedale, 2005; Tootell et al., 2003)vision, these tasks are performed quickly and and perhaps others mammals as well (Kendrickeasily by the primate brain. In laboratory experi- et al., 2001).ments, humans can initiate saccades toward an As revealed through behavioral studies, theanimal in a novel photograph in as little as 120 ms gaze-control system must recognize and respond(Kirchner & Thorpe, 2006), and in unconstrained appropriately to biological targets. We speculateviewing, animate stimuli and especially other that parallel pathways accomplish this goal,humans are quickly targeted for visual inspection. which can largely be grouped into a subcortical Serre and colleagues (2007) partially addressed pathway and a cortical pathway (Adolphs, 2002;these issues by developing a model that uses bio- Vuilleumier, 2002). Ultimately, both pathwayslogically inspired filters based on neurons in the must converge upon the three neural tissues,

NEUROETHOLOGY OF ATTENTION IN PRIMATES 541which jointly serve as the final common pathway likely to reflect the native function of LIP,governing orienting in primates: the parietal eye only overridden in tasks where researchers arbi-fields (in macaque, located in the lateral intra- trarily map juice rewards onto specific oculo-parietal sulcus, called LIP), the frontal eye fields motor behaviors.(FEFs), and the superior colliculus (SC). To Ultimately, however, the social significanceeffectively govern orienting, these areas must of visual information is probably not computedweigh decision variables comprising the within the gaze-control network itself. Recentexpected rewards and risks associated with a evidence has begun to reveal how social vari-given orienting behavior. For example, Platt ables, such as dominance status and reproduc-and Glimcher showed that neurons in LIP are tive state, are processed in primate brains. Whilesensitive to target value when visual stimuli are social cognition involves broad swaths of brain,arbitrarily assigned different amounts of juice we believe it can be simplified into two streams: areward (Platt & Glimcher, 1999), and subse- streamlined but inflexible subcortical pathway,quent studies have confirmed that neuronal and a nuanced—and labyrinthine—corticalactivity throughout this network is similarly network.modulated by orienting value (LIP, see alsoSugrue et al., 2004; SC, Ikeda & Hikosaka, THE SUBCORTICAL PATHWAY2003; but not FEF, Leon & Shadlen, 1999). Critically, we have recently shown that in the In this pathway, retinotectal inputs provide‘‘pay-per-view’’ paradigm discussed previously, coarse visual information in which crude biolo-neurons in LIP are modulated by the intrinsic gical primitives can quickly be identified andsocial value of orienting to images in much the analyzed, for example, to locate other creatures,same way that they are modulated by primary detect social signals, and extract gaze directionjuice rewards (Klein et al., 2008). Specifically, (Johnson, 2005). The subcortical system appearsLIP neurons respond most strongly when mon- to by phylogenetically old, shared by all terres-keys evaluate targets associated with the acquisi- trial vertebrates, and is the major pathway fortion of information about female reproductive innate recognition and response to animate tar-signals and the identity of dominant males, but gets including predators, prey, and conspecificsrespond weakly when the same target offers (Sewards & Sewards, 2002).information about subordinate males—despite In humans, this system has been suggested tothe fact that monkeys were never explicitly play a crucial role in early visual tracking of faces,trained to orient toward these stimuli. These and may play a lifelong role in the rapid detectionobservations directly predict the orienting beha- of socially salient or threatening signals, bothvior of macaques in the same task. Echoing these directly and through interactions with visualfindings, a recent fMRI study in humans found cortex and the pulvinar nucleus of the thalamusstronger activation of parietal cortex when (Grieve et al., 2000; Johnson, 2005; Sewards &subjects played a game against a dominant Sewards, 2002). Just such a relay of social threatopponent compared to an inferior opponent signals, from retina to SC, pulvinar nucleus, and(Zink et al., 2008). finally amygdala, has already been identified in These observations indicate that LIP spon- humans by neuroimaging (Morris et al., 1999).taneously integrates information about target Furthermore, neurons in the macaque amygdalavalue from multiple sources, in the absence of are sensitive to the expression, gaze, and socialany explicit training. This, in turn, implies that dominance of viewed faces (Gothard et al., 2007;brain pathways that process social information Hoffman et al., 2007; Kawashima et al., 1999).must ultimately transmit this information to The amygdala, in turn, sends this first-pass ana-parietal cortex, and likely to other areas lysis of social targets toward gaze-control centersinvolved in orienting behavior as well. This and higher visual areas, acting rapidly tomodulation of neural activity by the intrinsic strengthen social and threat-related processingvalue of acquired visual information seems (Vuilleumier, 2002).

542 PRIMATE NEUROETHOLOGY While a major function of this pathway is identity, and expression, respectively. Theseindubitably to provide an ‘‘early warning’’ areas then transmit output to a broad array ofsystem detecting threats, there is evidence that areas in the extended face processing network,the amygdala also mediates prosocial behaviors. interacting with contextual signals from hippo-For example, while eye contact can signal threat, it campus, amygdala, and orbitofrontal cortexmore generally indicates approach, and often (OFC) (Ishai et al., 2005; Smith et al., 2006).serves to initiate grooming and sexual behavior These socially influenced areas include multiple(Hrdy & Whitten, 1987). Thus, eye contact regions implicated in modulating both atten-responses in amygdala may serve to indicate not tional allocation and reward processingonly threat but also sexual opportunity, and (Sabbagh, 2004; Vuilleumier, 2002).indeed, amygdala is strongly activated by sexual It is important to note that both orienting tostimuli (Aharon et al., 2001; Hamann et al., 2004). others and gaze following is regulated by social The amygdala pathway may be highly sensi- milieu—as well as by intrinsic factors includingtive to biological factors that mediate sex differ- sex hormones, such as testosterone, and neuro-ences (Bayliss et al., 2005; Deaner et al., 2007; modulators, such as serotonin. Supporting thisGoldstein et al., 2001; Hamann et al., 2004) as idea, amygdala, OFC, and hippocampus form awell as psychosocial disorders (Holmes et al., functional circuit important for associating emo-2006; Hori et al., 2005; Mathews et al., 2003; tional and social salience with mnemonic andPutman et al., 2006), and may, when compro- perceptual information (Fig. 26.9) (Sabbagh,mised, contribute to the development of autism 2004; Smith et al., 2006; Vuilleumier, 2002), and(Schultz, 2005, though note also Amaral et al., are actively involved in the perception of faces2003). While amygdala influences the function (Ishai et al., 2005). Each of these brain structuresof a broad swath of cortex, it does not directly is sexually dimorphic (Goldstein et al., 2001),interact with the gaze-control network and suggesting that sexual differentiation in thesewould instead influence orienting indirectly, areas may directly pattern responses to socialthrough cortical or subcortical intermediaries. cues. This supposition is strengthened by various results showing fetal testosterone negatively impacts both social attention and social relation-THE CORTICAL NETWORK ships in human juveniles (Knickmeyer & Baron-In parallel to the subcortical pathway, a slower, Cohen, 2006). Ultimately, signals from these ven-more nuanced, and more recently evolved cor- tral (‘‘what’’) areas must relay social informationtical pathway leads from V1 through the ventral to dorsal (‘‘where’’) orienting and attention con-visual stream to extrastriate body area (EBA) trol systems. Signals from the higher-order areas(Downing et al., 2001), fusiform face area of the ventral pathway ramify to multiple targets(FFA) (Haxby et al., 1994), and superior tem- in the visual orienting system, but exactly how thisporal sulcus (STS) (Allison et al., 2000). occurs remains an open question, since much ofIt remains unclear whether these areas assess visuo-social cortex (Tsao et al., 2003) is connectedsubordinate-level distinctions between hier- in one or two steps to posterior parietal (7A andarchically classifiable objects or are optimized LIP; Seltzer & Pandya, 1991), frontal (SEF andfor processing animate objects or specifically FEF; Seltzer & Pandya, 1989), and subcorticalfor perception of conspecifics. It seems likely orienting areas (pulvinar nucleus; Romanskithat the development of these areas depends on et al., 1997) and superior colliculus (Fries, 1984).experience (Gauthier et al., 1999) and may relyupon signals arising in the subcortical pathwayfor appropriate patterning during development GAZE FOLLOWING AND THE(Johnson, 2005; Schultz, 2005; Sewards & ‘‘MIRROR NETWORK’’Sewards, 2002). It is known, however, that eachof these ventral stream visual areas is strongly One intriguing possibility is that gaze followingactivated by social stimuli, extracting posture, may be supported by specialized neurons that

NEUROETHOLOGY OF ATTENTION IN PRIMATES 543Figure 26.9 Key circuits involved in social attention. Connectivity of social (red), reward (blue), andattention (green) pathways. In addition to the cortical pathway, a fast subcortical pathway connects superiorcolliculus to amygdala via the thalamus (not shown). Note that several social processing areas lie alongsuperior temporal sulcus, occupying both posterior and anterior temporal lobes, and that functional activityin imaging tasks has not yet been systematically related to past anatomical studies. AMYG, amygdala; FEF,frontal eye fields; OFC, orbitofrontal cortex; PPC, posterior parietal cortex, including 7A and lateralintraparietal sulcus; SEF, supplementary eye fields; STS, superior temporal sulcus regions.simultaneously map concordant observed/sen- motor cortices. To date, however, neither thesory and performed/motor components of this responses of mirror neurons nor more globalbehavior. Similarly specialized ‘‘mirror’’ neu- sensory-motor mirroring activations observedrons, first discovered in macaque premotor with fMRI have been directly linked to specificcortex, respond not just when the subject social behaviors in monkeys or humansreaches for an object but also when it observes (Dinstein et al., 2008; but see Prather et al.,the experimenter perform a similar movement 2008, for a description of sensory-motor mir-(di Pellegrino et al., 1992). ‘‘Mirroring’’ roring by neurons in songbirds during socialresponses have been observed in other brain interaction).areas (e.g., single units within parietal reach Recently, we discovered a population of neu-areas; Fogassi et al., 2005), and for other types rons in macaque LIP that ‘‘mirror’’ the observedof movements (e.g., mouth movements; Ferrari gaze of individuals (Shepherd et al., 2009), muchet al., 2003). Because mirror neurons appear to as neural populations in adjacent areas mirrorsignal abstract actions and goals independent of reaching behavior (Fogassi et al., 2005). LIPactor, they have been hypothesized to underlie neurons respond when monkeys orient atten-a host of abilities ranging in sophistication tion, either overtly or covertly, to regions offrom action- and emotion-perception to space known as response fields (Colby et al.,empathizing, mentalizing, and theory of mind 1996; Gnadt & Andersen, 1988; Platt &(reviewed in Rizzolatti & Craighero, 2004). Glimcher, 1998). We found that a subpopula-Supporting these findings from macaques, ima- tion of LIP neurons responded not only to directging studies in humans suggest that homolo- attention toward their response field but alsogous brain regions are activated by both the when viewing another monkey that looked inproduction and observation of specific gestures the neuron’s preferred direction. Neuronal(Dinstein et al., 2007), and that activations responses to the observed orienting behavior ofassociated with sensory-motor ‘‘mirroring’’ another individual occurred despite the fact thatmay be widespread in human sensory and there was no visual stimulus in the neuron’s

544 PRIMATE NEUROETHOLOGYresponse field, and that any gaze shift toward the neurological mechanisms. Behavioral andresponse field would abort the trial without Cognitive Neuroscience Reviews, 1, 21–61.reward. Intriguingly, the time course of social Agnetta, B., Hare, B., & Tomasello, M. (2000). Cuesgaze cue signals in LIP strongly paralleled the to food location that domestic dogs (Canistime course of gaze-following behavior— familiaris) of different ages do and do not use. Animal Cognition, 3, 107–112.appearing within a tenth of a second, persisting Aharon, I., Etcoff, N., Ariely, D., Chabris, C. F.,for several hundred milliseconds, and ultimately O’Connor, E., & Breiter, H. C. (2001).fading at the longest intervals tested. These data Beautiful faces have variable reward value:provide strong support for the idea that some fMRI and behavioral evidence. Neuron, 32,populations of mirror neurons provide an 537–551.important mechanistic foundation for the Allison, T., Puce, A., & McCarthy, G. (2000). Socialabstraction, interpretation, and imitation of perception from visual cues: Role of the STSbehaviors and mental states. region. Trends in Cognitive Science, 4, 267–278. Allman, J. M. (1999). Evolving brains. New York: W.H. Freeman & Company.CONCLUSIONS Amaral, D. G., Bauman, M. D., & Mills Schumann, C. (2003). The amygdala and autism:Laboratory research using arbitrary tasks and sti- Implications from non-human primatemuli have identified two complementary systems studies. Genes, Brain and Behavior, 2, 295–302.for visual orienting—one fast and reflexive, the American Psychological Association. (1994).other slow and deliberative. Neuroethological Autistic disorder. In: Diagnostic and Statisticalstudies of visual attention, by contrast, have Manual of Mental Disorders (4th ed., p. 66). Washington, DC: Author.revealed a suite of socially motivated and socially Anderson, J. R., & Mitchell, R. W. (1999).cued orienting behaviors that do not cleave neatly Macaques but not lemurs co-orient visuallyalong these lines. Specifically, primates and other with humans. Folia Primatologica (Basel), 70,animals are motivated to look at one another, 17–22.preferentially orient to high-value social targets Argyle, M., & Cook, M. (1976). Gaze & mutualsuch as the faces of dominant males, and follow gaze. Cambridge, England: Cambridgethe orienting movements of others with their own University Press.attention. Moreover, these responses are regu- Bar-Haim, Y., Lamy, D., & Glickman, S. (2005).lated by behavioral context, sex hormones, and Attentional bias in anxiety: A behavioral andserotonin. These observations strongly support ERP study. Brain Cognition, 59, 11–22.the idea that the primate brain is specialized for Baron-Cohen, S. (1994). How to build a baby that can read minds: Cognitive mechanisms inacquiring behaviorally useful visual information mindreading. Current Psychology of Cognition,from the social world, and that these adaptations 13, 513–552.rely on the integration of multiple neural circuits Barton, R. A., & Dunbar, R. L. M. (Eds.). (1997).involved in identifying social stimuli and social Evolution of the social brain. Cambridge:cues, determining their meaning, and responding Cambridge University Press.appropriately. Despite the commonalities of these Bayliss, A. P., diPellegrino, G., & Tipper, S. P.systems across primates and even other mam- (2005). Sex differences in eye gaze andmals, the challenge for future neuroethological symbolic cueing of attention. Quarterly Journalresearch is to determine how these mechanisms of Experimental Psychology A, 58, 631–650.contribute to adaptive differences in social beha- Bercovitch, F. B., & Ziegler, T. E. (2002). Currentvior in different species. topics in primate socioendocrinology. Annual Review of Anthropology, 31, 45–67. Bornstein, R. F. (1989). Exposure and affect:REFERENCES Overview and meta-analysis of research, 1968- 1987. Psychological Bulletin, 106, 265–289.Adolphs, R. (2002). Recognizing emotion from Bradley, B. P., Mogg, K., Millar, N., Bonham- facial expressions: Psychological and Carter, C., Fergusson, E., Jenkins, J., et al.

NEUROETHOLOGY OF ATTENTION IN PRIMATES 545 (1997). Attentional biases for emotional faces. social images by rhesus macaques. Current Cognition and Emotion, 11, 25–42. Biology, 15, 543–548.Brauer, J., Call, J., & Tomasello, M. (2005). All great Deaner, R. O., & Platt, M. L. (2003). Reflexive ape species follow gaze to distant locations and social attention in monkeys and humans. around barriers. Journal of Comparative Current Biology, 13, 1609–1613. Psychology, 119, 145–154. Deaner, R. O., Shepherd, S. V., & Platt, M. L.Bugnyar, T., Stowe, M., & Heinrich, B. (2004). (2007). Familiarity accentuates gaze cuing Ravens, Corvus corax, follow gaze direction of in women but not men. Biology Letters, 3, humans around obstacles. Proceedings of the 64–67. Royal Society: B Biological Science, 271, 1331– di Pellegrino, G., Fadiga, L., Fogassi, L., Gallese, V., 1336. & Rizzolatti, G. (1992). Understanding motorButler, R. A. (1954). Incentive conditions which events: A neurophysiological study. Experi- influence visual exploration. Journal of mental Brain Research, 91, 176–180. Experimental Psychology, 48, 19–23. Dinstein, I., Hasson, U., Rubin, N., & Heeger, D. J.Caine, N. G., & Marra, S. L. (1988). Vigilance and (2007). Brain areas selective for both observed social organization in two species of primates. and executed movements. Journal of Animal Behaviour, 36, 897–904. Neurophysiology, 98, 1415–1427.Carmi, R., & Itti, L. (2006). Visual causes versus Dinstein, I., Thomas, C., Behrmann, M., & Heeger, correlates of attentional selection in dynamic D. J. (2008). A mirror up to nature. Current scenes. Vision Research, 46, 4333–4345. Biology, 18, R13–18.Cartmill, M. (1972). Arboreal adaptations and the Dixson, A. F. (1998). Primate sexuality: Compa- origin of the order primates. In: R. Tuttle (Ed.), rative studies of the prosimians, monkeys, apes, The functional and evolutionary biology of and human beings. New York: Oxford primates (pp. 97–212). Chicago: Aldine- University Press. Atherton Press. Downing, P. E., Jiang, Y., Shuman, M., &Chance, M., & Jolly, C. (1970). Social groups of Kanwisher, N. (2001). A cortical area selective monkeys, apes and men. New York: E.F. Dutton. for visual processing of the human body.Chawarska, K., Klin, A., & Volkmar, F. (2003). Science, 293, 2470. Automatic attention cueing through eye Driver, J., Davis, G., Kidd, P., Maxwell, E., movement in 2-year-old children with autism. Ricciardelli, P., & Baron-Cohen, S. (1999). Child Development, 74, 1108–1122. Gaze perception triggers reflexive visuospatialCheney, D. L., & Seyfarth, R. M. (1990). How orienting. Visual Cognition, 6, 509–540. monkeys see the world: Inside the mind of another Edwards, D. H., & Kravitz, E. A. (1997). Serotonin, species. Chicago: University Of Chicago Press. social status and aggression. Current OpinionsColby, C. L., Duhamel, J. R., & Goldberg, M. E. in Neurobiology, 7, 812–819. (1996). Visual, presaccadic, and cognitive Egeth, H. E., & Yantis, S. (1997). Visual attention: activation of single neurons in monkey lateral Control, representation, and time course. intraparietal area. Journal of Neurophysiology, Annual Review of Psychology, 48, 269–297. 76, 2841–2852. Emerson, R. W. (1876). The conduct of life. Boston:Corbetta, M., & Shulman, G. L. (2002). Control of James R. Osgood & Company. goal-directed and stimulus-driven attention Emery, N. J. (2000). The eyes have it: The in the brain. Nature Reviews: Neuroscience, 3, neuroethology, function and evolution of 201–215. social gaze. Neuroscience and BiobehavioralDe Martino, B., Kumaran, D., Seymour, B., & Reviews, 24, 581–604. Dolan, R. J. (2006). Frames, biases, and Engbert, R., & Kliegl, R. (2003). Microsaccades rational decision-making in the human brain. uncover the orientation of covert attention. Science, 313, 684–687. Vision Research, 43, 1035–1045.de Waal, F. B. (2003). Darwin’s legacy and the Ferrari, P. F., Gallese, V., Rizzolatti, G., & Fogassi, study of primate visual communication. L. (2003). Mirror neurons responding to the Annals of the New York Academy of Sciences, observation of ingestive and communicative 1000, 7–31. mouth actions in the monkey ventralDeaner, R. O., Khera, A. V., & Platt, M. L. (2005). premotor cortex. European Journal of Monkeys pay per view: Adaptive valuation of Neuroscience, 17, 1703–1714.

546 PRIMATE NEUROETHOLOGYFogassi, L., Ferrari, P. F., Gesierich, B., Rozzi, S., Guo, K., Robertson, R. G., Mahmoodi, S., Tadmor, Chersi, F., & Rizzolatti, G. (2005). Parietal lobe: Y., & Young, M. P. (2003). How do monkeys From action organization to intention view faces? A study of eye movements. understanding. Science, 308, 662–667. Experimental Brain Research, 150, 363–374.Fries, W. (1984). Cortical projections to the Hafed, Z. M., & Clark, J. J. (2002). Microsaccades superior colliculus in the macaque monkey: as an overt measure of covert attention shifts. A retrograde study using horseradish Vision Research, 42, 2533–2545. peroxidase. Journal of Comparative Neurology, Hamann, S., Herman, R. A., Nolan, C. L., & 230, 55–76. Wallen, K. (2004). Men and women differ inFriesen, C. K., & Kingstone, A. (1998). The eyes amygdala response to visual sexual stimuli. have it! Reflexive orienting is triggered by Nature Neuroscience, 7, 411–416. nonpredictive gaze. Psychonomic Bulletin and Hare, B., & Tomasello, M. (2004). Chimpanzees are Review, 5, 490–495. more skilful in competitive than in cooperativeFriesen, C. K., Ristic, J., & Kingstone, A. (2004). cognitive tasks. Animal Behaviour, 68, 571–581. Attentional effects of counterpredictive gaze Haude, R. H., Graber, J. G., & Farres, A. G. (1976). and arrow cues. Journal of Experimental Visual observing by rhesus monkeys: Some Psychology: Human Perception and relationships with social dominance rank. Performance, 30, 319–329. Animal Learning and Behavior, 4, 163–166.Galfano, G., Rusconi, E., & Umilta, C. (2006). Haxby, J. V., Horowitz, B., Ungerleider, L. G., Number magnitude orients attention, but not Maisog, J. M., Peietrini, P., & Grady, C. L. against one’s will. Psychonomic Bulletin and (1994). The functional organization of human Reviews, 13, 869–874. extrastriate cortex: A PET-rCBF study ofGauthier, I., Tarr, M. J., Anderson, A. W., selective attention to faces and locations. Skudlarski, P., & Gore, J. C. (1999). Activation Journal of Neuroscience, 14, 6336–6353. of the middle fusiform face area increases with Hayden, B. Y., Parikh, P. C., Deaner, R. O., & Platt, expertise in recognizing novel objects. Nature M. L. (2007). Economic principles motivating Neuroscience, 2, 568–573. social attention in humans. Proceedings of theGilad, Y., Wiebe, V., Przeworski, M., Lancet, D., & Royal Society: B Biological Science, 274, 1751– Paabo, S. (2004). Loss of olfactory receptor 1756. genes coincides with the acquisition of full Hietanen, J. K., & Leppanen, J. M. (2003). Does trichromatic vision in primates. PLoS Biology, facial expression affect attention orienting by 2, E5. gaze direction cues? Journal of ExperimentalGnadt, J. W., & Andersen, R. A. (1988). Memory Psychology: Human Perception and related motor planning activity in posterior Performance, 29, 1228–1243. parietal cortex of macaque. Experimental Brain Hoffman, K. L., Gothard, K. M., Schmid, M. C., & Research, 70, 216–220. Logothetis, N. K. (2007). Facial-expression andGoldstein, J. M., Seidman, L. J., Horton, N. J., gaze-selective responses in the monkey Makris, N., Kennedy, D. N., Caviness, V. S., amygdala. Current Biology, 17, 766–772. et al. (2001). Normal sexual dimorphism of Holmes, A., Richard, A., & Green, S. (2006). Anxiety the adult human brain assessed by in vivo and sensitivity to eye gaze in emotional faces. magnetic resonance imaging. Cerebral Cortex, Brain and Cognition, 60, 282–294. 11, 490–497. Hori, E., Tazumi, T., Umeno, K., Kamachi, M.,Gothard, K. M., Battaglia, F. P., Erickson, C. A., Kobayashi, T., Ono, T., et al. (2005). Effects of Spitler, K. M., & Amaral, D. G. (2007). Neural facial expression on shared attention responses to facial expression and face identity mechanisms. Pysiology and Behavior, 84, 397–405. in the monkey amygdala. Journal of Hrdy, S. B., & Whitten, P. L. (1987). Patterning of Neurophysiology, 97, 1671–1683. sexual activity. In: B. B. Smuts, D. L. Cheney, R.Grandin, T. (1999). Social problems: Under- M. Seyfarth, R. W. Wrangham, & T. T. standing emotions and developing talents. Struhsaker (Eds.), Primate societies (pp. 370– http://www.autism.org/temple/social. html. 384). Chicago: University of Chicago Press.Grieve, K. L., Acuna, C., & Cudeiro, J. (2000). The Ikeda, T., & Hikosaka, O. (2003). Reward-dependent primate pulvinar nuclei: Vision and action. gain and bias of visual responses in primate Trends in Neuroscience, 23, 35–39. superior colliculus. Neuron, 39, 693–700.

NEUROETHOLOGY OF ATTENTION IN PRIMATES 547Ishai, A., Schmidt, C. F., & Boesiger, P. (2005). Face during viewing of naturalistic social situations perception is mediated by a distributed cortical as predictors of social competence in network. Brain Research Bulletin, 67, 87–93. individuals with autism. Archives of GeneralItakura, S. (1996). An exploratory study of gaze Psychiatry, 59, 809–816. monitoring in non-human primates. Japanese Klin, A., Jones, W., Schultz, R., Volkmar, F., & Psychological Research, 38, 174–180. Cohen, D. (2002b). Defining and quantifyingItakura, S. (2004). Gaze-following and joint visual the social phenotype in autism. American attention in nonhuman animals. Japanese Journal of Psychiatry, 159, 895–908. Psychological Research, 46, 216–226. Knickmeyer, R. C., & Baron-Cohen, S. (2006). FetalJames, W. (1890). The principles of psychology. New testosterone and sex differences. Early Human York: H. Holt and Company. Development, 82, 755–760.Johnson, M. H. (2005). Subcortical face processing. Kobayashi, K. (2001). Unique morphology of the Nature Reviews: Neuroscience, 6, 766–774. human eye and its adaptive meaning:Jolly, A. (1966). Lemur social behavior and primate Comparative studies on external morphology intelligence. Science, 153, 501–506. of the primate eye. Journal of HumanJonides, J. (1981). Voluntary versus automatic Evolution, 40, 419–435. control over the mind’s eye’s movement. In: Kyes, R. C., & Candland, D. K. (1987). Baboon J. B. Long & A. D. Baddeley (Eds.), Attention (Papio hamadryas). visual preferences for and performance IX (pp. 187–203). Hillsdale, regions of the face. Journal of Comparative NJ: Erlbaum. Psychology, 101, 345–348.Kaminski, J., Riedel, J., Call, J., & Tomasello, M. Kylliainen, A., & Hietanen, J. K. (2004). Attention (2005). Domestic goats, Capra hircus, follow orienting by another’s gaze direction in children gaze direction and use social cues in an object with autism. Journal of Child Psychology and choice task. Animal Behaviour, 69, 11–18. Psychiatry, 45, 435–444.Kawashima, R., Sugiura, M., Kato, T., Nakamura, Land, M. F., & Hayhoe, M. (2001). In what ways do A., Hatano, K., Ito, K., et al. (1999). The human eye movements contribute to everyday amygdala plays an important role in gaze activities? Vision Research, 41, 3559–3565. monitoring: A PET study. Brain, 122, 779–783. Langton, S. R. H., & Bruce, V. (1999). ReflexiveKeating, C. F., & Keating, E. G. (1982). Visual scan visual orienting in response to the social patterns of rhesus monkeys viewing faces. attention of others. Visual Cognition, 6, 541–567. Perception, 11, 211–219. Leon, M. I., & Shadlen, M. N. (1999). Effect ofKendrick, K. M., da Costa, A. P., Leigh, A. E., expected reward magnitude on the response of Hinton, M. R., & Peirce, J. W. (2001). Sheep neurons in the dorsolateral prefrontal cortex of don’t forget a face. Nature, 414, 165–166. the macaque. Neuron, 24, 415–425.Keverne, E. B., Leonard, R. A., Scruton, D. M., & Logothetis, N. K., Guggenberger, H., Peled, S., & Young, S. K. (1978). Visual monitoring in social Pauls, J. (1999). Functional imaging of the groups of Talapoin Monkeys. Animal Behaviour, monkey brain. Nature Neuroscience, 2, 555–562. 26, 933–944. Mangun, G. R. (1995). Neural mechanisms ofKirchner, H., & Thorpe, S. J. (2006). Ultra-rapid visual selective attention. Psychophysiology, 32, object detection with saccadic eye movements: 4–18. Visual processing speed revisited. Vision Martin, R. (1990). Primate origins and evolution: A Research, 46, 1762–1776. phylogenetic reconstruction. Princeton:Klein, J. T., Deaner, R. O., & Platt, M. L. (2008). Princeton University Press. Neural correlates of social target value in Mathews, A., Fox, E., Yiend, J., & Calder, A. (2003). macaque parietal cortex. Current Biology, 18, The face of fear: Effects of eye gaze and emotion 419–424. on visual attention. Visual Cognition, 10,Klein, J. T., & Platt, M. L. (2008). Topography of social 823–835. and nutritive reward coding in striatum [Abstract]. McNelis, N. L., & Boatright-Horowitz, S. L. (1998). Program No. 691.8. 2008 Neuroscience Meeting Social monitoring in a primate group: The Planner. Washington, DC: Society for relationship between visual attention and Neuroscience. Online. hierarchical ranks. Animal Cognition, 1, 65–69.Klin, A., Jones, W., Schultz, R., Volkmar, F., & Morris, J. S., Ohman, A., & Dolan, R. J. (1999). A Cohen, D. (2002a). Visual fixation patterns subcortical pathway to the right amygdala

548 PRIMATE NEUROETHOLOGY mediating ’unseen’ fear. Proceedings of the Brain Research: Cognitive Brain Research, 24, National Academy of Sciences, 96, 1680–1685. 715–718.Muller, H. J., & Rabbitt, P. M. (1989). Reflexive and Rizzolatti, G., & Craighero, L. (2004). The mirror- voluntary orienting of visual attention: Time neuron system. Annual Review of Neuroscience, course of activation and resistance to inter- 27, 169–192. ruption. Journal of Experimental Psychology: Romanski, L. M., Giguere, M., Bates, J. F., & Human Perception and Performance, 15, 315–330. Goldman-Rakic, P. S. (1997). TopographicOkamoto-Barth, S., Call, J., & Tomasello, M. (2007). organization of medial pulvinar connections Great apes’ understanding of other individuals’ with the prefrontal cortex in the rhesus line of sight. Psychological Science, 18, 462–468. monkey. Journal of Comparative Neurology,Pelphrey, K. A., Sasson, N. J., Reznick, J. S., Paul, 379, 313–332. G., Goldman, B. D., & Piven, J. (2002). Visual Rosa, M. G., & Tweedale, R. (2005). Brain maps, scanning of faces in autism. Journal of Autism great and small: Lessons from comparative and Developmental Disorders, 32, 249–261. studies of primate visual cortical organization.Perrett, D. I., & Emery, N. J. (1994). Understanding Philosophical Transactions of the Royal Society the intentions of others from visual signals: of London Series B: Biological Science, 360, Neurophysiological evidence. Cahiers de 665–691. Psycholgoie Cognition, 13, 683–694. Sabbagh, M. A. (2004). UnderstandingPeters, R. J., Iyer, A., Itti, L., & Koch, C. (2005). orbitofrontal contributions to theory-of-mind Components of bottom-up gaze allocation in reasoning: Implications for autism. Brain and natural images. Vision Research, 45, 2397–2416. Cognition, 55, 209–219.Platt, M. L., & Glimcher, P. W. (1998). Response Sackett, G. P. (1966). Monkeys reared in isolation fields of intraparietal neurons quantified with with pictures as visual input: Evidence for an multiple saccadic targets. Experimental Brain innate releasing mechanism. Science, 154, Research, 121, 65–75. 1468–1473.Platt, M. L., & Glimcher, P. W. (1999). Neural Sauther, M., Sussman, R., & Gould, L. (1999). The correlates of decision variables in parietal socioecology of the ringtailed lemur: Thirty-five cortex. Nature, 400, 233–238. years of research. Evolutionary Anthropology, 8,Posner, M. I. (1980). Orienting of attention. Quarterly 120–132. Journal of Experimental Psychology, 32, 3–25. Schultz, R. T. (2005). Developmental deficits in socialPosner, M. I., & Cohen, Y. (1984). Components of perception in autism: The role of the amygdala visual orienting. In: H. Bouma & D. Bouwhuis and fusiform face area. International Journal of (Eds.), Attention and performance (pp. 531– Developmental Neuroscience, 23, 125–141. 556). Hillsdale, NJ: Erlbaum. Seltzer, B., & Pandya, D. N. (1989). Frontal lobePrather, J. F., Peters, S., Nowicki, S., & Mooney, R. connections of the superior temporal sulcus in (2008). Precise auditory-vocal mirroring in the rhesus monkey. Journal of Comparative neurons for learned vocal communication. Neurology, 281, 97–113. Nature, 451, 305–310. Seltzer, B., & Pandya, D. N. (1991). Post-rolandicPutman, P., Hermans, E., & van Honk, J. (2006). cortical projections of the superior temporal Anxiety meets fear in perception of dynamic sulcus in the rhesus monkey. Journal of expressive gaze. Emotion, 6, 94–102. Comparative Neurology, 312, 625–640.Raleigh, M. J., McGuire, M. T., Brammer, G. L., Serre, T., Oliva, A., & Poggio, T. (2007). A Pollack, D. B., & Yuwiler, A. (1991). Sero- feedforward architecture accounts for rapid tonergic mechanisms promote dominance categorization. Proceedings of the National acquisition in adult male vervet monkeys. Academy of Sciences, 104, 6424–6429. Brain Research, 559, 181–190. Sewards, T. V., & Sewards, M. A. (2002). Innate visualRaymond, J. E., Fenske, M. J., & Tavassoli, N. T. object recognition in vertebrates: Some proposed (2003). Selective attention determines emo- pathways and mechanisms. Comparative tional responses to novel visual stimuli. Biochemistry and Physiology Part A: Molecular Psychological Science, 14, 537–542. and Integrative Physiology, 132, 861–891.Ristic, J., Mottron, L., Friesen, C. K., Iarocci, G., Seyfarth, R. M., & Cheney, D. L. (2003). Signalers Burack, J. A., & Kingstone, A. (2005). Eyes are and receivers in animal communication. special but not for everyone: The case of autism. Annual Review of Psychology, 54, 145–173.

NEUROETHOLOGY OF ATTENTION IN PRIMATES 549Shepherd, S. V., Deaner, R. O, & Platt, M. L. (2006). triggered by gaze cue? Journal of Comparative Social status gates social attention in monkeys. Psychology, 121, 156–170. Current Biology, 16, R119–120. Tootell, R. B., Tsao, D., & Vanduffel, W. (2003).Shepherd, S. V., & Platt, M. L. (2006). Noninvasive Neuroimaging weighs in: Humans meet telemetric gaze tracking in freely moving macaques in "primate" visual cortex. Journal of socially housed prosimian primates. Methods, Neuroscience, 23, 3981–3989. 38, 185–194. Tsao, D. Y., Freiwald, W. A., Knutsen, T. A.,Shepherd, S. V., & Platt, M. L. (2008). Spontaneous Mandeville, J. B., & Tootell, R. B. (2003). Faces social orienting and gaze following in ringtailed and objects in macaque cerebral cortex. Nature lemurs (Lemur catta). Animal Cognition, 11, Neuroscience, 6, 989–995. 13–20. Tschudin, A., Call, J., Dunbar, R. I., Harris, G., &Shepherd, S. V., Klein, J. T., Deaner, R. O., & Platt, van der Elst, C. (2001). Comprehension of signs M. L. (2009). Mirroring of attention by neurons by dolphins (Tursiops truncatus). Journal of in macaque parietal cortex. PNAS, 106, 9489– Comparative Psychology, 115, 100–105. 9494. Tversky, A., & Kahneman, D. (1981). The framingShimojo, S., Simion, C., Shimojo, E., & Scheier, C. of decisions and the psychology of choice. (2003). Gaze bias both reflects and influences Science, 211, 453–458. preference. Nature Neuroscience, 6, 1317–1322. Ungerleider, L. G., & Mishkin, M. (1982). TwoSkuse, D. (2006). Genetic influences on the neural cortical visual systems. In: D. J. A. Ingle, M. basis of social cognition. Philosophical Goodale, & R. V. W. Mansfield (Eds.), Analysis Transactions of the Royal Society of London of visual behavior. Cambridge, MA: MIT Press. Series B: Biological Science, 361, 2129–2141. van der Geest, J. N., Kemner, C., Camfferman, G.,Smith, A. P. R., Stephan, K. E., Rugg, M. D., & Verbaten, M. N., & van Engeland, H. (2002). Dolan, R. J. (2006). Task and content modulate Looking at images with human figures: amygdala-hippocampal connectivity in Comparison between autistic and normal emotional retrieval. Neuron, 49, 631–638. children. Journal of Autism and DevelopmentalSugrue, L. P., Corrado, G. S., & Newsome, W. T. Disorders, 32, 69–75. (2004). Matching behavior and the represen- van Hoof, J. A. (1967). The facial displays for the tation of value in the parietal cortex. Science, catarhine monkeys and apes. In: D. Morris 304, 1782–1787. (Ed.), Primate ethology (pp. 7–68). Chicago:Swettenham, J., Condie, S., Campbell, R., Milne, E., Aldine Publishing Company. & Coleman, M. (2003). Does the perception of Vuilleumier, P. (2002). Facial expression and moving eyes trigger reflexive visual orienting in selective attention. Current Opinions in autism? Philosophical Transactions of the Royal Psychiatry, 15, 291–300. Society of London Series B: Biological Science, Wassink, T. H., Hazlett, H. C., Epping, E. A., Arndt, 358, 325–334. S., Dager, S. R., Schellenberg, G. D., et al.Tomasello, M., Call, J., & Hare, B. (1998). Five (2007). Cerebral cortical gray matter primate species follow the visual gaze of overgrowth and functional variation of the conspecifics. Animal Behaviour, 55, 1063– serotonin transporter gene in autism. Archives 1069. of General Psychiatry, 64, 709–717.Tomasello, M., Carpenter, M., Call, J., Behne, T., & Yarbus, A. (1967). Eye movements during Moll, H. (2005). Understanding and sharing perception of complex objects. In: Eye intentions: The origins of cultural cognition. movements and vision (pp. 171–211). New Behavioral and Brain Sciences, 28, 675–691; York: Plenum Press. discussion 691–735. Zajonc, R. B. (1968). Attitudinal effects of mereTomasello, M., Hare, B., Lehmann, H., & Call, J. exposure. Journal of Personality and Social (2006). Reliance on head versus eyes in the gaze Psychology, 9, 1–27. following of great apes and human infants: The Zink, C. F., Tong, Y., Chen, Q., Bassett, D. S., Stein, cooperative eye hypothesis. Journal of Human J. L., & Meyer-Lindenberg, A. (2008). Know Evolution, 52(3), 314–320. your place: Neural processing of socialTomonaga, M. (2007). Is chimpanzee (Pan hierarchy in humans. Neuron, 58, 273–283. troglodytes) spatial attention reflexively

CHAPTER 27Neuroethology of Decision MakingDaeyeol LeeDuring the last several decades, anatomical commonly face such problems as mate selectionstudies on the brain of rhesus monkeys have and foraging. Therefore, a set of available matesrevealed various functional maps and their con- or food patches can define a particular problemnectivity in great detail. Guided by this knowl- of decision making. Second, given a set ofedge, neurophysiological studies have also alternative actions, the animal must evaluateuncovered how various features of external sen- the merits of outcomes expected from eachsory stimuli and multiple parameters of move- action and the amount of efforts necessary toments are encoded by the pattern of activity in execute it. It must evaluate not only the qualityensembles of cortical and subcortical neurons. and quantity of expected reward but also theOften, fundamental principles of information likelihood and immediacy of obtaining theprocessing in the sensory and motor systems desired outcome. It is on this second step thathave been found using artificially simple stimuli a great number of theoretical and experimentalor movements. Investigators need to employ, studies have focused (Carmerer, 2003; Kagel &however, more complex stimuli (Ghazanfar Roth, 1995; Kahneman & Tversky, 1979; Lee,et al., 2005; Mazer & Gallant, 2003; Sugihara 2008; von Neumann & Morgenstern, 1944).et al., 2006; Vinje & Gallant, 2000; Woolley et al., Finally, after the animal performs the chosen2005), and allow the animals to produce less action, its actual outcomes must be evaluatedrestricted movements (Georgopoulos et al., 1986; and compared to the expected outcomes.Graziano et al., 2002), in order to gain the insights When they differ, the animals need to adjustnecessary to understand the neural basis of more their expectations for the outcomes from thenatural primate behaviors. chosen actions. Reinforcement learning theory A common element in many natural beha- describes how this can be accomplished com-viors is decision making. Formally, the problem putationally, and its predictions have beenof decision making can be divided into several borne out relatively well in human andsteps. First, a set of alternative actions must be animal experiments (Daw et al., 2006; Harunodelineated. This is convenient both theoretically & Kawato, 2006; Lee et al., 2004; Samejimaand experimentally, and often one considers et al. 2005).only a relatively small number of alternative As in the neurobiological studies of perceptionactions. In practice, it is impossible to know and motor control, nonhuman primates havethe complete set of actions that are available to provided an invaluable model to investigate thea particular animal. Nevertheless, we can focus neural bases for these multiple components ofon interesting cases, by considering a set of decision making. Indeed, neural activity relatedactions that are likely to be chosen in a given to the number and probabilities of alternativecontext and closely related to each other. actions have been identified in multiple areas inFor example, all animals, including primates, the brain, such as the premotor cortex (Cisek550

NEUROETHOLOGY OF DECISION MAKING 551& Kalaska, 2005) and superior colliculus (Basso & produced in a proper order (Tomasello & Call,Wurtz, 1998). In addition, neurons modulating 1997). When reward is delivered after multipletheir activity according to various aspects of movements, the value of a particular movementexpected reward have been identified in many has to be evaluated not only from its immediatedifferent areas, including the amygdala (Paton consequence but also from its long-term benefitet al., 2006), the basal ganglia (Cromwell et al., to the animal. This is commonly referred to as2003; Kawagoe et al., 1998; Samejima et al., 2005), the problem of temporal credit assignmentthe lateral prefrontal cortex (Barraclough et al., (Sutton & Barto, 1998). This chapter sum-2004; Leon & Shadlen, 1999; Watanabe, 1996), the marizes the findings from recent neurophysio-anterior cingulate cortex (Ito et al., 2003; Seo & logical studies designed to address these twoLee, 2007; Shidara & Richmond, 2002), the pos- aspects of decision making in nonhuman pri-terior cingulate cortex (McCoy & Platt, 2005), the mates. Specifically, the results from behavioralsupplementary motor area (Sohn & Lee, 2007), and neurophysiological studies on decisionthe orbitofrontal cortex (Padoa-Schioppa & making during simple competitive games areAssad, 2006; Roesch & Olson, 2004; Tremblay & discussed. The choice behavior of rhesus mon-Schultz, 1999), and the posterior parietal cortex keys in computer-simulated competitive games(Dorris & Glimcher, 2004; Platt & Glimcher, 1999; could be relatively well accounted for by a rela-Sugrue et al., 2004). As predicted by reinforce- tively simple reinforcement learning algorithmment learning theory, signals related to the dis- (Lee et al., 2004, 2005). Single-neuron recordingcrepancy between the expected and actual rewards studies have also shown that multiple corticalhave also been identified in the primate brain. In regions in rhesus monkeys, including the lateralparticular, dopamine neurons in the ventral teg- prefrontal cortex, dorsal anterior cingulatedmental area and substantia nigra pars compacta cortex, and posterior parietal cortex play impor-display phasic activity when the animal receives tant roles in evaluating the outcomes of theunexpected reward and depress their activity animal’s decisions (Barraclough et al., 2004;when the expected reward is omitted (Bayer & Seo & Lee, 2007; Seo et al., 2007, 2009). In addi-Glimcher, 2004; Schultz, 1998), as predicted for tion, the functions of the frontal cortex in thereward prediction error in reinforcement learning selection of movement sequences are discussed.theory (Kawato & Samejima, 2007; Schultz et al., Neural activity in the lateral prefrontal cortex1997; Sutton & Barto, 1998). closely tracks the animal’s knowledge of the cor- Although characterization of neural signals rect movement sequence during both sequencerelated to expected and actual rewards provides planning (Averbeck & Lee, 2007) and sequenceimportant insights into the brain processes execution (Averbeck et al., 2006). Nevertheless,involved in reward valuation and reinforcement multiple brain areas are likely to be involvedlearning, there are two important features of during the evaluation and selection of movementprimate behaviors that deserve special attention. sequences. Medial frontal cortex, for example,First, primates are highly social animals, and it may encode not only the sequence of movementshas been proposed that the complexity of their chosen by the animal but also the immediacy ofsocial structures has played an important role in reward expected after the completion of a parti-the evolution of their relatively large brains cular movement sequence (Sohn & Lee, 2007).(Byrne & Whiten, 1988; Whiten & Byrne, Such signals would be useful in evaluating the1997). Making appropriate decisions in a social desirability of a given movement sequence.group is particularly challenging, because theoutcomes expected from a particular action can DECISION MAKING OFchange frequently depending on the behaviors of NONHUMAN PRIMATES IN Aother animals in the group (Lee, 2008). Second, MATCHING PENNIES GAMEmany natural behaviors in primates, such as tooluse and social interactions, produce desired Decision making in a social group is formallyeffects only after multiple movements are studied by game theory. In game theory, a game

552 PRIMATE NEUROETHOLOGYis characterized by a payoff matrix, which assigns game known as the matching pennies gamea particular reward or payoff to each participant (Fig. 27.1A). This game is played by two playersor player according to the actions chosen by all and is an example of zero-sum games, in whichparticipating players. A strategy to choose a par- the sum of the payoffs to all players is alwaysticular action exclusively is referred to as a pure zero. Each player chooses from two alternativestrategy, whereas a mixed strategy refers to a actions, and one of the players (matcher) wins ifprobability density function defined over a set he or she chooses the same action (e.g., head of aof actions and therefore can describe a strategy coin) and loses otherwise. Although some gamesto choose multiple actions probabilistically. can have multiple Nash equilibria, the matchingTechnically, therefore, a pure strategy is also a pennies game has only one Nash equilibrium,mixed strategy in which the probability is 1 for and this is to choose each of the two actionsone of the alternative actions and 0 for all other with the probability of 0.5. Any other strategiesactions. A main task in game theory is to deter- can be potentially exploited by the opponent.mine how a group of rational players trying to Therefore, the Nash equilibrium in the matchingmaximize their individual payoffs would choose pennies game is a mixed strategy. When thetheir strategies. One of these solutions is known matching pennies game is played repeatedly,as Nash equilibrium, which is defined as a set of the Nash equilibrium is for each player tostrategies from which no individual players can choose each of the two alternative actions withdeviate unilaterally to increase their payoffs the probability of 0.5 and independently across(Nash, 1950). Consider, for example, a simple successive trials.Figure 27.1 Payoff matrix and spatiotemporal sequence of the matching pennies task (A) and rock-paper-scissors task (B).

NEUROETHOLOGY OF DECISION MAKING 553 Despite its mathematical elegance, the equili- condition in which the computer opponentbrium predictions of game theory are often vio- blindly applied the strategy given by the Nashlated for a variety of games (Camerer, 2003), equilibrium in the matching pennies game. Inincluding matching pennies (Mookherjee & other words, the computer opponent chose theSopher, 1994; Rapoport & Budescu, 1992). two alternative targets randomly with the sameInstead, various learning models have been pro- probabilities. This was referred to as algorithm 0.posed to account for the discrepancies between In this condition, the probability that thethe predictions from game theory and the animal’s choice would be rewarded was theobserved choice behaviors of human subjects same for both targets. Therefore, this made it(Fudenberg & Levine, 1998). In particular, rein- possible to examine any initial bias the animalsforcement learning models have been might have had before they were introduced to asuccessfully applied to the data obtained from more competitive setting. Not surprisingly, eachvarious experimental games (Camerer, 2003; of the three animals displayed a strong preferenceErev & Roth, 1998; Feltovich, 2000; Lee et al., to choose one of the targets more frequently. The2004, 2005; Mookherjee & Sopher, 1994, 1997). probability that the animal would choose the To test how closely the choice behaviors of rightward target throughout the period of algo-rhesus monkeys follow the predictions of Nash rithm 0 was 0.70, 0.90, and 0.33 for monkeys C, E,equilibrium in zero-sum games, three male and F, respectively (Lee et al., 2004).rhesus monkeys (monkeys C, E, and F) were In the next stage of the experiment, the com-trained to indicate their choices in a binary puter opponent adopted a more exploitativeoculomotor free-choice task that was modeled strategy by analyzing statistical biases in the ani-after the matching pennies game (Lee et al., mal’s choice sequences and using this informa-2004). In this experiment, the animals were tion to avoid choosing the target that the animalseated in a primate chair and faced a computer was more likely to choose. To this end, thescreen on which all visual stimuli were pre- computer saved the animal’s entire choice his-sented. The animal’s eye position was moni- tory in a given day and estimated the probabilitytored using a scleral search coil (DNI, Newark, that the animal would choose the right-handDelaware) or a high-speed video-based eye target and a series of conditional probabilitiestracker (ET 49, Thomas Recording, Germany). that the animal would choose the right-handEach trial began when the animal fixated a target given the animal’s choices in the lastsmall yellow square presented at the center four trials. Then, for each of these probabilities,of the computer screen (Fig. 27.1A). After a the null hypothesis that the animal chose each0.5-second fore-period, two green choice tar- target with the 0.5 probability was tested. If nonegets were presented along the horizontal mer- of these null hypotheses was rejected, the com-idian. After a 0.5-second delay period, the puter assumed that the animal’s choice was con-central fixation target was extinguished, and sistent with the Nash equilibrium, and chosethe animal was required to shift its gaze each target randomly with the probability oftoward one of these peripheral targets within 0.5. Otherwise, the computer biased its choice1 second. The animal was required to maintain using the conditional probability that was sig-fixation on its chosen target for a 0.5-second nificantly different and deviated maximally fromhold period. At the end of this hold period, the 0.5. For example, if the conditional probabilitycomputer presented a red feedback ring around chosen by this criterion indicated that thethe target it selected, and the animal was animal would choose the rightward target inrewarded only when it chose the same target the next trial with the probability of 0.8, thenas the computer. the computer chose the leftward target with 0.8 To investigate how the animal’s choice beha- probability. This reduced the reward probabilityvior might be influenced by the increasing degree for the animal as its probability of choosing theof exploitation by its opponent, we first examined rightward target deviated further from 0.5.the animal’s behavior during the baseline Therefore, the optimal strategy for the animal

554 PRIMATE NEUROETHOLOGYto maximize its overall reward was to choose the To test whether and the extent to which the animalstwo targets with the same probability, namely, could reduce the use of WSLS strategy when thisthe Nash equilibrium strategy. becomes disadvantageous, the algorithm used by When the computer opponent switched to the computer for its target selection was modified.algorithm 1, the animals adapted quickly and In this new algorithm, referred to as algorithm 2,started choosing the two targets with more or less the computer opponent tested all of the conditionalequal probabilities. Overall, during the period of probabilities used in algorithm 1. In addition, italgorithm 1, the probability that the animal would also tested the conditional probabilities based onchoose the rightward target was within 0.01 from the animal’s choices and their outcomes in the0.5 in all three animals. In addition, the animal’s previous four trials. Therefore, the animal’s biaschoices in successive trials were quite independent. to use the WSLS strategy frequently could beThis was quantitatively examined by calculating detected and penalized in algorithm 2. Forthe entropy of the animal’s choice sequences in example, if the animal used the WSLS strategythree successive trials. Since the animal’s choice in with a 0.9 probability, and if the animal waseach trial corresponds to a binary variable, the rewarded for choosing the rightward target in themaximum entropy for three successive choices is previous trial, the conditional probability that3 bits, corresponding to a case in which the ani- the animal would choose the rightward target inmal’s successive choices are completely indepen- the current trial, given the animal’s choice and itsdent. The average entropy during the period of outcome in the previous trial, would be 0.9. If thisalgorithm 1 was greater than 2.95 for all animals, conditional probability is significantly differentindicating that their choices were nearly indepen- from 0.5 and this deviation is larger than anydent (Lee et al., 2004). In contrast, all three animals other conditional probabilities that were signifi-displayed the tendency to choose the same target cantly different from 0.5, then the computeragain when the previous choice was rewarded and would choose the leftward target with the prob-switch to the other target otherwise. In other ability of 0.9 and the animal would be rewardedwords, they displayed the tendency to use the so- with the probability of 0.1. Therefore, in algorithmcalled win-stay-lose-switch (WSLS) strategy. Since 2, the animal was required to choose the two targetsthe animal was rewarded in approximately 50% of randomly with the same probabilities and indepen-the trials in algorithm 1, using the WSLS strategy dently from its previous choices and theirresulted in switching to the other target in about outcomes.half of the trials. Therefore, a relatively frequent use All three animals tested in both algorithmsof the WSLS strategy was possible without 1 and 2 successfully reduced the probability ofdeviating significantly from the Nash equilibrium WSLS strategy once the computer opponentstrategy or introducing significant correlation in switched to algorithm 2. During the periodsuccessive choices. In addition, the WSLS strategy of algorithm 2, the average probability that thewas not penalized in algorithm 1, since the com- animal would choose its target according to theputer did not analyze the animal’s reward history. WSLS strategy was 0.55, 0.54, and 0.57 for mon-During the period of algorithm 1, the probability keys C, E, and F, respectively (Lee et al., 2004). Allthat the animal would choose its target according of these values were significantly lower comparedto the WSLS strategy was 0.65, 0.73, and 0.63 for to the results for algorithm 1. However, they weremonkeys C, E, and F, respectively. In addition, all still significantly higher than 0.5, suggestingduring the course of the experiments with algo- that the animals did not remove the bias to relyrithm 1, which lasted several months, the prob- on the WSLS strategy completely. Despite thisability of using the WSLS strategy increased small bias, the reward probability was greatersteadily and significantly in all animals. than 0.47 for all animals, which is relatively close The WSLS strategy is a straightforward imple- to the optimal value of 0.5. Therefore, the benefitmentation of Thorndike’s law of effect (Thorndike, of removing the bias for the WSLS strategy com-1911). Thus, the animal’s tendency to use this pletely would have been relatively small, and thisstrategy may be relatively robust and hard-wired. might account for the residual bias.

NEUROETHOLOGY OF DECISION MAKING 555REINFORCEMENT LEARNING IN Otherwise, the value functions are updatedA MATCHING PENNIES GAME according to the difference between the reward received in trial t, Rt, and the reward expected byThe fact that during the matching pennies game the current value functions. In other words:the animals persistently displayed the bias touse the WSLS strategy suggests that they might Qt þ 1 ðCt Þ ¼ Qt ðCt Þ þ a½Rt À Qt ðCt Þ;have used a reinforcement learning algorithm toapproximate the optimal strategy in this game. where Ct (= L or R) indicates the animal’sThe objective of reinforcement learning (Sutton choice in trial t and a corresponds to the& Barto, 1998) is to identify an action at each learning rate. This model has two free para-time step that would maximize the decision meters, a and b, and they were estimatedmaker’s long-term reward. Since the future separately for the data collected in each dailyreward is generally not known, the decision session, using a maximum likelihood proce-maker uses the expected value of the reward, dure (Pawitan, 2001; Seo & Lee, 2007). Thiswhich is referred to as a value function. For the analysis was performed only for algorithmsmatching pennies game, the value functions for 1 and 2, since algorithm 0 was tested only forthe leftward and rightward targets at trial t can a small number of sessions.be denoted as Qt(L) and Qt(R), respectively. The Values of the two parameters in this reinforce-probability that the animal would choose the ment learning model can describe different stra-right-hand target in trial t, Pt(R), is then given tegies that might be adopted by the animal. Forby the soft-max transformation as follows: example, if the animal always chooses its target according to the WSLS strategy frequently, the Pt ðRÞ ¼ exp fbQt ðRÞg=½exp fbQt ðLÞg learning rate would be 1 and the inverse tempera- þexp fbQt ðRÞg; ture would be relatively large. In contrast, a rela-where the inverse temperature, b, determines the tively small learning rate would imply that therandomness of the animal’s choices. The prob- animal’s strategy would change slowly accordingability that the animal would choose the leftward to the outcomes of its previous choices. Indeed,target in the same trial would be 1 À Pt(R). A large consistent with the frequent use of the WSLSinverse temperature implies that the animal strategy in algorithm 1, the learning rate esti-chooses the target with the higher value function mated for the behavioral data obtained with algo-deterministically, whereas a small inverse tem- rithm 1 was relatively close to 1, and this wasperature indicates a relatively stochastic choice significantly reduced in all animals when thebehavior. When the animal receives reward computer switched to algorithm 2 (t test,predicted by the value function for the chosen P <0.01; Fig. 27.2). The process in which theaction, the value functions remain unchanged. parameters of reinforcement learning algorithmsFigure 27.2 Model parameters for the reinforcement learning model fit to the choice behavior during thematching pennies task. Empty circles and small dots correspond to the results from the sessions tested withalgorithms 1 and 2, respectively.

556 PRIMATE NEUROETHOLOGYare adjusted by the long-term reward rate is there is an important difference in the cognitivereferred to as meta-learning (Schweighofer & processes necessary to implement the twoDoya, 2003). Therefore, the results from the learning models. In particular, belief learningexperiments on the matching pennies game models require that the animal maintains ademonstrate that rhesus monkeys applied a mental model for the behaviors of other players.meta-learning algorithm to adjust their choice The predictions of these two classes of learningbehavior to approximate the optimal strategy models diverge more clearly for the rock-paper-during a competitive game (Soltani et al., 2006). scissors game. If a player chooses rock and loses because the opponent chooses paper, a reinforcement learning model would reduce the valueDECISION MAKING AND function for choosing rock, but the value func-REINFORCEMENT LEARNING IN tions for paper and scissors would remainA ROCK-PAPER-SCISSORS GAME unchanged, making the player equally likely toAs described previously, during the matching choose paper or scissors in the next trial. Inpennies game, monkeys tended to choose their contrast, a belief learning model would increasetargets according to the WSLS strategy. Although the decision maker’s estimate for the probabilitythis is largely consistent with a reinforcement that the opponent would choose paper, and thislearning algorithm, it is also consistent with would encourage the player to choose scissors inanother type of learning model referred to as the next trial.belief learning. In reinforcement learning, To test whether monkeys might choose theirif the animal chose a particular target and targets according to the predictions of a beliefwas rewarded, then it is more likely to choose learning model during a zero-sum game, twothe same target again, simply because the same rhesus monkeys were trained to choose amongresponse was rewarded and therefore its associa- three targets that were designated as rock, paper,tion with given sensory stimuli strengthened. and scissors by making an eye movement. As inIn contrast, belief learning theories postulate experiments on the matching pennies, eachthat the decision maker chooses a particular animal was tested in three successive stages. Inaction that would produce the best outcome in algorithm 0, the computer opponent chose eachresponse to the action that the opponent is most of the three targets with the probability of onelikely to choose (Camerer, 2003; Feltovich, 2000; third, which corresponds to the Nash equili-Fudenberg & Levine, 1998; Lee, 2008). If the brium strategy for the rock-paper-scissorsanimal chose the leftward target and was game. In algorithm 1, the computer exploitedrewarded, for example, this indicates that the statistical biases in the animal’s choice history,computer opponent chose the leftward target. If whereas in algorithm 2, it exploited the biases inthe animal believes that the computer would the animal’s choice and reward history. Thechoose the same target in the next trial, the results from these experiments showed that theanimal would also choose the leftward target animal’s choice behavior during the rock-paper-again, not because this action was rewarded in scissors game was largely consistent with a rein-the previous trial, but because this would be the forcement learning model, and there was littlebest response to the predicted behavior of the evidence for belief learning (Lee et al., 2005).computer opponent. This demonstrates that for This is consistent with the findings from pre-the matching pennies game, reinforcement vious studies on experimental games in humanslearning and belief learning models tend to that also failed to find strong evidence for beliefmake the same predictions about the animal’s learning (Erev & Roth, 1998; Feltovich, 2000;behavior, making it difficult to distinguish Mookherjee & Sopher, 1997). However, com-between them empirically. pared to a simple reinforcement learning Although reinforcement learning and belief model, a hybrid model that incorporated thelearning models make similar predictions for features of both reinforcement learning andchoice behavior in relatively simple games, belief learning performed better (Lee et al.,

NEUROETHOLOGY OF DECISION MAKING 5572005), suggesting that these two different types delay period, only a relatively small percentageof learning algorithms might operate in parallel of neurons encoded signals related to the ani-(Camerer & Ho, 1999). mal’s upcoming choice. In the DLPFC, 12.1% and 19.9% of the neurons significantly modulated their activity according to the animal’sNEURAL BASIS OF DECISION choice in the same trial during the fore-periodMAKING IN COMPETITIVE and delay period, respectively. In the ACCd, theGAMES corresponding percentages were 18.2% andTo investigate the neural mechanisms for rein- 17.5%. The difference between the two corticalforcement learning and decision making, we areas was not statistically significant (Fig. 27.3).recorded the activity of 322 neurons from the During the 0.5-second window immediatelydorsolateral prefrontal cortex (DLPFC; after the target onset, many more neuronsBarraclough et al., 2004; Seo et al., 2007) and responded differently depending on the position154 neurons in the dorsal anterior cingulate of the target chosen by the animal. This corre-cortex (ACCd; Seo & Lee, 2007) of monkeys sponds to the time period in which the animalperforming the matching pennies task described produced an eye movement toward one of theabove. These recording experiments were car- two targets. In the DLPFC, 66.2% of the neuronsried out using algorithm 2, namely, while the responded differently according to the animal’scomputer opponent exploited the statistical choice (Fig. 27.4, top), whereas 37.0% of thebiases in the animal’s choice and reward history. neurons in the ACCd showed the same effect.For each neuron, the rates of action potentials or This difference was statistically significantspikes during a series of 0.5-second windows (w2 test, P < 0.05). Therefore, consistent withaligned at the time of target onset or at the the findings from previous studies, ACCd neu-time of feedback ring onset were calculated. rons encoded the spatial aspects of the animal’sThese spike rates were then analyzed using a movements relatively infrequently (Hoshi et al.,linear regression model that included the ani- 2005; Ito et al., 2003; Matsumoto et al., 2007;mal’s choice, the choice of the computer oppo- Seo & Lee, 2007). In contrast, reward-relatednent, and reward in the current and previous signals were encoded more frequently in thethree trials. In other words: ACCd than in the DLPFC. During the 0.5- second window immediately following the feed- St ¼ a0 þ AC ½Ct CtÀ1 CtÀ2 CtÀ3 0 back ring onset, 68.9% of the DLPFC neurons þAP ½Pt PtÀ1 PtÀ2 PtÀ3 0 þAR ½Rt RtÀ1 RtÀ2 RtÀ3 0 ; (Fig. 27.4, bottom) and 81.8% of the ACCdwhere St denotes the spike rate for a given time neurons (Fig. 27.5, bottom) showed significantwindow in trial t; Ct, Pt, and Rt correspond to the modulations in their activity, depending oncomputer’s choice, the choice of the computer whether the animal would be rewarded or notopponent, and the animal’s reward in trial t, in the current trial. During this feedback period,respectively; and AC, AP, and AR are row vectors some neurons in both areas also modulated theirof the corresponding regression coefficients. activity according to the choice of the computerThe statistical significance for each regression opponent. Such neurons were found more fre-coefficient was determined with a t test (P < 0.05). quently in the DLPFC (44.4%) than in the ACCd The results from these regression analyses are (22.7%).summarized in Figure 27.3, which shows the The changes in the activity of many neurons inpercentage of neurons in each cortical area that the DLPFC related to the animal’s choice, thesignificantly modulated their activity in each choice of the computer opponent, and reward0.5-second time window, grouped together in a given trial were often maintained acrossaccording to the behavioral variable (monkey’s multiple trials. For example, the DLPFC neuronchoice, computer’s choice, and reward) and the illustrated in Figure 27.4 modulated its activitytrial lag (0 for the current trial, 1 for the previous significantly during the fore-period and delaytrial, and so on). During the fore-period and period according to the animal’s choice, the

558 PRIMATE NEUROETHOLOGYFigure 27.3 Time course of activity related to the animal’s choice (top), the choice of the computeropponent (middle), and reward (bottom) in the current (Trial Lag=0) or previous (Trial Lag=1–3) trialswithin the population of neurons recorded from the dorsolateral prefrontal cortex (DLPFC) (black) anddorsal anterior cingulate cortex (ACCd) (red). Each symbol indicates the fraction of neurons that displayedsignificant modulations in their activity according to the corresponding variable (t test, P <0.05). Largesymbols indicate that the percentage of neurons was significantly higher than the significance level used inthe regression analysis (binomial test, P <0.05). Asterisk indicates that the difference between the twocortical areas was statistically significant (w2 test, P <0.05). The dotted vertical lines in the left panelscorrespond to the onset of the fore-period, and the gray background the delay (left panels) or feedback (rightpanels) period.computer’s choice, and reward in the previous the ACCd were more likely to encode signalstrial. Overall, during the delay period, 39.8%, related to the animal’s reward history than the18.0%, and 32.9% of the DLPFC neurons DLPFC neurons. For example, during the delayshowed significant modulations in their activity period, 54.6% and 18.2% of the ACC neuronsaccording to the animal’s choice, the computer’s changed their activity significantly according tochoice, and reward in the previous trial, respec- the reward in each of the previous two trials,tively (Fig. 27.3, Trial Lag ¼ 1). A fewer but sig- respectively. A small but significant number ofnificant number of neurons modulated their ACCd neurons displayed signals related toactivity during the delay period according to the whether the animal was rewarded or not threeanimal’s choice and reward two trials before the trials before (Fig. 27.3, bottom).current trial (Fig. 27.3, Trial Lag ¼ 2). Compared The above results clearly demonstrate thatto the DLPFC, significantly fewer neurons in the during the matching pennies game, signals relatedACCd modulated their activity according to the to the previous choices of the animal and itschoices of the animal and the computer opponent opponent, as well as the outcomes of the animal’sin the previous trials. In contrast, the neurons in previous choices, are maintained for several trials

NEUROETHOLOGY OF DECISION MAKING 559Figure 27.4 Activity of an example neuron in the dorsolateral prefrontal cortex (DLPFC) during thematching pennies task. Each pair of small panels displays the spike density functions (convolved with aGaussian kernel, s = 50 ms) estimated relative to the time of target onset (left panels) or feedback onset(right panels). They were estimated separately according to the animal’s choice (top), the computer’s choice(middle), or reward (bottom) in the current trial (Trial Lag=0) or according to the corresponding variablesin three previous trials (Trial Lag=1, 2, or 3). Cyan (black) lines correspond to the activity associated withrightward (leftward) choices (top and middle) or rewarded (unrewarded) trials (bottom). Circles show theregression coefficients from a multiple linear regression model, which was performed separately for a seriesof 0.5-second windows. Filled circles indicate the coefficients significantly different from zero (t test,P <0.05). The dotted vertical lines in the left panels correspond to the onset of the fore-period, and thegray background the delay (left panels) or feedback (right panels) period.in the DLPFC. In contrast, neurons in the ACCd as the eligibility trace (Sutton & Barto, 1998).maintained primarily signals related to the ani- Therefore, signals related to the animal’s pre-mal’s reward history. These findings suggest that vious choices found in the DLPFC might cor-the DLPFC might play a more integrative role in respond to eligibility trace (Seo et al., 2007). Onreinforcement learning, whereas the ACCd might the other hand, the signals related to the com-be more specialized for the evaluation of the puter’s previous choices might be related to theanimal’s choice outcome in the context of pre- value functions for specific actions. During thevious reward history. In order to update the matching pennies game, the likelihood that thevalue functions appropriately for a particular animal would be rewarded for choosing a par-action that resulted in a reward after some ticular target is determined by the probabilitydelay, the brain needs to maintain signals that the computer opponent would choose therelated to previously chosen actions responsible same target. Therefore, the neurons that modu-for the outcome. Such memory signals related lated their activity according to the computer’sto the animal’s previous actions are referred to choice in the previous trial might encode the

560 PRIMATE NEUROETHOLOGYFigure 27.5 Activity of an example neuron in the anterior cingulate cortex (ACCd) during the matchingpennies task. Same format as in Figure 27.4.animal’s expectation that the selection of a parti- NEURAL BASIS OF MULTISTAGEcular target might be rewarded. It is possible, DECISION MAKING ANDhowever, that such signals may not be fully uti- TEMPORAL DISCOUNTINGlized by the animal during the matching pennies Humans have unparalleled abilities to deliberate agame, since overly simple strategies, such as long chain of actions to accomplish their beha-WSLS strategy, could be potentially disadvanta- vioral goals, but to a lesser extent, similar abilitiesgeous to the animal. Finally, the signals related to exist in nonhuman primates (Tomasello & Call,the rewards in the previous trials might be used to 1997). Nonhuman primates are also able toestimate the reward rate. This would be possible switch to a different sequence of actions whenwhen the activity of a given neuron is affected previously successful action sequences are noconsistently by the rewards in multiple trials, as longer appropriate. To investigate the underlyingobserved for some neurons in the ACCd (Seo & neural mechanisms for this flexible selection ofLee, 2007). Some neurons in the ACCd increased action sequences, two rhesus monkeys weretheir activity when the animal was rewarded in trained in an oculomotor sequence selectionthe current trial but decreased their activity if the task (Fig. 27.6; Averbeck et al., 2006). In thisanimal was rewarded in the previous trial. This task, the animal began each trial by fixating apattern was reversed in other neurons in the central target. After a 1-second fore-period, twoACCd. Such neurons modulating their activity targets were presented along the horizontal mer-antagonistically according to the reward in the idian (Fig. 27.6A). When the animal fixated thecurrent and previous trials might encode the correct target, the second pair of targets was pre-temporal difference in the animal’s reward, ana- sented along the vertical meridian. When thelogous to the reward prediction error (Seo & Lee, animal chose the second target correctly, the2007). final pair of the targets was presented. Since

NEUROETHOLOGY OF DECISION MAKING 561Figure 27.6 (A) Spatial layout of the oculomotor sequence selection task. (B) Eight different sequencesused in the same task.there were three successive binary choices, a total of DLPFC neurons. The results showed that sig-of eight possible eye movement sequences were nals related to the correct sequence in the pre-possible (Fig. 27.6B), and one of them was chosen vious block gradually decayed in the DLPFCas the correct sequence in a given block of trials. following a time course quite similar to theThis must be discovered by the animal by trial change in the probability that the animal wouldand error, and the correct sequence was changed choose its target according to the sequence thatrandomly without any cues when the animal exe- was previously correct (Fig. 27.7A). Similarly,cuted the correct sequence 10 times. signals related to the correct sequence in the Activity was recorded simultaneously from a new block appeared in the DLPFC with a timesmall ensemble of neurons in the DLPFC while course similar to the change in the probabilitythe animal performed this oculomotor sequence that the animal would choose its target accordingselection task. Similar to the findings in previous to the new sequence (Fig. 27.7B; Averbeck et al.,studies in the DLPFC (Barone & Joseph, 1989) 2006). In this oculomotor sequence selection task,and medial frontal cortex (Tanji, 2001), many the animal was required to remember the correctneurons displayed movement-related activity sequence during the intertrial intervals, since thethat also depended on the ordinal position and correct sequence was fixed for a block of trials. Itthe sequence in which the movement was was found that the information about theembedded. To investigate how such sequence- correct sequence was encoded in the activity ofspecific movement-related activity evolved dyna- DLPFC neurons during the intertrial intervals.mically after the correct sequences were changed, Furthermore, this was true only when thea decoding algorithm was applied to classify each animal chose the correct sequence in the nextmovement according to the sequence it belonged trial (Fig. 27.7C; Averbeck & Lee, 2007). Theseto, using the activity recorded from an ensemble results suggest that the subjective knowledge of

562 PRIMATE NEUROETHOLOGYFigure 27.7 Encoding of information about action sequences in the dorsolateral prefrontal cortex. (A)Time course of the probability that the animal’s choice would be consistent with the correct sequence inthe previous block (blue) and the corresponding log posterior probability evaluated from the neuralactivity (green). All the movements in a new block before the animal correctly chose all three movementsin the new sequence were analyzed and were distributed evenly along the x-axis so that the first movementin a new block corresponded to 0 and the last movement before the first correct trial corresponded to 1.The lines were obtained by a moving average. (B) Time course of the probability that the animal’s choicewould be consistent with the correct sequence in the new block (blue) and the corresponding log posteriorprobability evaluated from the neural activity (red). (C) The average probability that the sequencedecoded from the ensemble activity recorded in the dorsolateral prefrontal cortex (DLPFC)immediately before and after the fixation onset was the correct sequence in a given block. This wascomputed separately for the trials in which the animal chose the correct (blue) and incorrect sequences(green).the correct action sequence used to guide the animal is rewarded predictably after several move-animal’s individual movements might be repre- ments, the performance improves gradually oversented in the DLPFC. time, suggesting that the animal is more motivated In the natural environment for most animals, to perform the task when the reward is immediateincluding primates, the animals are often rewarded or when the amount of required work is reducedonly after producing many movements. When the (Bowman et al., 1996; Perin, 1943; Sohn & Lee,

NEUROETHOLOGY OF DECISION MAKING 5632006, 2009). Similarly, when people and animals ordinal position tend to be more prevalent inchoose between alternative outcomes, they tend the pre-SMA than in the SMA (Clower &to prefer more immediate reward, even when Alexander, 1998; Shima & Tanji, 2000).they have to forego larger reward available only However, whether such signals related to theafter long delays. Therefore, the preference for a ordinal position might in fact encode the tem-particular reward is reduced according to its delay, poral delay of reward or the effort necessary toand this is referred to as temporal discounting receive reward after multiple movements was(Frederick et al., 2002). Accordingly, information not known. To test this, two rhesus monkeysabout the temporal proximity of reward and the were trained in a manual serial reaction timeamount of work necessary to receive reward must task, in which the ordinal position and thebe encoded in the brain and appropriately inte- number of movements remaining before rewardgrated into the signals related to the subjective were manipulated separately (Sohn & Lee, 2007).value of reward. This is particularly important In each trial, the animal was required to producewhen the animal needs to choose among a large multiple hand movements instructed by a seriesnumber of alternative sequences of actions. In of visual targets (Fig. 27.8). Although the animalparticular, the medial frontal cortex has been was required to make a number of movementsimplicated in the control of animals’ motivation that varied across trials, it was rewarded afterand sequential movements (Rushworth, 2004; capturing the target that appeared in a particularTanji, 2001). The study described next tested location, which was fixed in a block of 20 trials.whether the medial frontal cortex is involved in In addition, the targets stepped through a deter-the subjective evaluation of reward expected from ministic trajectory, which was chosen randomlya sequence of movements. and fixed for a block of trials. Therefore, in most The medial portion of Brodmann area 6 in cases, the animal could determine how manythe primate brain is subdivided into the pre- movements were required in each trial fromsupplementary motor area (pre-SMA) rostrally the position of the first target in a given trialand the supplementary motor area (SMA) (Fig. 27.8). The number of movements in a givencaudally (Matelli et al., 1991; Matsuzaka et al., trial was two, four, six, or eight, and this made1992). Many neurons in both SMA and pre- it possible to manipulate separately the ordinalSMA encode information about specific move- position (OP) of the movement and the numberment sequences, but signals related to the of remaining movements (NRM) that must beFigure 27.8 Spatial layout of the manual serial reaction time task. NRM, number of remainingmovements; OP, ordinal position.

564 PRIMATE NEUROETHOLOGYcompleted before reward delivery. For example, determine whether the activity was significantlythe target shown in the third panel of Figure 27.8 affected by OP and NRM. Many neurons incorresponds to the third target in this trial both SMA and pre-SMA encoded signals(OP = 3) and the animal would be required to related to OP. Overall, 58.8% of the SMA neu-capture two additional targets after it acquires rons and 47.9% of the pre-SMA neuronsthis target (NRM = 2). The same target could showed significant effects of OP. In addition,appear as the first target in another trial more than 70% of the neurons showed signifi-(OP=1), whereas the NRM could be changed cant modulations in their activity according toin another block of trials by changing the posi- the NRM (70.2% and 77.8% for the SMA andtion of the rewarded target. pre-SMA, respectively; Figs. 27.9 and 27.10). Activity recorded from 114 neurons in the Therefore, whereas the proportions of neuronsSMA and 117 neurons in the pre-SMA were encoding the OP and NRM were similar in theanalyzed with a linear regression model to SMA, neurons in the pre-SMA were more likelyFigure 27.9. An example neuron in supplementary motor area (SMA) showing activity negativelycorrelated with number of remaining movements (NRM) Top: Raster plots showing the activityseparately for different movement directions and NRM (0, 2, 4, and 6). Bottom: Spike-density functions(convolved with a Gaussian kernel, s = 40 ms) estimated separately for different NRM and differentmovement directions.

NEUROETHOLOGY OF DECISION MAKING 565Figure 27.10 An example neuron in presupplementary motor area (pre-SMA) showing activity positivelycorrelated with number of remaining movements (NRM). Same format as in Figure 27.9.to encode the NRM than OP. These results play a more important role in the selection ofsuggest that the information about the imme- relatively long movement sequences.diacy of reward or the amount of work andeffort necessary to acquire reward might beencoded in these two cortical areas. In addition, CONCLUSIONcompared to neurons in the SMA, pre-SMA Compared to simple motor responses com-neurons were more likely to increase their monly studied in neurophysiological studies,activity with NRM. In the pre-SMA, approxi- natural behaviors of nonhuman primates oftenmately half of the neurons (47.0%) showed take place in the context of social interactionssignificant positive correlation between their with other conspecifics. The behaviors of non-activity and NRM (Fig. 27.10), whereas this human primates can change more flexibly thanwas true only for 30.7% of the neurons in the other inanimate objects, and therefore thisSMA. This suggests that the pre-SMA might makes it necessary for the animals to monitor

566 PRIMATE NEUROETHOLOGYthe outcomes of their previous actions and REFERENCESmodify their behavioral strategies frequently Averbeck, B. B., & Lee, D. (2007). Prefrontal neuralduring social interactions. Primates are also correlates of memory for sequences. Journal ofcapable of constructing a novel sequence of Neuroscience, 27, 2204–2211.actions to accomplish their behavioral objec- Averbeck, B. B., Sohn, J. W., & Lee, D. (2006). Activitytives. To investigate the basic neural mechan- in prefrontal cortex during dynamic selectionisms involved in social decision making, recent of action sequences. Nature Neuroscience, 9,studies have examined the behavioral dynamics 276–282.during computerized competitive games and Barone, P., & Joseph, J. P. (1989). Prefrontal cortexthe neural mechanisms involved in reinforce- and spatial sequencing in macaque monkey.ment learning. The results from these studies Experimental Brain Research, 78, 447–464.have found that the signals related to the ani- Barraclough, D. J., Conroy, M. L., & Lee, D. (2004).mals’ previous choices and their outcomes are Prefrontal cortex and decision making in a mixed-strategy game. Nature Neuroscience, 7,sustained in the frontal cortex and might con- 404–410.tribute to appropriate revisions of the animals’ Basso, M. A., & Wurtz, R. H. (1998). Modulation ofbehavioral strategies. In addition, studies based neuronal activity in superior colliculus byon a sequence selection task have found that changes in target probability. Journal ofprefrontal cortex encodes signals related to the Neuroscience, 18, 7519–7534.animals’ subjective knowledge of the correct Bayer, H. M., & Glimcher, P. W. (2005). Midbrainaction sequence. In addition, information dopamine neurons encode a quantitativeabout the immediacy of reward or the amount reward prediction error signal. Neuron, 47,of work necessary to acquire reward is repre- 129–141.sented in the medial frontal cortex. However, Bowman, E. M., Aigner, T. G., & Richmond, B. J.signals related to the immediacy of reward and (1996). Neural signals in the monkey ventral striatum related to motivation for juice andthe amount of work or effort necessary to cocaine rewards. Journal of Neurophysiology,acquire reward could not be separated in the 75, 1061–1073.studies described in this chapter. Indeed, Byrne, R. W., & Whiten, A. (1988). Machiavellianwhether these two different types of informa- intelligence: Social expertise and the evolution oftion are processed by different areas in the intellect in monkeys, apes, and humans. Oxford:frontal cortex (Rushworth et al., 2007) remains Oxford University Press.to be investigated more systematically. Future Camerer, C. F. (2003). Behavioral game theory:studies will also need to investigate more closely Experiments in strategic interaction. Princeton:the extent to which different subdivisions of the Princeton University Press.primate prefrontal cortex are functionally spe- Camerer, C., & Ho, T. H. (1999). Experience-cialized to make it possible for the animals to weighted attraction learning in normal form games. Econometrica, 67, 827–874.choose the most appropriate course of action Cisek, P., & Kalaska, J. F. (2005). Neural correlatesin a dynamic and often competitive social of reaching decisions in dorsal premotorenvironment. cortex: Specification of multiple direction choices and final selection of action. Neuron, 45, 801–814.ACKNOWLEDGMENTS Clower, W. T., & Alexander, G. E. (1998). Movement sequence-related activity reflecting numericalI am grateful to Bruno Averbeck, Dominic order of components in supplementary andBarraclough, Michelle Conroy, Ben McGreevy, presupplementary motor areas. Journal ofHyojung Seo, and Jeong-Woo Sohn for partici- Neurophysiology, 80, 1562–1566.pating in the experiments described in this Cromwell, H. C., & Schultz, W. (2003). Effects ofchapter. This research was supported by the expectations for different reward magnitude ongrants from the National Institutes of Health neuronal activity in the primate striatum.(MH059216, MH073246, and NS044270). Journal of Neurophysiology, 89, 2823–2838.

NEUROETHOLOGY OF DECISION MAKING 567Daw, N. D., O’Doherty, J. P., Dayan, P., Seymour, Kahneman, D., & Tversky, A. (1979). Prospect B., & Dolan, R. J. (2006). Cortical substrates for theory: An analysis of decision under risk. exploratory decisions in humans. Nature, 441, Econometrica, 47, 263–292. 876–879. Kawagoe, R., Takikawa, Y., & Hikosaka, O. (1998).Dorris, M. C., & Glimcher, P. W. (2004). Activity in Expectation of reward modulates cognitive posterior parietal cortex is correlated with the signals in the basal ganglia. Nature relative subjective desirability of action. Neuron, Neuroscience, 1, 411–416. 44, 365–378. Kawato, M., & Samejima, K. (2007). EfficientErev, I., & Roth, A. E. (1998). Predicting how reinforcement learning: Computational theories, people play games: Reinforcement learning in neuroscience and robotics. Current Opinions in experimental games with unique, mixed Neurobiology, 17, 205–212. strategy equilibria. American Economic Review, Lee, D. (2008). Game theory and neural basis of 88, 848–881. social decision making. Nature Neuroscience, 11,Feltovich, N. (2000). Reinforcement-based and 404–409. belief-based learning models in experimental Lee, D., Conroy, M. L., McGreevy, B. P., & asymmetric-information games. Econometrica, Barraclough, D. J. (2004). Reinforcement learning 68, 605–641. and decision making in monkeys during aFrederick, S., Loewenstein, G., & O’Donoghue, T. competitive game. Cognitive Brain Research, (2002). Time discounting and time preference: 22, 45–58. A critical review. Journal of Economic Literature, Lee, D., McGreevy, B. P., & Barraclough, D. J. 40, 351–401. (2005). Learning and decision making inFudenberg, D., & Levine, D. K. (1998). The theory of monkeys during a rock-paper-scissors game. learning in games. Cambridge, MA: MIT Press. Cognitive Brain Research 25, 416–430.Georgopoulos, A. P., Schwartz, A. B., & Kettner, R. E. Leon, M. I., & Shadlen, M. N. (1999). Effect of (1986). Neuronal population coding of expected reward magnitude on the response of movement direction. Science, 233, 1416–1419. neurons in the dorsolateral prefrontal cortex ofGhazanfar, A. A., Maier, J. X., Hoffman, K. L., & the macaque. Neuron, 24, 415–425. Logothetis, N. K. (2005). Multisensory Matelli, M., Luppino, G., & Rizzolatti, G. (1991). integration of dynamic faces and voices in Architecture of superior and mesial area 6 and rhesus monkey auditory cortex. Journal of the adjacent cingulate cortex in the macaque Neuroscience, 25, 5004–5012. monkey. Journal of Comparative Neurology,Graziano, M. S. A., Taylor, C. S. R., & Moore, T. 311, 445–462. (2002). Complex movements evoked by Matsumoto, M., Matsumoto, K., Abe, H., & microstimulation of precentral cortex. Neuron, Tanaka, K. (2007). Medial prefrontal cell 34, 841–851. activity signaling prediction errors of actionHaruno, M., & Kawato, M. (2006). Different values. Nature Neuroscience, 10, 647–656. neural correlates of reward expectation and Matsuzaka, Y., Aizawa, H., & Tanji, J. (1992). A motor reward expectation error in the putamen and area rostral to the supplementary motor area caudate nucleus during stimulus-action- (presupplementary motor area). in the monkey: reward association learning. Journal of Neuronal activity during a learned motor task. Neurophysiology, 95, 948–959. Journal of Neurophysiology, 68, 653–662.Hoshi, E., Sawamura, H., & Tanji, J. (2005). Mazer, J. A., & Gallant, J. L. (2003). Goal-related Neurons in the rostral cingulate motor area activity in V4 during free viewing visual search: monitor multiple phases of visuomotor Evidence for a ventral stream visual salience behavior with modest parametric selectivity. map. Neuron, 40, 1241–1250. Journal of Neurophysiology, 94, 640–656. McCoy, A. N., & Platt, M. L. (2005). Risk-sensitiveIto, S., Stuphorn, V., Brown, J. W., & Schall, J. D. neurons in macaque posterior cingulate cortex. (2003). Performance monitoring by the Nature Neuroscience, 8, 1220–1227. anterior cingulate cortex during saccade Mookherjee, D., & Sopher, B. (1994). Learning countermanding. Science, 302, 120–122. behavior in an experimental matching penniesKagel, J. H., & Roth, A. E. (1995). The handbook of game. Games and Economic Behavior, 7, 62–91. experimental economics. Princeton: Princeton Mookherjee, D., & Sopher, B. (1997). Learning and University Press. decision costs in experimental constant sum

568 PRIMATE NEUROETHOLOGY games. Games and Economic Behavior, 19, Seo, H, Barraclough, D. J., & Lee, D. (2009). Lateral 97–132. intraparietal cortex and reinforcement learningNash, J. F. (1950). Equilibrium points in n-person during a mixed-strategy game. Journal of games. Proceedings of the National Academy of Neuroscience, 29, 7278-7289. Sciences, 36, 48–49. Seo, H., & Lee, D. (2007). Temporal filtering ofPaton, J. J., Belova, M. A., Morrison, S. E., & Salzman, reward signals in the dorsal anterior cingulate C. D. (2006). The primate amygdala represents the cortex during a mixed-strategy game. Journal of positive and negative value of visual stimuli during Neuroscience, 27, 8366–8377. learning. Nature, 439, 865–870. Seo, H, & Lee, D. (2009). Behavioral and neuralPadoa-Schioppa, C., & Assad, J. A. (2006). Neurons changes after gains and losses of conditioned in the orbitofrontal cortex encode economic reinforcers. Journal of Neuroscience, 29, value. Nature, 441, 223–226. 3627–3641.Pawitan, Y. (2001). In all likelihood: Statistical Shima, K., & Tanji, J. (2000). Neuronal activity in the modelling and inference using likelihood. supplementary and presupplementary motor areas Oxford: Oxford University Press. for temporal organization of multiple move-Perin, C. T. (1943). A quantitative investigation of ments. Journal of Neurophysiology, 84, 2148–2160. the delay-of-reinforcement gradient. Journal of Sohn, J.-W. & Lee, D. (2006). Effects of reward Experimental Psychology, 32, 37–51. expectancy on sequential eye movements inPlatt, M. L., & Glimcher, P. W. (1999). Neural monkeys. Neural Networks, 19, 1181-1191. correlates of decision variables in parietal Sohn, J. W., & Lee, D. (2007). Order-dependent cortex. Nature, 400, 233–238. modulation of directional signals in theRapoport, A., & Budescu, D. V. (1992). Generation supplementary and presupplementary motor of random series in two-person strictly areas. Journal of Neuroscience, 27, 13655–13666. competitive games. Journal of Experimental Shidara, M., & Richmond, B. J. (2002). Anterior Psychology: General, 121, 352–363. cingulate: Single neuronal signals related toRoesch, M. R., & Olson, C. R. (2004). Neuronal degree of reward expectancy. Science, 296, activity related to reward value and motivation 1709–1711. in primate frontal cortex. Science, 304, 307–310. Soltani, A., Lee, D., & Wang, X. J. (2006). NeuralRushworth, M. F. S., Behrens, T. E. J., Rudebeck, P. mechanism for stochastic behavior during a H., & Walton, M. E. (2007). Contrasting roles competitive game. Neural Networks, 19, for cingulate and orbitofrontal cortex in 1181–1191. decisions and social behaviour. Trends in Sugihara, T., Diltz, M. D., Averbeck, B. B., & Cognitive Science, 11, 168–176. Romanski, L. M. (2006). Integration of auditoryRushworth, M. F. S., Walton, M. E., Kennerley, S. and visual communication information in the W., & Bannerman, D. M. (2004). Action sets primate ventrolateral prefrontal cortex. Journal and decisions in the medial frontal cortex. of Neuroscience, 26, 11138–11147. Trends in Cognitive Science, 8, 410–417. Sugrue, L. P., Corrado, G. S., & Newsome, W. T.Samejima, K., Ueda, Y., Doya, K., & Kimura, M. (2004). Matching behavior and the (2005). Representation of action-specific reward representation of value in the parietal cortex. values in the striatum. Science, 310, 1337–1340. Science, 304, 1782–1787.Schultz, W. (1998). Predictive reward signal of Sutton, R. S., & Barto, A. G. (1998). Reinforcement dopamine neurons. Journal of Neurophysiology, learning: An introduction. Cambridge, MA: MIT 80, 1–27. Press.Schultz, W., Dayan, P., & Montague, P. R. (1997). Tanji, J. (2001). Sequential organization of multiple A neural substrate of prediction and reward. movements: Involvement of cortical motor areas. Science, 275, 1593–1599. Annual Review of Neuroscience, 24, 631–651.Schweighofer, N., & Doya, K. (2003). Meta- Thorndike, E. L. (1911). Animal intelligence: learning in reinforcement learning. Neural Experimental studies. New York: Macmillan. Networks, 16, 5–9. Tomasello, M., & Call, J. (1997). Primate cognition.Seo, H., Barraclough, D. J., & Lee, D. (2007). Oxford: Oxford University Press. Dynamic signals related to choices and Tremblay, L., & Schultz, W. (1999). Relative reward outcomes in the dorsolateral prefrontal cortex. preference in primate orbitofrontal cortex. Cerebral Cortex, 17, i110–i117. Nature, 398, 704–708.

NEUROETHOLOGY OF DECISION MAKING 569Vinje, W. E., & Gallant, J. L. (2000). Sparse coding Whiten, A., & Byrne, R. W. (1997). Machiavellian and decorrelation in primate visual cortex intelligence II: Extensions and evaluations. during natural vision. Science, 287, 1273–1276. Cambridge: Cambridge University Press.von Neumann, J., & Morgenstern, O. (1944). Woolley, S. M. N., Fremouw, T. E., Hsu, A., & Theory of games and economic behavior. Theunissen, F. E. (2005). Tuning for spectro- Princeton: Princeton University Press. temporal modulations as a mechanism forWatanabe, M. (1996). Reward expectancy in primate auditory discrimination of natural sounds. prefrontal neurons. Nature, 382, 629–632. Nature Neuroscience, 8, 1371–1379.

CHAPTER 28Out of Our Minds: The Neuroethology ofPrimate Strategic BehaviorLouise Barrett and Drew RendallINTRODUCTION Cartesian view that animals were mere auto- mata, and embrace the notion that the behaviorPrimates, as an order, are marked by large brains of nonhumans was underpinned by cognitiveand considerable behavioral flexibility, which is processes of varying degrees of flexibility andperhaps most developed in the social domain. In complexity. Evolutionarily speaking, such afact, the unusually intense sociality of the pri- move also dispensed with the idea that a pro-mates, compared to other taxa, sparked a profound discontinuity existed between human andposal, now firmly entrenched, that the primate nonhuman animals, picking up on Darwin’sniche is largely a social one where selection pres- idea that the difference was likely to be one ofsures favoring increasingly complex social stra- degree and not of kind (see Penn et al., 2008, fortegies are responsible for the markedly larger a review and critique of this stance). In fact, inbrains of primates (e.g., Dunbar, 1998; recent years, an explicitly anthropocentric, orHumphrey, 1976; Jolly, 1966). Furthermore, even anthropomorphic, program of compara-there is a natural assumption that, as our closest tive research has (re)emerged and been defendedliving relatives, the monkeys and apes have as scientifically legitimate and expressly licensedsomething special to tell us about the evolution by this evolutionary perspective (e.g., De Waal,of our own cognitive capacities as the culmina- 2001, 2005).tion of such processes. The programs of com- Such a stance has no doubt been helped alongparative cognition that these proposals have by the fact that much of primate cognitivesparked have been exciting and highly produc- ethology has attempted to combine the studytive, but we also feel that, as often practiced, they of mechanism with that of evolutionary func-promote a certain view of the brain and cognition, where, following Dennett’s influential pro-tion that, increasingly, is leading us astray. posal, intentional language has come to be usedIn what follows, we articulate these concerns, frequently as a metaphorical short-hand to char-and suggest how we might deal with them. acterize the nature of adaptive behavior (i.e., the so-called ‘‘free-floating’’ rationale [of mother nature]; Dennett, 1989). Dennett’s intentionalOUT OF OUR MINDS gambit was initially only a practical one—Following the ‘‘cognitive revolution’’ and the namely, without easy access to the underlyingloosening of the stranglehold of behaviorism psychology of animals’ behavior, the field ofon comparative psychology, the study of non- ethology could nevertheless make progress onhuman minds became respectable again the adaptive functions of behavior on the(e.g., Griffin, 1978). One could abandon the assumption that, like humans, animals behave570

OUT OF OUR MINDS 571‘‘as if’’ they have goals, beliefs, and desires about continuity in other cognitive domains (e.g.,the world that guide their engagement with it. transitivity, numerosity, causality) on theHowever, this gambit also makes it all too easy assumption that, if one could demonstrateeither to side-step the issue of determining the continuity in language, the sine qua non ofproximate mechanisms that actually do produce humanity, then surely it would be found inbehavior or to favor an argument in which beha- almost every other psychological domain.vior that looks similar to our own, and that The fact that, as subsequent work demon-achieves a similar functional goal, can, by an strated, primate vocalizations lack the flexibleappeal to evolutionary parsimony, be assumed production needed to respond to or influenceto be produced by the same intentional psycho- the intentional states of others has tended tological processes (e.g., De Waal, 1997). Even undercut these inferences, as does the lack ofwhen not expressed in such overtly anthropo- unequivocal evidence to suggest that primatesmorphic terms, a focus only on continuity can represent either their own intentional statesbetween human and nonhuman minds, without or those of others (Cheney & Seyfarth, 2005;equal attention to the difference, can lead, and Penn et al., 2008). The many attempts to teachhas led, to a rather narrow focus, where only language to captive apes provide corroborativethose questions that speak to this intentional results: While ape subjects have shownconception of action are tackled, and possible impressive learning capacities and, in somealternatives are seldom even considered cases, can use symbols effectively, their use of(Owren & Rendall, 2001). artificial languages remains strikingly instru- Studies of primate communication are a mental and egocentric. Given that human lan-case in point. Seminal work by Seyfarth and guage and meaning hinge on the flexible,colleagues (1980) on vervet monkeys docu- intentional nature of communications (Grice,mented a small repertoire of alarm calls spe- 1957), it is difficult to conceive of nonhumancific to different classes of predator, each call communication as being language-like in anyprompting a functionally different escape meaningful, psychological sense. We are thusresponse appropriate to the predator encoun- forced to confront the distinction between thetered. The calls clearly functioned as if they successful pragmatic application of ‘‘as if’’ rea-stood for, or represented, those predators in soning that uses language-based metaphors tothe same way our own human words for these explain the evolutionary function of vocaliza-animals do, and these functional similarities tions in primates and a principled under-were taken to reflect deeper cognitive simila- standing of the underlying psychologicalrities, rooted in language-like representational mechanisms that actually govern their vocaland intentional processes in both groups. behavior.These findings were thus a timely confirmation As the debate regarding the degree to whichof Griffin’s (1978) instinct that the key to the an anthropocentric approach to nonhuman cog-cognitive revolution in animal psychology lay nition is justified rumbles on in various placesin the natural communication behaviors of and in various guises, our aim here is not toanimals that would offer privileged insights rehearse it all again. Rather, our aim is to pointinto animal minds, and, more specifically, out that there is a more pervasive anthropo-reveal continuity with human minds in pro- centrism that drives much current research,portion to the language-like properties of their and it is one that seems to go almost entirelycommunications. The vervet monkey research unnoticed. Specifically, the underlying assump-also then naturally spawned a generation of tion of all work in comparative cognition is thatsimilar studies focused on identifying addi- the cognitive load is borne entirely by the brain,tional language-like properties in primate and that it does so by using information proces-communications (e.g., semantics and syntax; sing routines operating on and over some kindreviewed in Hauser et al., 2002; Zuberbuhler, ¨ of internal representational structure. Or, to put2000). It also encouraged research into it more simply, it is assumed that the brain is a

572 PRIMATE NEUROETHOLOGYcomputer that controls the body and tells it what and without hesitation, both scientifically andto do. colloquially (‘‘I was so shocked I couldn’t pro- As is well known, the computer metaphor of cess what she was saying’’; ‘‘That much informa-the brain rose to prominence during the cognition will fry my brain’’). This has then led totive revolution of the 1950s. Prior to this, it was similar reasoning about nonhuman minds sothe telephone exchange that served as our best that, for example, claims are made for themetaphor for the brain (Draaisma, 2000). The ‘‘propositional’’ (Seyfarth & Cheney, 2005) andbrain has also variously been compared to an ‘‘hierarchically structured’’ (Bergman et al.,abbey, cathedral, aviary, theater, and ware- 2003) nature of their knowledge about thehouse, as well as a filing cabinet, clockwork social world.mechanism, camera obscura, phonograph, and There is no doubt that the computer meta-railway network, as well as Locke’s ‘‘blank slate’’ phor of the brain has been, and remains, valu-and Socrates’ ‘‘wax tablet’’ (Draaisma, 2000). able; it clearly helped to rescue experimentalThe use of a computer metaphor is simply the psychology from the intellectual sterility ofmost recent in a long line of metaphors that behaviorism (Miller, 2003), and there is also nopick up on the most advanced and complex doubt that work in classical artificial intelligencetechnology of the day. This observation, in has had its successes. These successes have, how-itself, should make us at least a little sceptical ever, been somewhat limited. While the classicalabout whether we really have finally hit upon approach has, finally, generated a computer thatthe correct metaphor for understanding biolo- can beat the world chess champion and helpedgical cognitive systems, as opposed to one that to design a range of so-called ‘‘expert systems,’’ itmerely reflects something about the time in has so far failed to give us any real insight intowhich we now currently live. the mechanisms that underlie more natural What should really arouse our suspicions forms of intelligence, such as how we recognizeregarding its appropriateness is that, as Dreyfus a face in a crowd, how we coordinate and manip-(1992), Brooks (1999), and, more recently, ulate all the objects necessary to make cup ofPfeifer and Bongard (2007) have all pointed coffee, or even something as apparently simpleout, the original artificial intelligence project as how we (and other animals) manage to walk,that arose out of the cognitive revolution (good run, or hop over uneven ground without fallingold-fashioned artificial intelligence, or GOFAI) flat on our faces.was strongly anthropocentric, where the focus of The problem here should be obvious. Ourresearch rested squarely on some peculiarly metaphor of the brain, and hence of cognitivehuman aspects of intelligence, like natural lan- processes, is one that was originally derived fromguage, formal reasoning, planning, mathe- a heavily anthropocentric focus on a few peculiarmatics, and playing games like chess—that is, cognitive achievements, all of which involvedtasks that involved the manipulation and pro- abstract symbol manipulation. Moreover, thesecessing of abstract symbols in a logical fashion. processes are ‘‘all talk and no action’’: Chess isUnfortunately, this initial, rather arbitrary not a very athletic pastime, nor is solving logicalemphasis on logical, algorithmically based tasks syllogisms. With the key to cognition seen purelygained such momentum, aided and abetted by as internal symbol manipulation, the role of thethe Church-Turing thesis and the sway of func- body and the environment was sidelined com-tionalism, that we are now in a position where pletely, so that brains became entirely divorcedbrains are not seen as merely analogous to com- from both the bodies they inhabit and the worldputer-like logical reasoning devices, but are con- that they encounter (Dreyfus, 1992). Cognitionsidered to actually be a form of computer. has, in turn, become a process that has no realPsychological processes are therefore assumed link to the body or the outside world, takingto be algorithmic and to involve various kinds place purely in the head, hermetically sealedof ‘‘computation’’ and ‘‘information proces- from reality. In this sense, it is a view that, insing.’’ Indeed, we use such metaphors freely fact, remains very strongly Cartesian (Rockwell,

OUT OF OUR MINDS 5732005; Wheeler, 2005). We have then taken this artificial intelligence (AI) approach because,strange view of cognition—the disembodied, in the real world, there is no such thing aslogical manipulation of internal representa- abstract, disembodied cognition. Moreover, ations—and applied it to other animals, despite more holistic perspective forces the recogni-the fact that it does not even adequately explain tion that natural selection has elaborated onmost facets of natural human cognition. nervous systems as a whole, and not the brainConsequently, we have used a flawed, dualist alone. The nervous system that lies outside themodel of human cognition to model many brain is not merely a network of ‘‘messageother, potentially very different species of cogni- cables’’ that relay the commands of the braintion in ways that can only serve to compound back and forth to the body (e.g., Churchland,our initial error. What were we thinking? 1986) but an integrated, functional system thatClearly, it seems, we were out of our minds. includes the brain, but is not superseded by it, and that is integrated into both the endocrine and the immune systems of the body (e.g.,OUT OF THE HEAD AND INTO Gershon, 1998; Pert, 1997).THE WORLD Once we reject our singular focus onSo, what to do? How can we approach the brains, we automatically reject a secondstudy of neuroethology and cognition in a aspect of the ‘‘classical’’ Cartesian perspective,way that avoids this misplaced anthropocentric which assumes that as ‘‘thinking things’’ weprojection? Our suggestion is that we should contact the world only indirectly as detachedattempt to get out of our minds more produc- observers, where the problem of knowledgetively by embracing models of cognition that has famously been one of how we gain accesstake into account the ‘‘loopy’’ integration of to this world and those of others. As bothbrain, body, and world (Brooks, 1999; Clark, James (1890) and Dewey (1929) pointed out,1997; Dreyfus, 1992; Pfeifer & Bongard, 2007; this view amounts to nothing more than aPfeifer & Scheier, 1999; van Gelder, 1995). This false dichotomy between body and brain,requires that we take seriously the notion that organism and environment. The problem ofnatural cognitive systems are ‘‘embodied’’ knowledge, then, is how patterns of organism-(indeed, it makes no sense to refer to any environment interaction can be adapted andother kind of cognition once we leave the con- transformed to deal with the new problemsfines of the computer science laboratory). That and challenges that such interaction itself cre-is, cognitive systems evolved to enable animals ates continually (Johnson & Rohrer, in press).to deal effectively with and engage the worlds This position is not one that currently informsthat they encounter, but not necessarily to comparative primate social cognition. Insteadreflect upon that world in ‘‘thoughtful’’ disem- of viewing behavior as constitutive of mentalbodied ways (Barrett & Henzi, 2005; Barrett states and intentionality, as James (1890) andet al., 2007; Brooks, 1999; Clark, 1997; Dewey (1929) argued it should, comparativeGibson, 1979). This effectively turns our studies typically view behavior merely as a cueanthropocentric perspective on its head. Once to (or symptom of) the hidden mind, whichwe recognize, as Brooks (1999) has argued so we can only know indirectly by inference. Thiscogently, that all animals possess bodies and is completely understandable in the context ofthat all did so long before they possessed any- a view that sees cognition as a wholly internal,thing remotely recognizable as a brain, then we purely brain-based enterprise, but this viewcome to realize also that most of evolutionary ultimately limits studies of animal social cog-history has been spent refining the perception nition to those that can provide the kind ofand action mechanisms that enable survival in controlled environment needed to see thisa dynamic world. One cannot, therefore, study ‘‘pure brain’’ in action and infer the structurecognition in the abstract, disembodied way of the ‘‘hidden mind.’’ This, in turn, is likelypromoted by proponents of the classical to lead to an ever more distorted view of the

574 PRIMATE NEUROETHOLOGYcognitive abilities of other animals for two house rather than trying to remember a ‘‘safer’’interlinked reasons. storage spot for them. In fact, this habit is all The first of these can be summed up by what pervasive (now and almost certainly in prehis-Clark (1989: 64) refers to as the ‘‘007 principle’’: tory as well) and it underscores just how much ofthat, generally speaking, ‘‘evolved creatures will routine human cognition may be largely percep-neither store nor process information in costly tual and enacted in the context of environmentalways when they can use the structure of the supports (Clark, 1997). Consider, for example,environment and their operations on it as a how one can suddenly ‘‘remember’’ a telephoneconvenient stand-in for the information proces- number only when actually looking at the phonesing operations concerned.’’ In other words, and about to dial, or how one can fail to recallas James Bond would have had it, animals the name of a favorite old song, yet a few intro-should operate on a ‘‘need to know basis’’; they ductory notes can bring the full melody andshould know no more than they need to in order lyrics flooding back. Consequently, treating theto get the job done. Clark (1989) illustrates this primates (and other creatures, includingprinciple by reference to filter-feeding sponges humans) as if their cognition can be understoodthat exploit water currents to reduce the amount independently of an understanding of theirof pumping they need to do and to mole crickets environment is likely to produce both an incom-that construct underground tunnels that func- plete account—given that some of the cognitivetion as Klipsch horns and thereby increase the load may well be borne by the environment andvolume of their mating calls. Cognitive systems not by the animal—and, more worryingly, anshould be no different to these other physiolo- inaccurate one, given that, as Clark (1989: 66)gical systems. In this respect, the stigmergic sys- suggests, we risk placing into ‘‘the modelledtems of the social insects—the pheromone trails head what nature leaves to the world.’’of ants, for example, or the pheromone-impreg- This leads to the second reason why wenated soil balls that form the building blocks of ignore the body and world at our peril: If wetermites’ intricate and elaborate mounds—are assume that an organism’s behavior arises as aclear examples of how natural selection has dis- consequence of purely internal, brain-based cog-covered the simple trick of throwing things out nitive processes, we may mistakenly attributeinto the world for reperception, rather than more complexity to the organism than is war-storing, processing, and retrieving internal ranted, and overestimate the cognitive require-representations, and in so doing, achieving ments of certain tasks. The ‘‘cognition’’ we see,amazing feats of what, from our frame of refer- therefore, may be entirely illusory, reflectingence, looks like highly ‘‘intelligent’’ behavior. only our own frame of reference and not that Indeed, humans may be the masters of this of the animal itself (Brooks, 1999; Pfeifer &reperception approach to cognition, where a Bongard, 2007). A perfect illustration of thisgreat deal of daily behavior, and much of the is provided by Maris and te Boekhorst’smaterial culture developed to support it, reflects (1996) work on the collective heap-buildinga habit of offloading to the environment for behavior of simple robots (didabots) (see alsosubsequent easy reperception the details of Pfeifer & Scheier, 1999, for a review of this andwhat would otherwise be cognitively demanding other work).tasks. Simple, oft-cited examples include scrib- When placed into an arena in which smallbling shopping and other to-do lists for later cubes are scattered randomly, the robots moveconsultation rather than trying to commit around clustering the cubes together, so thatthem to memory, doing multiplication and eventually there are usually only two large clus-long division by hand on paper where you can ters of cubes, with a few cubes left here and theresee the work in simple steps rather than doing it against the walls. The robots are thereforeall ‘‘in your head’’, and leaving your keys on the known as ‘‘Swiss robots,’’ because their aim,window ledge by the front door where you are apparently, is to tidy up the arena. At abound to encounter them on your way out of the minimum, then, one would suppose that the

OUT OF OUR MINDS 575robots are programmed with mechanisms for in the same way as those off to the side, whichdetecting objects, pushing them in a given direc- means that no pushing behavior occurs. Here,tion toward other objects, and then clustering then, an interpretation of the didabots’ behaviorthem together. In fact, the robots have only one as reflecting the operation of a proximate ‘‘clus-kind of sensor, for proximity, and are pro- tering’’ mechanism, reflecting their ‘‘goal’’ ofgrammed with one simple control rule: If sen- cleaning the arena, would be entirely erroneous:sory information is received on the left, then both inaccurate and incomplete.turn right, and if sensory information is received This example should give us pause whenon the right, turn left. In other words, the robots we attempt to attribute particular proximateare programmed exclusively to avoid obstacles. ‘‘goals,’’ and their supporting cognitive mechan-Why then does clustering occur? To understand isms, to biological organisms. It also highlightsthis, we have to move beyond the robots’ the fact that the proximate means by which ainternal structure and consider the specific behavior is produced need bear no relation tonature of its embodiment and its interaction the form that behavior takes (who would ima-with the environment. gine that a rule for object avoidance would be a A didabot is a small wheeled robot, shaped good way to produce object clustering?), andsomewhat like a toy car. In Maris and te that there is no necessary, nor simple, one-to-Boekhort’s experiment, they were fitted with one mapping between the complexity of a prox-two sensors at the front end of the body, at an imate mechanism and the complexity of theangle. Consequently, as the robots moved forbehavior that it produces. If we take on boardward, they would detect cubes off to the side, but the lessons that work in robots and artificial lifenot straight in front of them. This meant that, can offer us, then it is clear that greater recogni-while they turned away and avoided cubes on tion is needed of the fact that, unlike classicaleither side, a cube directly in front of them computer scientists, natural selection is uninter-would end up being pushed along, because the ested in a ‘‘pure’’ computational brain and is,didabot couldn’t ‘‘see’’ it (i.e., its sensors would instead, focused on the kinds of emergentreceive no stimulation from it). If the didabot ‘‘mind’’ that brains, bodies, and environmentsthen encountered another cube off to the side, can achieve in concert. A biological cognitivetriggering its sensor, it would produce avoidance system, therefore, consists not of a brain, but ofbehavior, moving off to the left or right, leaving a brain within a body that acts in an environ-the object it had just been pushing next to the ment, and it is also a dynamical system whereobject it had just avoided. In other words, the each component is simultaneously influencing,didabot clustered the two cubes together. This and is influenced by, all the other componentsalternation in the environment then increased (Van Gelder, 1995).the chance that another cube, being blindlypushed around, would also be deposited in the EMERGENT MINDS:vicinity, because the cluster of two cubes was ‘‘INTELLIGENCE WITHOUTmore detectable than a single cube. In other REPRESENTATION’’words, a simple self-organizing process canexplain the didabots’ behavior, which results in The idea that cognition or, rather, psychologicalthe formation of ever larger clusters, and gener- processes in general are emergent phenomenaates the abiding impression in those watching that depend crucially on the interaction of anthat the didabots are ‘‘trying’’ their best to ‘‘clean animal with its world was recognized long ago byup.’’ Moreover, the clustering behavior is ¨ Von Uexkull (1957) and his notion of theentirely dependent on the placement of the sen- ‘‘umwelt’’; that an animal’s representation ofsors on the didabot’s body: Move one of the the world (if indeed it represents it at all) willsensors around to the front, and the clustering be shaped by, and grounded in, the means bybehavior disappears entirely, because now which it perceives and acts in it. These percep-objects directly in front of the robot are avoided tion-action mechanisms, in turn, reflect the kind

576 PRIMATE NEUROETHOLOGYof body it has and the kind of ecological niche it example, cannot be divorced from its function,occupies. More recently, J.J. Gibson (1979) took from what we can do with it, because sucha similar approach, which he characterized as possibilities for action are built directly into‘‘ecological psychology,’’ to get across the idea our perceptions (Gabarini & Adenzato, 2004).that psychological phenomena were to be found When we consider mirror neurons from thisin organism-environment relations, and not in perspective (those that fire both when an indivi-the organism alone. While Gibson’s views have dual acts on an object and when he or she merelyreceived rather a bad press, being seen as both observes another individual acting on it), we canstrongly antimentalist and antirepresentational easily see how the perception of other indivi-(see Reed, 1996, for a review and response to duals will generate similar affordances, andsuch criticisms), this is only a crude caricature of hence the possibilities for action that other indi-a more nuanced position, which argued that viduals allow. Indeed, Catmur and colleaguesrepresentational systems are not mental phe- (2007) have demonstrated recently that, innomena alone, but also ways of behaving and humans, ‘‘reverse mirror’’ contingencies can beregulating action in the world (Reed, 1996). induced by training (i.e., participants are trainedMore important, it is the false separation of to perform one action by watching another),organism and environment that is denied by suggesting that mirror properties are neitherecological psychologists, rather than the exis- innate nor fixed once acquired but, instead,tence of mental representations per se. In this, develop through sensorimotor learning. Asit becomes clear how ecological psychology Catmur and colleagues (2007) conclude, thisembraces the earlier views of James (1890) and implies that mirror systems are both a productDewey (1896, 1929) and anticipated many of the and a process of social interaction. This, asinsights that come from more recent work in Gallese (2001, 2003, 2005, 2007; Gallese et al.,robotics and artificial life (Brooks, 1999; Clark, 2004) has argued persuasively, provides us1997; Pfiefer & Bongard, 2007; Wheeler, 2005). with an implicit, unconscious, and automatic A most important aspect of an ecological mechanism for engaging others as goal-directedapproach, from our perspective, is the idea agents, without the need for any overt mentalthat organisms regulate their behavior with state understanding. Most significantly of all, therespect to the ‘‘affordances’’ of the environ- theory of affordances provides a natural meansment. Affordances are the opportunities and of grounding this knowledge in the worldpossibilities for action that particular objects because it recognizes the essentially dynamicaland resources offer to an animal (Gibson, and ‘‘loopy’’ nature of behavior, something that1979; Reed, 1996). In this way, perception is Dewey (1896) made clear in his classic paper‘‘written in the language of action’’ (Michaels refuting the notion of the reflex ‘‘arc.’’ That is,& Carello, 1981) so that, for example, we do not behavior is not the result of a one-way link thatsee ‘‘chairs’’ as such, but places to sit; a chim- goes from stimulus to response, as the behavior-panzee doesn’t see a ‘‘tree,’’ but a place to climb. ists assumed, but is best characterized as a con-As we have argued in detail elsewhere (Barrett & tinuous loop where action generates perceptualHenzi, 2005; Barrett et al., 2007), the discovery feedback that, in turn, generates further action,of the mirror and canonical neuron systems of and so on. Nothing remains internal to thethe primate brain offer neurobiological support organism because action in the world is alwaysfor a theory of affordances. Canonical neurons, an integral part of its conception of the world.for example, which fire both when an object isacted on by an individual and when an object is APPLYING THE CONCEPTS:merely observed, illustrate how our perceptual PRIMATE SOCIAL AFFORDANCESrepresentations of objects contain motor as wellas sensory components, and hence blur the dis- The manner in which baboons engage in certaintinction between perception and conceptual forms of social interaction suggests that aknowledge: Our perception of a cup, for stronger, more ethological focus on the

OUT OF OUR MINDS 577affordances that other individuals offer (i.e., the back as the handler approaches, forcing theopportunities for certain forms of action that handler to begin circling round the mother inothers may or may not pick up and act on) will order to keep the mother’s face, and the infant,pay real dividends in understanding the basis of in view. If mothers cannot keep the handler outprimate strategic behavior and its likely proxi- of their own field of view in this way, then theymate underpinnings. When a female becomes a simply do not glance or gaze in the handler’smother, for example, the presence of her new direction. As long as this situation persists,infant generates intense interest in other females, handlers appear paralyzed and unable to act:who will try to approach and handle the infant. They will circle around the edge of the apparentThey will not always do this gently, and some- zone of tolerance, but do not approach anytimes might make off with the infant altogether, closer. Occasionally, they may emit contactputting it at risk of dehydration and, in extreme grunts of a characteristic kind. Eventually, acases, death (Silk et al., 2003). Mothers, not potential handler will drift away from thesurprisingly, display agitation and avoidance mother if she remains motionless and does notwhen others approach and attempt to handle gaze in the handler’s direction. If, on the othertheir offspring. Grooming the mother, however, hand, she does gaze toward the handler, or evenapparently increases her tolerance for this kind orient her head in that direction, this apparentlyof interaction, with the level of grooming depen- affords the handler the opportunity to approach.dent on the number of infants present in the Handlers may not always do so, of course, butgroup. The latter effect presumably reflects the approaches tend to occur only if mothers afforddegree of stress a mother is experiencing: the opportunity to do so in this way.A single infant in a troop means a high level of Once the handler is in proximity, motherssocial attention, and consequently females may may again turn away from the handler. This isrequire more grooming before its effects especially likely to be the case if the mother’s(in terms of endorphin release) are felt (Barrett glance in the handler’s direction was inadvertent& Henzi, 2006; Henzi & Barrett, 2002). (e.g., the mother oriented in that direction in This grooming in exchange for handling has, response to a call from another animal, orhowever, been described in terms of more cog- because the mother apparently failed to noticenitive processes, where it represents a potential the initial approach of a handler). Once in proxi-case of ‘‘tactical deception’’ on the part of hand- mity, and with the mother turned away, pre-lers, who approach and groom in order to dis- senting her back, handlers more often than notguise their true intention and lull the mother begin to groom. Seen in the context of the inter-into a feeling of false security (given as example action as a whole, this does not seem to be a ploy#88 in Byrne & Whiten, 19901). Careful analysis to deceive the mother, but occurs because theof video-taped interactions of this nature handler is in an aroused state, is socially moti-(Barrett & Henzi, unpublished data) indicate vated, and is afforded the opportunity to groom.that most handling events, in fact, follow a spe- Infant handling, then, is more about two indivi-cific sequence that speaks more to behavioral duals ‘‘bringing their behavior into coordina-attunement of the kind Gallese (2005) espouses, tion’’ in a real-time dynamic fashion (Johnson,rather than any form of truly intentional or even 2001), and not about the execution of preformed‘‘functional’’ deception. goals and intentions that exist in one or other of Initially, a female will approach a mother the animal’s heads.and, particularly if lower ranking, will sit at a This emphasis, as Johnson (2001) furtherdistance of 2 to 3 m, a distance that places a argues, is at the heart of a distributed approach,female sufficiently out of reach of any form of where cognition emerges from, and during,aggressive attack. Once in this position, poten- ongoing interaction. In this, it is quintessentiallytial handlers monitor the mother closely, ecological, since the psychological phenomenafollowing her head orientation and gaze. are a function of animal-environment relations,Mothers, for their part, will often turn their rather than of individual animals alone. From a

578 PRIMATE NEUROETHOLOGYpractical perspective, adopting a distributed, less rival. Such options require a high degree ofrepresentational approach also means that we visual contact with others, a significant amountare able to move away from using terms like of behavioral coordination, and, therefore, suffi-‘‘intend’’ and ‘‘understand,’’ which refer to invi- cient experience with other animals to engagesible mental states that can only be inferred, to and deflect them successfully.descriptors like ‘‘anticipate’’ and ‘‘notice,’’ On the sleeping cliffs, however, these optionswhich are both observable and definable (e.g., were constrained by topography. The height andanticipation can be measured by an increased narrowness of the cliffs resulted in changes in thereadiness to respond or a reduced lag time for mobility of males, their proximity to other ani-certain behaviors; noticing can be defined by mals, and reduced overall visibility. All of thesedirection of gaze, or the relative timing of sig- factors served to reduce older males’ ability tonaling behaviors: Johnson, 2001). direct the situation socially while at the same Strum and colleagues (1997) similarly have time, they favored the more direct, aggressiveused an analysis of consort takeovers by male approach of younger males. In other words, itbaboons to illustrate how much of the behavior was the change in topography that was the key tothat is often held up as an example of success of younger males on the cliffs, not any‘‘Machiavellian intelligence’’ (i.e., as sophisti- specific change in male tactics as such. Malescated cognitive strategizing to achieve a specific were employing those behaviors in their reper-goal) may actually be the result of how particular toire that were afforded by the environment,animals are either constrained or afforded rather than developing entirely novel means tocertain courses of action by the environment. displace rivals. This example illustrates that, inMale baboons socially and sexually monopolize addition to the social affordances offered byadult females during their fertile periods, other individuals, the affordances of the physicalremaining in close proximity to them at all environment can also bring individuals’ beha-times. These close spatial relationships vior into coordination or prevent this from hap-(‘‘consortships’’) can last from a few hours to a pening, so that similar situations play out inweek, depending on the specific population of entirely different ways depending on how thebaboons. Among East African populations, these environment serves to structure the event.consortships are often disrupted by aggression In both baby handling and consort takeovers,from other males, who then take over the con- then, we can see how easy it is to attribute ansort male’s position. There are various social animal’s actions to a fairly complex social cal-options that males can employ to either avoid culus, based on rich internal representationalor facilitate a takeover. Strum and colleagues mechanisms that enable the animal to assess its(1997) focus on one they call ‘‘sleeping near the options and respond accordingly. Once we takeenemy.’’ This designation was based on the the physical environmental into account, how-observation that, while older males were able to ever, or pay closer attention to the actual sequen-resist consort takeover attempts by younger, cing of interactions between animals, we can seemore aggressive males during the day, they that baboons may be rather more like didabotswere less able to do so at sleeping sites, where and rather less strategically Machiavellian thanyounger males were able to displace them and we have assumed. This is not to deny the com-leave with the female in the morning. Strum and plexity of primate social behavior. Instead, itcolleagues (1997) demonstrated that a change in suggests that any such complexity is as an emer-topography from the plains, where the animals gent property of the entire cognitive system, andforaged during the day, to the cliffs, where they not of an individual animal’s brain in isolation.slept at night, was the key factor leading to this As such, we should at least entertain the hypoth-difference. Older, socially experienced males esis that the complex and flexible social behaviorcould resist takeover on the plains, diverting on display may be the product of a dynamicaggression by grabbing a younger animal and interaction between a set of simple social rou-interposing it between himself and his male tines (e.g., biological motion detection, gaze

OUT OF OUR MINDS 579detection-following mechanisms, sensitivity to function to incline behavioral responses infacial expressions) with a particular social and listeners through the relatively direct effectsphysical environment. In this sense, behavior that specific signals have on the entire nervouscan be considered ‘‘strategic’’ really only from system of the listener, especially by tappinga functional, evolutionary point of view. From low-level perceptual, attentional, and motiva-a proximate, mechanistic perspective, the ways tional processes that guide a listener’s basicin which animals coordinate, control, and engagement with the world. And the lattermanage their own and others’ behavior may effects are importantly modulated both by themore productively be treated as contingent short-term motivational and response biasesresponses to the affordances of an ongoing that the immediate social and environmentalsituation. Breazeal (2002), for example, has context prompts in the listener and by theshown how such simple social routines can longer-term dispositional biases the listenerproduce highly engaging, emotionally invol- has as a result of an accumulated history ofving social interactions between Kismet, her experience in the same and other contexts.social robot, and human interactants, under Among other things, this emphasis can explaincircumstances where the humans know there why certain classes of calls, like alarm calls, allis ‘‘no one home’’ in Kismet’s head. The very share a suite of basic acoustic featurespower of her analysis comes, in fact, from the (specifically those that capture listener atten-identification that ‘‘social intelligence’’ may tion and induce a startle response that ishave more to do with the smooth regulation obviously functionally appropriate for pre-and management of social interactions via var- dator avoidance) even if they function ulti-ious kinds of affect induction than with pre- mately to identify different classes of predatordicting and manipulating others’ mental states. and where, if calls were truly arbitrary referents for those different predators, one would certainly expect selection to have produced muchAFFECT INDUCTION greater differentiation in the calls in order toPrimate vocalizations also seem to fit this kind increase their discriminability. Consequently, ifof affect-induction interpretation much better the aim of vocalizations is, like baby handlingthan representationalist theories, which argue and consort takeovers, to manage and influ-that vocal signals literally ‘‘encode informa- ence the behavior of other individuals, andtion’’ that must then be ‘‘decoded’’ by receivers not to change their mental states per se, thenwho then ‘‘decide’’ how to respond based on there is no need to argue that such signals havethe received information. Indeed, the latter ‘‘meaning’’ in the standard representationalapproach falls precisely into the trap offered sense. Instead, as Owren and Rendall (2001)by classic disembodied cognitivist thinking, argue, we can take a more embodied perspec-where abstract information in one animal’s tive, and investigate how signals that act athead is transmitted via the conduit of a vocal these low levels of neural organization help tosignal straight into another’s. Here, the vocal shape behavior. This can occur both directly, assignal is then the only necessary vehicle for when a signal has an unconditioned impact onmessage transmission and the weight of effec- the perceiver’s nervous system (again thetive communication is borne entirely by repre- acoustic startle response is a prime example),sentational processes inside the head of the and indirectly, where vocalizations or visualsignaler and the listener. In contrast, Owren signals are linked with affectively significantand Rendall (1997, 2001; Owren et al., 2003) outcomes experienced by the perceiver via con-suggest that vocalizations often act not as ditioning processes. For example, dominantrepresentational information packets, but as animals often pair their threat vocalizationscomponents of a process of affect induction with actual aggression. As a consequence, sub-where the form of the signal itself plays an ordinates learn the contingency between theimportant role. In this view, vocal signals two, with the result that, in future, the

580 PRIMATE NEUROETHOLOGYvocalizations alone can produce an avoidance and sequencing of behavior all require closeresponse in subordinates (thus sparing the attention to what animals actually do and howdominant the cost of actual aggression). they do it, and how this varies with context. Such processes can also work affiliatively, as As such, it forces us to consider how an animal’swhen individuals grunt softly upon approach, sensory and motor systems actually enable it toprior to grooming or attempting infant hand- engage with the world, and the actions andling. By learning to associate such grunts with a opportunities this affords it.positive (or at least neutral) outcome, animals For primates, the social environment mightcan bring their behavior into coordination more indeed create some special challenges, but not inrapidly and will do so more readily. An affect- the sense that is traditionally assumed. For anyinduction account can also work to the advan- individual in a social species, a group consists oftage of ‘‘socially impotent’’ animals as well. The a large number of other individuals of variousharsh, noisy screams that baboons produce ages and sexes that afford varying opportunitieswhen under attack, for example, are highly aver- for engagement, coordination, competition, andsive stimuli, suggesting that subordinate ani- avoidance. A constantly changing social envir-mals, otherwise physically powerless, are onment thus naturally affords a shiftingnevertheless attempting to make themselves less landscape of opportunities and ‘‘on-the-fly’’appealing as objects of aggression (Owren & responses to them. But these responses needRendall, 1997, 2001). Animals may also regulate not result from internalized, strategic abstrac-their own affect through the use of vocalizations. tions. Rather, they can exist in moment-by-For example, the ‘‘contact calls’’ that baboons moment attunement and adjustment of move-and many other social primates make when ment and activity that bring actors into and outseparated from the rest of their troop are per- of various fields of engagement. So, to the extenthaps better characterized as ‘‘distress calls’’ that that primates have a particular social prowess, itreflect an animal’s stress at being alone and is as likely to lie in the manner in which theirunable to see familiar conspecifics. The fact perception-action routines are organizedthat such lost calls are most frequently around constantly shifting social perceptions.‘‘answered’’ by the animal itself, as opposed to The flexibility and variability of behavior thenindividuals within the body of the troop, argues arises as animals attempt to keep their percep-for a primarily self-regulatory, rather than tions of themselves and others on an even keel indirectly communicative, function (see Rendall the face of perturbations of various kindset al., 2000, for more information). (‘‘perceptual control theory’’: Cziko, 2000). This raises the possibility that the proximate basis for primate social engagement may be aREDEFINING SOCIAL form of nonanalytic pattern recognition: TheINTELLIGENCE social world is clearly structured in ways thatThe approach we have offered here, which powerful associative learning principles canemphasizes how animals can tap directly into exploit. Specifically, the juxtaposition of parti-the perceptual and action systems of others, cular individuals with particular behaviors withand how this can produce predictable physiolo- particular contexts can form the basis for angical responses, provides us with a way out of the association matrix from which individuals cananthropocentric bind in which comparative pri- distill higher-order associations that enablemate studies are situated currently. An under- them to respond appropriately to novel situa-standing of the manner in which one individual tions. Landauer and Dumais (1997), forcan manipulate and manage the way it is per- example, have shown how precisely this kind ofceived by others, how it perceives another’s process (singular value decomposition followedactions, how its own actions can induce parti- by dimension reduction) can produce neuralcular states of arousal or affect, and how these networks that comprehend written text, startingprocesses affect the subsequent coordination from only an association matrix of experienced

OUT OF OUR MINDS 581words and the contexts in which they occurred. mass of water molecules forming a block of iceIf associative processing can produce language or the formation of a complex termite moundcomprehension—that most exalted of human is an emergent property of individual termites’abilities—there seems no reason why similar stigmergic responses. The macroscopic propertiesassociative processes could not produce the of these neural nets, which are known as attrac-latent relational dimensions among individuals, tors, are capable of sustaining patterns of stablebehaviors, and contexts that would give the activity that could function as a form of pattern-functional appearance of sophisticated, strate- recognizing associative memory (Cossart et al.,gizing representational cognition while being 2003; Fuster, 2003; Goldberg, 1988; Hopfield,driven wholly by the perception of concrete 1982). Recognition of an external event has,observables. This kind of emphasis, then, can therefore, been suggested to occur when sensoryexplain the manifest hypersociality of the mon- input resonates with previously formed attractorskeys, in particular, and their attunement to (adaptive resonance theory, or ART; Grossberg,others, but also their abject failure in social cog- 1987).nitive tasks that test what they know about As attractors could potentially be distributedothers’ minds. It suggests that, in fact, they over different, widely distributed cortical areas,don’t know or understand others in the theo- input from different sensory modalities could berized sense of attributing internal belief states. combined in a single attractor, while the fact thatInstead, they ‘‘know’’ others in a far more they emerge as a result of experience means thatconcrete and procedural sense, as different they could be tuned to an animal’s specificperceptions to control. encounters with other individuals (and other This, in turn, suggests that what matters objects in the world). Over time, then, animalsmost, in terms of a social intelligence, is ‘‘exper- would acquire a variety of attractors or antise’’ or ‘‘wisdom’’ acquired by experience, ‘‘energy landscape’’ (Freeman, 2001) thatrather than any explicit Machiavellian ability to would allow them to recognize a wide varietythink several steps ahead. This is because in of similar kinds of situations and generate anorder to produce an appropriate response and appropriate response to them. Knowing whatcontrol one’s perceptions, one must have a suffi- to do next in a particular situation would, there-ciently large association matrix from which to fore, be part and parcel of pattern recognitionderive these kinds of second-order associations, with respect to the situation as a whole. This mayand such a matrix will obviously take time to explain why animals often act appropriately,build up. This can then explain why social despite never having learned a particular contin-learning and early socialization are so crucial gency or encountered a specific situation: beha-for normal adult functioning (e.g., Suomi & viors that Rumbaugh and Washburn (2003;Harlow, 1975), and why different social milieus see also Rumbaugh et al., 1996) refer to,and developmental backgrounds can influence appropriately enough, as ‘‘emergents.’’the degree to which individuals are socially The great advantage of this kind of computa-skilled as adults (e.g., Strum, 1982, 1994). tional neuroscience over classic AI research is Further support for emergent pattern recog- that empirical neurophysiological data seem tonition as the basis for primate social intelligence bear out these findings. For example, Waltercomes from computational neuroscience, which Freeman’s meticulous studies of the rabbit olfac-has shown that formal neural nets often show tory bulb indicate that the learning of differentemergent properties (Grossberg, 1988; Hopfield, odorants is a nonlinear, self-organizing process,1982). That is, while each neuron itself has only involving chaotic attractors (see e.g., Freeman,elementary properties and the network itself has 2001; Freeman & Skarda, 1990a,b; Skarda &little structure, the collective activity and interac- Freeman, 1987). Similarly, Scott-Kelso (1995)tions of these neurons give rise to emergent argues that the brain is fundamentally a self-macroscopic properties, in much the same way organizing pattern-forming dynamical system,that translucence is an emergent property of a perched on the brink of chaos. As importantly,

582 PRIMATE NEUROETHOLOGYFreeman and Skarda (1990a,b) describe brain- EXPLAINING BIG BRAINSbased evidence to support a ‘‘loopy’’ theory ofperception and action. The neural activity pat- An obvious objection to the kind of associativeterns seen are sensitive not simply to the pre- account we are promoting here is that it is com-sence of an odorant or to the response, but to the putationally inelegant. Compared to the repre-interaction of both of these with each other sentational alternative, the associative accountalong with the specific background context in would be much more memory intensive, com-which the behaviour is embedded. As Freeman putationally extensive, and therefore highlyand Skarda (1990a,b) argue, this is best charac- demanding of processing capacity. On reflec-terized as a dialectic between organism and tion, however, this proves not to be an objectionenvironment: The context in which these pat- at all. As we have suggested elsewhere (Barrettterns of neural activity are seen is defined as et al., 2007; Rendall et al., 2008), the empiricalmuch by the behavior of the animal and the data to be explained are the vast amounts ofexisting attractor/energy landscape that is pre- neural tissue that primates possess compared tosent as it is by the specific odorant involved other mammals, and not the complicated cogni-(Freeman & Skarda, 1990a,b). It should be tive structures they use to solve their problems.readily apparent that there is nothing intrinsi- There are, in fact, no empirical data to supportcally representational about such a dynamic pro- the existence of the latter; rather, they are merelycess because one cannot identify any stable entity inferred from the vast quantities of brain tissuethat could stand for something else in the stan- and the complexity of behavior seen. But, as wedard symbolic, computational sense (van have seen with the didabots, for example, there isGelder, 1995). In addition, these data indicate no necessity for complex behavior to requirethat animals are not passive recipients of envir- complex cognitive mechanisms, nor does aonmental stimulation; they do not sit there large and complex mass of neural tissue necessa-waiting for input like the computer on one’s rily imply a more complicated cognitive archi-desk and then simply react to it. Instead, percep- tecture. Furthermore, there is no requirementtion begins with internally generated neural pro- that natural selection must favor neuro-cesses that, by reafference, lay the groundwork computational efficiency per se, no matter howfor the processing of future receptor input congenial such efficiency might be in an expla-(Freeman & Skarda, 1990a,b) so that animals natory sense in our theories of cognition.can be the active initiators of their engagement Selection is, of course, free to favor whateverwith the world. Interestingly, Skarda and mechanisms get the job done relative to alter-Freeman (1990) point out that Merleau-Ponty native local solutions. Consequently, we need to(1942) had already reached this conclusion more consider far more seriously the possibility thatthan 50 years ago, when he argued that the taxonomically widespread associative, nonana-passive, reflex-based view of physiological func- lytic modes of cognition can account for thetioning was merely an illusion created by our broad sweep of primate cognition as well.attempts to understand the brain as a mechan- Viewed properly in the context of the actualical device. However, his view that brain func- data to be explained—the manifestly large, inter-tion was creative, selective, and holistic, and that connected, massively parallel processing brainsthere existed internally generated, global states that primates actually possess—the computa-of cortical activity that resisted reductionist tional inelegance and resulting tissue profligacyexplanations, was rejected as unscientific of such mechanisms prove not to be a weakness(Skarda & Freeman, 1990). The work of neuro- but a strength.physiologists, like Freeman, is now beginning tobear out Merleau-Ponty’s revolutionary CONCLUSIONideas, suggesting that we should focus muchmore seriously on his particular kind of In his seminal paper on social intelligence,‘‘biophilosophy’’ (Skarda & Freeman, 1990). Nicholas Humphrey (1976) likened social life to

OUT OF OUR MINDS 583a game of chess, where predicting in advance of one’s own local perception-action manage-what one’s partner was likely to do would give ment on those of others, and vice versa.an advantage by thwarting such action before it Our conclusion, then, is that to get the com-could have a detrimental impact on oneself. It parative project back on track, we should stopshould be clear from our arguments here that attempting to tap into abstract knowledge insideHumphrey’s analogy is wrong: It is too GOFAI, an animal’s head via field studies and experi-too ‘‘cerebral,’’ too slow, and too disembodied to ments that merely massage our own folk-psycho-capture the real-life social engagement of the logical rationalizations about how primate mindsprimates. While we generally eschew metapho- work. Instead, we should start taking ethologyrical heuristics in theorizing, if pressed for an much more seriously, and tap into what primatesalternative to Humphrey’s analogy for primate (and other organisms) are actually doing in thesociality, we might opt for a different sort of world, and how they do it. By the same token, wegame—the jigsaw puzzle. In doing a jigsaw need to take neuroethology more seriously, bypuzzle there is almost no value in planning which we mean taking it quite literally; we needahead or representing the completed picture to ground our ethological studies by reference tothroughout. Instead, jigsaw puzzles are all about the work of those neurophysiologists, likelocal pattern recognition and completion. You Freeman, who have begun to get a grip on whatsimply start by joining complementary pieces (at least a small part of) the brain actually does,and thereafter you respond dynamically based and how it does it. In other words, we need toon incipient local patterns. Your moves are in understand more about the brain’s behavior inno sense planned or strategic. Rather, they are the ethological sense (even though we may nevergenerated entirely on the fly and then only as understand its processes in strictly causal terms;contingent responses to local perceptual affor- Freeman & Skarda, 1990a,b), as a way of con-dances—that is, by which of the remaining straining our theories of primate cognition. Inpieces are suggested by emerging, embryonic this way, we are less likely to forget that the brainforms. The solution is also then entirely emergent is also a part of the body, and that its structurebased on the bottom-up accretion of ultimately and dynamics will affect how an animal canconjoined but locally generated patterns. It is engage with the world in the same way thatthen a game that places a very high premium other anatomical structures do. Put simply, weon dynamic perception-action integration. In need to forget computer-based cognitive meta-the words of David Kirsh (1991: 169), it is ‘‘per- phors and focus on brain and behavior in a wayceptually hard but intellectually simple.’’ The that allows us to escape our anthropocentricirony is, of course, that this is also exactly how frame of reference. In other words, we will needgrand masters play chess: They recognize pat- to get out of our minds as much (or more) as weterns, having learned to distinguish those that will need to get deeper into them.require one kind of response from those thatrequire another; there is no analysis or compar- ACKNOWLEDGMENTSison of alternatives, but instead a kind of fluidanticipation, as they notice the right things and Thanks to Peter Henzi for reading an earlierrespond at the right moment. In other words, draft of this chapter and to John Vokey forgrand masters play chess like the rest of us solve many enlightening discussions of the issuesjigsaw puzzles, by acting in the world, on the fly, raised here. Our work is supported by theand in the moment. So too, we say, might pri- National Sciences and Engineering Researchmates meet the challenges posed by their social Council of Canada (NSERC).groups—through dynamic, contingent responsesto a landscape of shifting social perceptions. Animportant difference is simply that their game NOTEnever ends. The social landscape is perpetually 1. Every day, one can see females approachfluid, driven by the infinite, iterative ripple effects mothers, pretend to be primarily interested in

584 PRIMATE NEUROETHOLOGY grooming the mother when what they are really Cziko, G. (2000). The things we do: Using the lessons after is an opportunity to sniff, touch or hold of Barnard and Darwin to understand the what, her infant . . . . But is the mother really deceived? how and why of our behaviour. Cambridge, MA: Surely the multiparous ones know exactly what s MIT Press. going on? (#88), Byrne and Whiten (1990: 41). De Waal, F. B. M. (1997). Are we in anthropodenial? Discover, 18, 50–53. De Waal, F. B. M. (2001). The ape and the sushiREFERENCES master. New York: Basic Books. De Waal, F. B. M. (2005). Animals and us:Barrett, L., & Henzi, S. P. (2005). The social nature Suspicious minds. New Scientist, 186, 48. of primate cognition. Proceedings of the Royal Dennett, D. (1989). The intentional stance. Society, 272, 1865–1875. Cambridge, MA: MIT Press.Barrett, L., & Henzi, S. P. (2006). Monkeys, markets Dewey, J. (1896). The reflex arc in psychology. and minds. In: P. Kappeler & C. P. van Schaik Psychological Review, 3, 357–370. (Eds.), Cooperation in primates and humans: Dewey, J. (1929). Experience and nature. New York: Mechanisms and evolution (pp. 209–232). Dover. Berlin: Springer-Verlag, Berlin. Draaisma, D. (2000). Metaphors of memory: ABarrett, L., Henzi, S. P., & Rendall, D. (2007). Social history of ideas about the mind. Cambridge: brains, simple minds: Does social complexity Cambridge University Press. really require cognitive complexity? Philosophical Dreyfus, H. (1992). What computers still can’t do. Transactions of the Royal Society, Series B, 362, Cambridge, MA: MIT Press. 561–575. Dunbar, R. I. M. (1998). The social brainBergman, T.J., Beehner, J.C., Cheney, D.L. & hypothesis. Evolutionary Anthropology, 6, Seyfarth, R.M. (2003) Hierarchical classification 178–190. by rank and kinship in baboons. Science, 302, Freeman, W. J. (2001). How brains make up 1234–1236. their minds. New York: Columbia UniversityBreazeal, C. L. (2002). Designing sociable robots. Press. Cambridge, MA: MIT Press. Freeman, W. J., & Skarda, C. A. (1990a).Brooks, R. (1999). Cambrian intelligence: The early Representations: Who needs them? In: history of the new A.I. Cambridge, MA: MIT J. L. McGaugh, N. Weinberger, & G. Lynch Press. (Eds.), Brain organization and memory cells,Byrne, R. W., & Whiten, A. (1990). Tactical systems and circuits (pp. 375–380). New York: deception in primates: The 1990 database. Oxford University Press. Primate Report, 27, 1–101. Freeman, W. J., & Skarda, C. A. (1990b). Mind/Catmur, C. Walsh, V., & Heyes, C. (2007). brain science: Neuroscience on philosophy of Sensorimotor learning configures the mind. In: E. Lepore and R. van Gulick (Eds.), human mirror system. Current Biology, 17, John Searle and his critics (pp. 115–127). 1527–1531. Oxford: Blackwell.Cheney, D. L., & Seyfarth, R. M. (2005). Fuster, J. (2003) Cortex and mind: unifying Constraints and preadaptations in the earliest cognition. New York: Oxford University stages of language evolution. Linguistic Review, Press. 22, 135–159. Gabarini, F., & Adenzato, M. (2004). At the root ofChurchland, P. (1996). Neurophilosophy. Cambridge, embodied cognition: Cognitive science meets MA: MIT Press. neurophysiology. Brain and Cognition, 56,Clark, A. (1989). Microcognition: Philosophy, cognitive 100–106. science and parallel distributed processing. Gallese, V. (2001). The ‘shared manifold’ Cambridge, MA: MIT Press. hypothesis: From mirror neurons toClark, A. (1997). Being there: Putting brain, body empathy. Journal of Consciousness Studies, 8, and world back together again. Cambridge, MA: 33–50. MIT Press. Gallese, V. (2003). The roots of empathy: TheCossart, R., Aronov, D., & Yuste, R. (2003). shared manifold hypothesis and the neural Attractor dynamics of networks UP states in basis of intersubjectivity. Psychopathology, 36, the neocortex. Nature, 423, 283–288. 171–180.

OUT OF OUR MINDS 585Gallese, V. (2005). Embodied simulation: From Jolly, A. (1966). Lemur social behaviour and neurons to phenomenal experience. Phenomenology primate social intelligence. Science, 153, and the Cognitive Sciences, 4, 22–48. 501–506.Gallese, V. (2007). Before and below ‘theory of Kirsh, D. (1991). Today the earwig, tomorrow mind’: Embodied simulation and the neural man? Artificial Intelligence, 47, 161–184. correlates of social cognition. Philosophical Landauer, T. K., & Dumais, S. T. (1997). A solution Transactions of the Royal Society, Series B, 362, to Plato’s problem: the latent semantic 659–669. analysis theory of acquisition, induction andGallese, V., Keysers, C., & Rizzolatti, R. (2004) representation of knowledge. Psychological A unifying view of the basis of social cognition. Review, 104, 211–240. Trends in Cognitive Sciences, 8: 396–403. Maris, M., & te Boekhorst, R. (1996). ExploitingGershon, M. (1998). The second brain. New York: physical constraints: Heap formation through Harper Collins. behavioural error in a group of robots.Gibson, J. J. (1979). The ecological approach to In M. Asada (Ed.), Proceedings of IROS ’96: visual perception. Hillsdale, NJ: Lawrence IEEE/RSJ International Conference on Intelligent Erlbaum Associates. Robots and Systems (pp. 1655–1660).Grice, P. (1957). Meaning. Philosophical Review, 66, Merleau-Ponty, M. (1942). The structure of 377–388. behaviour. Pittsburgh: Duquesne UniversityGriffin, D. (1978). The question of animal Press. awareness: Evolutionary continuity of mental Michaels, C. F., & Carello, C. (1981). Direct experience. New York: The Rockefeller perception. New Jersey: Prentice Hall. University Press. Miller, G. A. (2003). The cognitive revolution:Grossberg, S. (1987). Competitive learning: From A historical perspective. Trends in Cognitive interactive activation to adaptive resonance. Sciences, 7, 141–144. Cognitive Science, 11, 23–63. Owren, M. J., & Rendall, D. (1997). An affectGrossberg, S. (1988). Neural networks and natural conditioning model of non-human primate intelligence. Cambridge, MA: MIT Press. vocalizations. In D. W. Owings, M. D. Beecher, &Hauser, M. D. Chomsky, N., & Fitch, W. T. N. S. Thompson (Eds.), Perspectives in ethology, (2002). The faculty of language: What is it, Vol. 12, communication (pp. 299–346). New who has it, and how did it evolve? Science, 298, York: Plenum Press. 1569–1579. Owren, M. J., & Rendall, D. (2001). Sound on theHenzi, S. P., & Barrett, L. (2002). Infants as a rebound: Returning function to the forefront in commodity in a baboon market. Animal. understanding nonhuman primate vocal Behaviour, 63, 915–921. signaling. Evolutionary Anthropology 10, 58–71.Hopfield, J. J. (1982). Neural networks and physical Owren, M. J., Rendall, D., & Bachorowski, J. systems with emergent collective computational (2003). Non-verbal vocal communication. In abilities. Proceedings of the National Academy of D. Maestripieri (Ed.), Primate psychology: Sciences, 79, 2554–2558. Bridging the gap between mind and behaviourHumphrey, N. K. (1976). The social function in human and non-human primates (pp. of intellect. In: P. P. G. Bateson & 359–394). Cambridge: Harvard University R. A. Hinde (Eds.), Growing points in ethology Press. (pp. 303–317). Cambridge: Cambridge Penn, D. C.. Holyoak, K. J., & Povinelli, D. J. University Press. (2008). Darwin’s mistake: Explaining theJames, W. (1890). Principles of psychology. New discontinuity between human and nonhuman York: Dover. minds. Behavioral and Brain Sciences, 31,Johnson, C. (2001). Distributed primate cognition: 109–130. A review. Animal Cognition, 4, 167–183. Pert, C. B. (1997). Molecules of emotion. New York:Johnson, M., & Rohrer, T. (in press). We are live Scribner. creatures: Embodiment, American pragmatism, Pfeifer, R., & Bongard, J. (2007). How the body and the cognitive organism. In: J. Zlatev, shapes the way we think. Cambrdige, MA: MIT T. Ziemke, R. Frank, & R. Dirven (Eds.), Body, Press. Language, and Mind (vol. 1, pp. 17–54). Berlin: Pfeifer, R., & Scheier, C. (1999). Understanding Mouton de Gruyter. intelligence. Cambrdige, MA: MIT Press.

586 PRIMATE NEUROETHOLOGYReed, E. (1996). Encountering the world: Toward an Skarda, C. A., & Freeman, W. J. (1990). Chaos and ecological psychology. Oxford: Oxford University the new science of the brain. Concepts in Press. Neuroscience, 1, 275–285.Rendall, D., Notman, H., & Vokey, J. R. (2008). Strum, S. C. (1982)Agonistic dominance in male Quotidian cognition and the human- baboons: An alternative view. International nonhuman divide: Just more or less of a good Journal of Primatology, 3, 175–202. thing? Behavioral and Brain Sciences, 31, 144–145. Strum, S. C. (1994). Reconciling aggression andRendall, D., Cheney, D. L., & Seyfarth, R. M. social manipulation as means of competition: (2000). Proximate factors mediating contact 1. Life history perspective. International Journal calls in adult female baboons and their infants. of Primatology, 15, 739–765. Journal of Comparative Psychology, 114, 36–46. Strum, S. C., Forster, D., & Hutchins, E. (1997).Rockwell, W. T. (2005). Neither brain nor ghost: Why Machiavellian intelligence may not be A nondualist alternative to the mind-brain Machiavellian. In A. Whiten and R. W. Byrne identity theory. Cambridge, MA: MIT Press. (Eds.), Machiavellian intelligence II: ExtensionsRumbaugh, D. M., & Washburn, D. A. (2003). and evaluations. Cambridge: Cambridge Intelligence of apes and other rational beings. University Press. New Haven: Yale University Press. Suomi, S. J., & Harlow, H. F. (1975). Effects ofRumbaugh, D. M., Washburn, D. A., & Hillix, W. differential removal from group on social A. (1996). Respondents, operants, and development of rhesus monkeys. Journal emergents: Toward an integrated perspective of Child Psychology and Psychiatry, 16, on behavior. In K. Pribram & J. King (Eds.), 149–164. Learning as a self-organizing process (pp. 57–73). Van Gelder, T. (1995). What might cognition be, if Hillsdale, NJ: Lawrence Erlbaum. not computation? Journal of Philosophy, 92,Scott-Kelso, J. A. (1995). Dynamic patterns: The 345–81. self-organization of brain and behaviour. ¨ Von Uexkull, J. (1957). A stroll through the worlds Cambridge, MA: MIT Press. of animals and men. In: C. H. Schiller & K. S.Seyfarth, R. M., Cheney, D. L., & Marler, P. (1980). Lashley (Eds.), Instinctive behaviour: The Monkey responses to three different alarm calls: development of a modern concept (pp. 5–82). Evidence of predator classification and semantic New York: International University Press. communication. Science, 210, 801–803. (Original work published in 1934).Silk, J.B, Rendall, D., Cheney, D.L. & Seyfarth, R.M. Wheeler, M. (2005). Reconstructing the cognitive (2003) Natal attraction in adult female baboons world: The next step. Cambridge, MA: MIT (Papio cyncocephalus ursinus) in the Moremi Press. reserve, Botswana. Ethology, 109, 627–644. ¨ Zuberbuhler, K. (2003). Referential signaling inSkarda, C. A., & Freeman, W. J. (1987). Brains nonhuman primates: Cognitive precursors and make chaos to make sense of the world. limitations for the evolution of language. Behavioral and Brain Sciences, 10, 161–195. Advances in the Study of Behavior, 33, 265–307.

CHAPTER 29The Comparative Neuropsychology of ToolUse in Primates with Specific Reference toChimpanzees and Capuchin MonkeysWilliam D. HopkinsThe human brain is roughly three times larger language are both human universal traits andthan it should be for a species of our body weight. predominantly lateralized to the left hemisphere,In addition, the evolution of the brain has not many have suggested that these two functionsbeen uniform, with distinct cortical and subcor- coevolved and are possibly unique to hominidtical regions being selected for in different species evolution (Corballis, 1992; Crow, 2004;within mammals, and specifically within primates Williams et al., 2006). Whether nonhuman ani-(Deacon, 1997; Finlay & Darlington, 1995; mals, particularly primates, exhibit population-Rilling, 2006; Semendeferi & Damasio, 2000; see level behavioral or brain asymmetries remains aalso chapters by Kaas and Preuss, this volume). strongly contentious issue in the scientific litera-For example, in primates, some have suggested ture (Crow, 1998; Hopkins, 2006, 2007; McGrewthere has been differential expansion of the & Marchant, 1997a; Palmer, 2002).frontal and temporal lobes relative to the other What ecological, psychological, or sociolo-lobes and that these changes might reflect specific gical factors contributed to the evolutionaryselection for cortical development in brain expansion of the human brain remains a topicregions associated with complex cognition, of considerable debate. The human brain isincluding language (Deacon, 2004; Rilling & metabolically expensive, suggesting that thereSeligman, 2002; Semendeferi et al., 2001, 2002). must have been strong selection pressure for its In addition to relative brain size, the human expansion related to specific dietary, ecological,brain is also highly lateralized with specific sen- or social factors. From the standpoint of ecolo-sory, motor, and cognitive processes differen- gical factors, it has been suggested that cognitivetially performed by the left and right cerebral and motor abilities that facilitated food extrac-hemispheres, respectively (Bradshaw & Rogers, tion and consumption may have selected for the1993; Corballis, 1992; Galaburda, 1995). Two development of the brain. Notably, some havenotably lateralized functions in the human suggested that extractive foraging, those beha-brain are handedness and language. Although viors that include the use of tools, would havethere is some cultural variation, right-handed- provided certain nutritive and dietary advan-ness is considered a universal trait in humans. tages (Gibson & Ingold, 1993; Greenfield, 1991;Similarly, language is considered a universal trait Parker & Gibson, 1977). The use of tools wouldin the human species and is predominantly later- have similarly selected for increased motor plan-alized to the left hemisphere, particularly among ning, motor skill, and coordination of the limbsright-handed individuals (Beaton, 1997; Knecht in the execution of certain behaviors. Thus,et al., 2000). Because right-handedness and some have suggested that the evolution of tool 587

588 PRIMATE NEUROETHOLOGYuse may have served as a preadaptation for the planning deficit) (Meador et al., 1999). Similarly,evolution of complex cognition, language, and functional imaging studies have shown that therehandedness in humans (Bradshaw & Rogers, are common neural systems involved in both1993; Frost, 1980). One aim of this chapter is speech and planned tool use actions, suggestingto summarize data on tool use in primates and that common cognitive mechanisms may underlieits neuropsychological correlates, at both the their execution, despite being metrically quite dis-phylogenetic and individual levels of analysis. tinct actions (Johnson-Frey, 2004; Lewis, 2006). The antithesis of the ecological arguments is These kinds of data should remind us that there arethe claim that social cognition was selected for in likely parallels between so-called ‘‘social’’ andprimate evolution (Dunbar, 1996). Primates live ‘‘nonsocial’’ cognitive abilities in primates, despitein complex social groups, and some have sug- the convenience of treating them as separategested that keeping track of and understanding domains. In the latter part of this chapter, I discussthe nuances of group-living primates, such as some examples of where there might be overlap indominance ranks, kin, etc., place significant cog- cognitive processes in tool use relative to intraspe-nitive demands on the individuals within the cies communication in chimpanzees.group. A number of studies have shown thatbrain size in primates correlates with group TOOL USE IN PRIMATESsize, and even their ability to engage in deception(a complex social cognitive ability implying Tool use has been described in many speciestheory of mind) (Byrne & Corp, 2004). (Beck, 1980; Candland, 1987; Tomasello & The basic theoretical frameworks described Call, 1997) but, save humans, the complexitypreviously characterize primate cognition pri- and variability in tool-using abilities of somemarily into two domains, physical cognition, great ape species are relatively unmatched inincluding complex object manipulation and tool the animal kingdom. Recently, Whiten and col-use, and social cognition, which would include leagues (1999) provided a summary of the evi-abilities such as social learning and imitation. dence of tool use in wild chimpanzees fromInterestingly, functional imaging studies in different sites in Eastern, Central, and Westernhuman and nonhuman primates are beginning to Africa (Table 29.1). As can be seen, the forms ofreport results that implicate common neural sys- tool use are quite variable between sites and aretems in the use of both social and nonsocial cogni- not easily attributable to local ecological factors.tion, thereby blurring the distinction between For instance, chimpanzees at Gombe termite-these two abilities. For example, Rizzolatti and fish, leaf-groom, and use objects for self-tickling,colleagues have reported the existence of mirror none of which are seen at other sites such as Taineurons in the homolog to Broca’s area in the and Bossou. In contrast, chimpanzees at Bossoumonkey brain (area F5) when monkeys either and Tai Forest use hammers to open nutswatch or engage in a reach-and-grasp task, a task whereas Gombe chimpanzees do not, despitepresumably requiring little if any social cognition the fact that the same rocks and nuts are avail-(Rizzolatti & Arbib, 1998). Similar mirror neuron able. Based on this conclusion, Whiten and col-activation can be found in humans when watching leagues (1999) argued that the diversity andor engaging in reach-and-grasp responding variability in tool use in chimpanzees may reflect(Grafton et al., 1996), but these same mirror neu- a form of culture. Not to be outdone, followingrons also seem to be involved in tasks involving the reports of geographical variation in tool useimitation or attribution of mental state. With spe- in chimpanzees, Van Schaik and colleaguescific reference to tool use, similar parallels can be (2003) examined variability in tool use beha-seen in terms of neuropsychological correlates in viors of wild orangutans from Borneo andlinguistic functions relative to tool use. For Sumtra. Like chimpanzees, the orangutansinstance, clinical studies suggest that lesions to showed significant variability in tool use in thethe left but not right frontal cortex can induce different study sites, and some of the variabilityboth aphasia (speech deficit) and apraxia (motor could not be attributed to ecological factors.

TABLE 29.1 Variation in Occurrence of Behavior Patterns Across Long-Term Study SitesSite Bs Ta Go Ma Mk Kib BdA 1 Investigatory probe (probe and sniff) H C C H H + (-) 2 Play start (invite play holding stem in mouth) + H C C C C H 3 Drag branch (drag large branch in display) H C C C C H H 4 Leaf-sponge (leaf mass used as sponge) C C C + e C C 5 Branch-clasp (clasp branch above, groom) H C C C C C C 6 Branch-shake (to attract attention, court) C C C C C C C 7 Buttress-beat (drum on buttress of tree) C C C C C C CB 8 Nasal probe (clear nasal passage with stick) - - - + - - À 9 Comb (stem used to comb through hair) - - - - - - + 10 Insect-pound (probe used to mash insect) + - - - - - - 11 Resin-pound (extract resin by pounding) + - - e? e? - - 12 Branch-hook (branch used to hook branch) + - - - - - - 13 Perforate (stout stick perforates termite nest) - e - - - e e? 14 Dig (stick used as spade to dig termite nest) + e - - - e e? 15 Brush-stick (probing stick with brush end) - - - - - - - 16 Seat-stick (stick protection from thorns) - - e e? e? e e 17 Stepping-stick (walking on sticks over thorns) - - e e? e? e e 18 Container (object used as container) - - + - - - - 19 Leaf-mop (leaves used to mop up insects) - - + - + e e? 20 Leaf-wipe (food wiped from skull, etc.) e? + + - - - - 21 Leaf-brush (leaf used to brush away bees) - - + - - - - 22 Open and probe (perforate, then probe) - - - - - - - 23 Sponge push-pull (stick and sponge tool) + + + + e e -C 24 Algae-scoop (scoop algae using wand) C e e e e e e 25 Ground night-nest (night-nests on ground) (-) e? + e? e? e? + 26 Anvil-prop (rock used to level anvil) H e e e e e eD 27 Food-pound onto wood (smash food) C C C - - e? H 28 Food-pound onto other (such as stone) - H C - - e? - 29 Nut-hammer, wood hammer on wood anvil - C - e e e? e 30 Nut-hammer, wood hammer on stone anvil - C - - - e? e 31 Nut-hammer, stone hammer on wood anvil + C - e e e? e 32 Nut-hammer, stone hammer on stone anvil C C - - - e? e 33 Nut-hammer, other (such as on ground) - H - - - e? e 34 Pestle-pound (mash palm crown with petiole) C - - e? e? e? e? 35 Club (strike forcefully with stick) + H H + - + - 36 Termite-fish using leaf midrib + e - - C e e? 37 Termite-fish using nonleaf materials - e C - C e e? 38 Ant-fish (probe used to extract ants) + - + C C - - 39 Ant-dip-wipe (manually wipe ants off wand) + - C - - - - 40 Ant-dip-single (mouth ants off stick) C C + - - - - 41 Fluid-dip (use of probe to extract fluids) - C C H H H - 42 Bee-probe (disable bees, flick with probe) - C - - + - - 43 Marrow-pick (pick bone marrow out) - C - - - - - 44 Lever open (stick used to enlarge entrance) - H C - - - - 45 Expel/stir (stick expels or stirs insects) - C H H H - - 46 Seat-vegetation (large leaves as seat) + H - - - + - 47 Fly-whisk (leafy stick used to fan flies) - H + - - - H 48 Self-tickle (tickle self using objects) - - H - - - - (continued) 589

590 PRIMATE NEUROETHOLOGYTable 29.1 (Continued)Site Bs Ta Go Ma Mk Kib Bd 49 Aimed-throw (throw object directionally) C C C C - + + 50 Leaf-napkin (leaves used to clean body) - + C + - C C 51 Leaf-dab (leaf dabbed on wound, examined) - + + - - C - 52 Leaf-groom (intense ‘‘grooming’’ of leaves) - - C C C C + 53 Leaf-clip, mouth (rip parts off leaf, with mouth) C C - C C H C 54 Leaf-clip, fingers (rip leaf with fingers) - H - + - H C 55 Leaf-strip (rip leaves off stem, as threat) + - H + - H - 56 Leaf-squash (squash ectoparasite on leaf) - - H ? ? - - 57 Leaf-inspect (inspect ectoparasite on hand) - - + ? ? - C 58 Index-hit (squash ectoparasite on arm) - C + - - - - 59 Hand-clasp (clasp arms overhead, groom) - H - C C C - 60 Knuckle-knock (knock to attract attention) + C H C C - - 61 Branch-din (bend, release saplings to warn) - - - - - - - 62 Branch-slap (slap branch, for attention) C C - + - - C 63 Stem pull-through (pull stems noisily) C - + H - H - 64 Shrub-bend (squash stems underfoot) H - - C - - C 65 Rain dance (slow display at start of rain) - H C C C C HA, patterns absent at no site; B, patterns not achieving habitual frequencies at any site; C, patterns for which any absencecan be explained by local ecological factors; D, patterns customary or habitual at some sites yet absent at others, with noecological explanation. To facilitate comparison, behaviors are listed so that adjacent categories share broad functions; inband D these are: 27–35, pounding actions; 36–40, fishing; 41–43, probing; 44 and 45, forcing; 46 and 47, comfortbehavior; 48 and 49, miscellaneous exploitation of vegetation properties; 50–57, exploitation of leaf properties; 56–59,grooming; 60–64, attention-getting. Sites (with subspecies, observation period in years by September 1998, site director): ¨Bs, Bossou, Guinea (verus, 23, Y.S.); Ta, Taı Forest, Ivory Coast (verus, 23, C.B.); Go, Gombe, Tanzania (schweinfurthii, 38,J.G.); Ma, Mahale M-group, Tanzania (schweinfurthii, 30, T.N.); Mk, Mahale K-group (schweinfurthii, 18, T.N.); Kib,Kibale Forest, Uganda (schweinfurthii, 11, R.W.W.); Bd, Budongo Forest, Uganda (schweinfurthii, 8, V.R.). C, customary;H, habitual; +, present; -, absent; e, absent with ecological explanation; e?, ecological explanation suspected; (-), absentpossibly because of inadequate observation; ?, answer uncertain (see text for full definitions). Branch din (behavior 61) is ´allocated to band D because it is known to be customary at Lope, Gabon (C.E.G.T.); behaviors 13, 15–17, and 22 areallocated to band B because they have been recorded at shorter-term sites (see Supplementary Information). For fulldefinitions of all behaviors, see Supplementary Information.Reprinted by permission from MacMillan Publishers Ltd.: Whiten, A., Goodall, J., McGrew, W. C., Nishida, T., Reynolds,V., Sugiyama, Y., et al. (1999). Cultures in chimpanzees Nature, 399, 682–685.Some of the tool-using and other social tradi- In captivity, the evidence of tool use is some-tions included the use of specific materials to what different with respect to the great apes. Notamplify vocalizations, use of materials to shield surprisingly, chimpanzees and orangutansthemselves from the sun and rain, and use readily use tools in captivity but, interestingly,of probing tools to extract food items so do gorillas and bonobos, largely in the context(Table 29.2). of foraging tasks used as enrichment in zoos or Interestingly, unlike orangutans and chim- research settings (Boysen et al., 1999; Harrison &panzees, neither gorillas nor bonobos are well Nystrom, 2008; Jordon, 1982; Nakamichi, 1999).known for their tool-using abilities in the wild. Thus, cognitively, gorillas and bonobos are cap-Some limitations of our knowledge of bonobo able of learning to use tools, but it seems thattool use are hampered by a lack of data and either ecological or social factors may limit theopportunity for sustained and long-term study, expression of this ability in the wild.but this cannot be said for gorillas, where there Most other reports of tool use in nonhumanhave been several long-term field projects yet primates have largely been restricted to captivelittle reports of tool use. environments (Beck, 1980; Candland, 1987;

TABLE 29.2 Geographic Variation in Orangutan Behavior Patterns Site and Island Gunung Tanjung Kutai Lower Kinaba- Leuser, Leuser, Palung Putnig (Borneo) tangan (Borneo) Ketambe Suaq (Borneo) (Borneo) (Sumatra) Balimbing (Sumatra) Observation intensity (increased ranks): 2 2 1 1 2 1 Very likely cultural variants 1. Snag riding: riding on pushed–over snag as it falls, then A C A A A A grab onto vegetation before it crashes on ground 2. Kiss-squeak with leaves: using leaves on mouth to C A H A A A amplify sound, then drop leaf 3. Kiss-squeak with hands: using fists (like trumpet) or R R H A C H flat hands on mouth to amplify sound 4. Leaf wipe: wiping face with fistful of squashed leaves, A C A A A A then drop (in kiss-squeak context) 5. Play nests: building nest for social play (no resting C C P A C H occurs) 6. Bunk nests: building nest a short distance above the A P A H A A nest used for resting (during rain) 7. Suncover: building cover on nest during bright A ? C C H A sunshine (rather than rain) 8. Hide under nest: seeking shelter under nest for rain A R C P R A 9. Scratch stick: using detached stick to scratch body parts A R H A A A 10. Autoerotic tool: using tool for sexual stimulation A A P A C A (female and male) (continued)591

592 Table 29.2 (Continued) Site and Island Gunung Tanjung Kutai Lower Kinaba- Leuser, Leuser, Palung Putnig (Borneo) tangan (Borneo) Ketambe Suaq (Borneo) (Borneo) (Sumatra) Balimbing (Sumatra) 11. Raspberry: spluttering sounds associated with nest A A A H A C building 12. Symmetric scratch: exaggerated, long, slow, symmetric A A A A R C scratching movements with both arms at same time 13. Twig biting: systematically passing ends of twigs used A A A A A C for lining of nest past the mouth (sometimes including actual bites) during last phase of nest building 14. Leaf napkin: using handful of leaves to wipe A A C A A A latex off chin 15. Branch as swatter: using detached leafy R A H H H H branches to ward off bees/wasps attacking subject (who is usually raiding its nest) 16. Leaf gloves: using leaf gloves to handle spiny fruits or A R A A H E spiny branch or as seat cushions in trees with spines 17. Tree-hole tool use: using tool to poke into tree A A A A A C holes to obtain social insects or their products 18. Seed extraction tool use: using tool to extract A A E A E C seeds from the protected fruits of Neesia sp. 19. Branch scoop: drinking water from deep tree hole using A A A A A H leafy branch (water dripping from leaves) Likely cultural variants (ecological explanation not excluded)

20. Snag crashing: aimed pushing of dead standing trees C C C A H C 21. Bouquet feeding: using lips to pick ants from fistful of C C A R C C dry, fresh, or rotting leaves (nests) 22. Nest destruction: rummaging through old orangutan H C P A H H nests for insects 23. Dead twig sucking: breaking hollow (dead) twigs to A? C A? A C C suck ants from inside 24. Slow loris eating: capturing and eating slow loris hiding A A A A H H in dense vegetation Rare behaviors 1. Females rubbing their genitals together R 1R A A A R 2. Using leaf to clean body surface R A A A A A 3. Sneaky nest approach: building series of nests while R A A A A A approaching conspecific and fruit tree 4. Leaf bundle while sleeping (‘‘doll’’) R R A A A A 5. Leaf scoop: drinking water from the ground, using leaf R A A A A A as vessel (drinking straight from vessel) 6. Bridge nest: building nest connecting two trees on A R A A A A opposite banks of river 7. Biting through vine to swing Tarzan-like across gap A R A A A R 8. Artistic pillows: similar twigs lining nest A P ? A ? ? 9. Branch dragging display on ground A A ? R A A 10. Stick as chisel to open termite nest in log on ground A A A A R A 11. Sponging: drinking water using crumpled leaves A A A A R A 12. Hiding behind detached branch from predators or A R P R R A humans C, customary; H, habitual; R, rare; P, present with unknown frequency, probably rare; E, absent for ecological reasons; A, absent; ?, unknown. From Van Schaik, C. P., Ancrenaz, M., Borgen, G., Galdikas, B., Knott, C. D., Singleton, I., et al. (2003). Orangutan culture and the evolution of material culture. Science, 299, 102–205. Reprinted with permission from AAAS.593

594 PRIMATE NEUROETHOLOGYTomasello & Call, 1997), with the exception of COGNITIVE MECHANISMScapuchin monkeys. There are numerous reports GOVERNING TOOL USEof tool use in captive capuchin monkeys thatinclude probing actions, nut-cracking, and the The cognitive processes underlying tool use inuse of sponges, behaviors not unlike those reported nonhuman primates has historically been andin wild and captive apes (McGrew & Marchant, continues to be a topic of intense discussion. In1997b; Visalberghi, 1990). Though originally terms of phylogeny, some have suggested thatthought to be an artifact of captivity, recent studies differences in tool-using abilities between greathave reported evidence of tool use in wild capuchin apes and monkeys are due to basic differences inmonkeys. Specifically, in certain dry, arid environmental representation and cognition. Forments, wild capuchins have been documented to example, differences in the attainment of stagesuse rocks to pound open nuts (Fragaszy et al., 2004; of cognitive development, such as thoseMoura & Lee, 2004)(Fig. 29.1). There are also some described by Piaget, among primates has beenanecdotal reports of capuchin monkeys using suggested as an explanation for phylogeneticleaves to sponge water from tree trunks (Phillips, differences in tool use (Dore & Dumas, 1987).1998). Thus, tool use in captive capuchin monkeys Thus, on certain Piagetian object permanence ordoes not appear to be an artifact of captivity but other tasks, chimpanzees reach stage 5 or 6 per-rather an inherent ability expressed in both settings. formance, whereas other nonhuman primatesFigure 29.1 Photograph of a wild capuchin monkey using a stone tool. Reprinted from Fragaszy, D., Izar,P., Visalberghi, E., Ottoni, E. B., & Oliveira, M. G. (2004). Wild capuchin monkeys (Cebus libidinosus) useanvils and stone pounding tools. American Journal of Primatology, 64, 359–366. Used with permission.

THE COMPARATIVE NEUROPSYCHOLOGY OF TOOL USE IN PRIMATES 595do not reach these same levels. The differences in food from the tube. Apes and monkeys had littleobject permanence performance between apes difficulty learning this task but the key manip-and monkeys presumably underlie their abilities ulation was the insertion of a transparent trap,to acquire and use tools (Parker & Gibson, adjacent to the location of the food inside the1977). Some have been critical of the neo- tube. Thus, inserting the stick from the wrongPiagetian explanations and have suggested that end of the tube and pushing out the food woulddifferences in manipulatory proclivities are suf- result in the food falling into the trap, whereasficient to explain tool-using abilities (Fragaszy, inserting from the opposite end would allow the1989). For instance, capuchin monkeys and food to be successfully pushed out of the tube. Inchimpanzees engage in similar types of object essence, the question was whether the subjectsmanipulation and object-object relational beha- understand the causal relationship between theirvior, despite differences in their performance on actions and consequences associated with thePiagetian tasks, and arguably this can more removal of the food. The results indicated thateasily explain the tendencies for these species to most chimpanzees were able to accommodateexcel at tool use in captivity and the wild. the trap problem relatively easily and without It has also been suggested that the endpoint prolonged training. In contrast, the capuchinof tool use as a cognitive process can be achieved monkeys performed poorly on these tasks andby different mechanisms among primate species. did not benefit from prolonged training. TheIn a series of elegant studies, Visalberghi and interpretation of these findings is that differentcolleagues (Limongelli et al., 1995; Visalberghi cognitive mechanisms associated with under-& Limongelli, 1994) devised a task that required standing causality explain variability in thethat chimpanzee and capuchin monkeys solve a capuchin and ape performance. Thus, capuchintask, referred to as the ‘‘trap-tube’’ task. In these monkeys and apes can learn to use tools but dostudies, food is placed inside a transparent tube so by different cognitive mechanisms.in the middle so that the food cannot be reached A variant of the ‘‘trap-tube’’ task that has alsowith the use of the hand (Fig. 29.2A). Instead, been used for testing primate abilities to inferthe subjects must insert a stick and push out the causality is the ‘‘cane-pulling’’ task. Like the B A From Visalberghi & Limongelli, 1994 From Santos et al. 2005Figure 29.2 Example photographs of (A) capuchin monkeys solving the trap-tube task and (B) a ring-tailed lemur solving the cane-pulling task. Reprinted from Visalberghi, E., & Limongelli, L. (1994). Lack ofcomprehension of cause-effect relations in tool-using capuchin monkeys (Cebus apella). Journal ofComparative Psychology, 108, 15–22; and Santos, L. R., Mahajan, N., & Barnes, J. L. (2005). Howprosimian primates represent tools: Experiments with two lemur species (Eulemur fulvus and Lemurcatta). Journal of Comparative Psychology, 119, 394–403. Used with permission.

596 PRIMATE NEUROETHOLOGYtrap-tube task, the cane-pulling task was designed is not a great deal of data and the paucity ofto test different primate species’ abilities about comparative data makes any analysis difficult.understanding causality, but it places far fewer Comparatively, it has been well documentedmotor demands on subjects; thus, many more that more closely related primates show greaterspecies with less developed motor skills can be levels of prehensile grasping, individual control oftested for their understanding of causality. the digits, and complex object manipulationBasically, in the cane-pulling task, there are two skills, particularly in the combinatorial use of(or more) sticks that are curved or have hooks at objects (Christel, 1994; Torigoe, 1985). Globalthe end of them. Initially during training, a food correlational analyses have shown that increasingreward is baited at the hooked end of one of the brain size is associated with increasing incidencestwo canes and the subjects must learn to pull the of tool use and other cognitive abilities (Deaner etcorrect cane in order to receive the reward (Fig. al., 2007; Reader & Laland, 2002), but these29.2B). A number of species including lemurs, results are not very specific with regard to identi-marmosets, tamarins, capuchin monkeys, gib- fying brain regions that were directly selected forbons, and chimpanzees have been tested on the in the evolution of tool use or related motorcane-pulling task, and all of them can learn to functions. In the following section, we presentsolve this task and adjust their responses to var- data on studies that have assessed neuropsycho-ious perceptual variants of the cane properties logical and neuroanatomical correlates of indivi-(Cunningham et al., 2006; Fujita et al., 2003; dual and species differences in tool use or otherHauser, 1997; Povinelli, 2000; Santos et al., complex motor actions.2005). Thus, to some extent, the results from thecane-pulling task are not entirely consistent withthe results obtained in the trap-tube task, and it PHYLOGENETIC CHANGES INremains unclear why this is the case. INTERHEMISPHERIC In my view, several important additional CONNECTIVITY AND THEstudies are needed to flush out the relationship CEREBELLUMbetween these two tasks (and perhaps others). Van Schaik and colleagues (1999) have sug-Notably, what is needed are data on the trap- gested that one important motor componenttube and cane-pulling tasks from the same indi- associated with tool use and other complexviduals to assess whether variation in performance motor actions is the asymmetrical use of theon one task can explain performance on the hands in a coordinated manner. Presumably,second. If this was found, it would suggest that coordinated actions of hands would facilitatecommon cognitive mechanisms underlie the two certain functions because of the differentialtasks, which seems to be the underlying assump- roles played by each hand during specific actionstion but for which there is no empirical support. (such as feeding, plant processing, or tool use)I also believe that another important question and would be facilitated by brain structures thatwith respect to the cane-pulling task is how pri- promote interhemispheric transfer of informa-mates would use this device once they have mas- tion between the left and right hemispheres.tered the basic task. In other words, if primates Beyond changes in the size of the brain, thelearn to use the cane-pulling task in a certain organization and interhemispheric connectivitycontext, would they use the cane in other contexts of the brain has changed in primate evolution.where its use would be functionally beneficial? Rilling and Insel (1999) have examined the association between the evolution of brain size, neocortical surface area, and corpus callosum (CC)NEURAL CORRELATES OF TOOL size. The CC is a major set of largely homotopicUSE IN PRIMATES fibers that connect the two hemispheres. In theFrom the standpoint of understanding the neural analysis by Rilling and Insel (1999), theycorrelates of the cognitive and motor demands of reported that humans and great apes have rela-tool use and complex object manipulation, there tively smaller CC values, after adjusting for brain

THE COMPARATIVE NEUROPSYCHOLOGY OF TOOL USE IN PRIMATES 597size, compared to Old World and New World even though the chimpanzees and orangutansmonkeys (Fig. 29.3). Thus, there are significant are more pronounced tool users. However, allgrade shifts in the relative size of the CC in four ape species do show high complementarydifferent primate species. Rilling and Insel bimanual manipulation but in very different(1999) argued from these results that as the contexts, such as feeding compared to tool usebrain got larger, the two hemispheres became (Byrne & Byrne, 1991). Lastly, within the Oldincreasingly isolated and asymmetries in struc- World monkeys, rhesus monkeys have a relativeture and function subsequently evolved from high CC ratio value compared to mangabeys andthat separation. baboons. How these differences might relate to Germane to this paper is the variation in variation in coordinated bimanual actionsrelative CC size within the different taxonomic remains unclear because of the paucity of beha-families (Fig. 29.3) as it relates to coordinated vioral data from these species.bimanual actions, including tool use, as sug- Another interesting brain area that has beengested by Van Schaik and colleagues (1999). examined comparatively in primates and isFor example, within New World monkeys, potentially relevant to the evolution of tool useCebus have a smaller ratio in size of the CC to and complex manipulation is the cerebellumbrain volume compared to Saimiri, consistent (MacLeod et al., 2003; Rilling & Insel, 1998)with differences in their tool using abilities and (Fig. 29.4). Although historically considered tomanipulatory propensities. The data are less be only important for coordination of motorclear within the other families though. For actions and motor learning, increasingly func-example, within the great apes, the CC-to-brain tional imaging studies in humans have shownvolumes ratios are comparable between the apes, that the cerebellum is involved in complex 1.4 1.2 1.0 CC / Brain volume 0.8 0.6 0.4 0.2 0.0 Cebus Saimiri Papio Macaca Cercopithecus Hylobates Pongo Gorilla Pan p. Pan t.Figure 29.3 Mean ratio in the size of the corpus callosum (CC) to brain volume in different primatespecies. Data regraphed from Rilling, J. K., & Insel, T. R. (1999). Differential expansion of neural projectionsystems in primate brain evolution. NeuroReport, 10, 1453–1459.

598 PRIMATE NEUROETHOLOGY Cerebellum/ Brain volume 50 Vermis / Cerebellum volume 40 Mean ratios 30 20 10 0 Cebus Saimiri Aotus Alouatta Ateles Papio Macaca Cercopithecus Cercocebus Erythrocebus Hylobates Pongo Gorilla Pan p. Pan t.Figure 29.4 Mean ratios in the size of the cerebellum to brain volume (blue) and vermis to cerebellumvolume (red) in different primate species. Data regraphed from MacLeod, C. E., Zilles, K., Schleicher, A.,Rilling, J. K., & Gibson, K. R. (2003). Expansion of the neocerebellum in Hominoidea. Journal of HumanEvolution, 44, 401–429.cognitive and motor functions, including lan- therefore largest expansion of the cerebellarguage and speech (Leiner et al., 1993; Ramnani, hemispheres). Similarly, within Old World2006). A couple of striking results emerge from monkeys, Papio have the relatively smallestthese analyses. First, like the CC ratio data, there vermis and there are certainly more reports ofare significant grade shifts in the relative size of tool use in this genus compared to other Oldthe cerebellum in primates; specifically, greater World monkeys. Lastly, among apes, there areand lesser apes have relatively large cerebellar no real discernable differences but again, it isvolumes compared to Old and New World mon- important to emphasis that all the apes showkeys, after adjusting for differences in brain size. complex object manipulation and motorSecond, in the specific measurement of the learning in both the wild and captivity.vermis, a central structure within the cere-bellum, great apes have relatively small regions,suggesting that the evolutionary expansion of INDIVIDUAL DIFFERENCES INthe cerebellum has been in more lateral regions, TOOL USE: NEUROANATOMICALareas known to be involved in the coordination AND BEHAVIORAL CORRELATESand planning of learned motor actions Cortical Asymmetries(MacLeod et al., 2003). Third, within the majorgrades, there is some variability that may or may The advent of modern imaging technologies hasnot explain variability in tool-using or motor advanced our understanding of primate neuroa-learning abilities at some level of analysis. For natomy and its functional correlates in non-example, within New World monkeys, Cebus are human primates. The use of structural (and tothe only tool-using primate and they have the a lesser extent functional) magnetic resonancerelatively smallest amount of vermis (and imaging (MRI) has allowed for investigators to

THE COMPARATIVE NEUROPSYCHOLOGY OF TOOL USE IN PRIMATES 599examine individual differences in brain-beha- represent the cortical homologs to Broca’s andvior relationships as well as comparative differ- Wernicke’s areas of the human brain and haveences in the organization of different regions of been implicated in studies of handedness andthe cerebral cortex. tool use in humans (Lewis, 2006). For both The use of MRI as it relates to handedness measures of handedness, right-handed chim-and tool use has received some attention recently panzees showed a significantly greater leftwardin the literature. In particular, recent structural asymmetry in FO and the PT compared to non–and functional MRI studies in humans have right-handed chimpanzees (Fig. 29.5A–D).shown that there is region within the precentral Interestingly, the very same neural correlates ofgyrus, referred to as the ‘‘knob,’’ which corre- handedness for tool use in chimpanzees (i.e.,sponds to the area in the motor cortex where the FO) also correlate with handedness for manualhand is represented. The knob is an omega- gestures (Taglialatela et al., 2006) (Fig. 29.6A,B),shaped landmark and is presumably created reinforcing a view that a common cognitive andwhen tissue from the precentral gyrus extends neural substrate may underlie these two beha-posteriorly into the central sulcus and has been viors (discussed later). Whether hand use forreported to be larger in hemisphere contralateral tool use in other primates is associated withto the subjects’ preferred hand (Hammond, asymmetries in the inferior frontal gyrus or2002; Yousry et al., 1997). related brain areas (such as F5 in the macaque Like humans, in chimpanzees the knob is brain) is not known. A recent single celllarger in the hemisphere contralateral to the recording and positron emission tomographysubjects’ preferred hand for coordinated (PET) study in a single macaque monkeybimanual actions as measured by the tube task reported asymmetries in the inferior frontal(Hopkins & Cantalupo, 2004). As noted pre- area for a raking task, but it is difficult to gen-viously, the knob cannot be readily identified eralize from these results in a single subjectin Old or New World monkeys but the motor- (see Chapter 30).hand area can be estimated based on existingtopographical maps of the primary motor Cerebellar Asymmetriescortex of primates. Applying this rationale to asample of capuchin monkeys, Phillips and Several laboratories have been interested in indi-Sherwood (2005) recently found that lateraliza- vidual differences in motor learning, tool use,tion in the dorsal portion of the precentral gyrus and other manual actions in relation to indivi-differed significantly depending on the handed- dual differences in the volume and lateralizationness of the subjects for the tube task. Thus, it of the cerebellum (Cantalupo et al., 2008;appears that hand preference for coordinated Phillips & Hopkins, 2007). Like the cerebralbimanual actions is associated with between- hemispheres, the cerebellum shows a torquehemisphere differences in the motor-hand area asymmetry and can be divided into the left andin chimpanzees and capuchins. right sides as well as along the anterior-posterior More recently, Hopkins and colleagues axis. The left and right cerebellar hemispheres as(2007a) examined the association between well as the anterior-posterior divisions havehandedness for tool use and brain asymmetries been delineated in chimpanzees and capuchinin a sample of captive chimpanzees. Handedness monkeys following the procedure employed byfor tool use was measured for a probing task Snyder and colleagues (1995) with human sub-(designed to simulate termite fishing) as well as jects. When comparing capuchin monkeys andfor opening a coconut (designed to simulate chimpanzees, several interesting differences andanvil use in wild chimpanzees). Right- and similarities emerge (Table 29.3). Specifically,non–right-handed apes were compared on two capuchins show a population-level leftwardbrain regions including the fronto-orbital (FO) asymmetry in the anterior but not posteriorsulcus and planum temporale (PT). FO and PT region of the cerebellum, whereas chimpanzeeswere selected as regions of interest because they do not show population-level asymmetries for

Left- Ambiguously Right- handed handed handed N = 19 N = 12 N = 27 C 0.00 A –0.05 –0.10 Mean AQ Score (±SE) –0.15 –0.20 –0.25 –0.30 Planum temporale –0.35 Frontal orbital sulcus –0.40 –0.45 Non-right Right- B handed handed N = 27 N = 16 D 0.00 –0.05 ` Mean AQ Value (±SE) –0.10 –0.15 –0.20 –0.25 Planum temporale Frontal orbital sulcus –0.30 –0.35 –0.40 BFigure 29.5 Mean asymmetry quotient (AQ) values (– SE) for the planum temporale and fronto-orbitalsulcus in right-, left-, and ambiguously handed chimpanzees when classified on the basis of their handednessfor tool use. Data regraphed from Hopkins, W. D., Russell, J. L., & Cantalupo, C. (2007a). Neuroanatomicalcorrelates of handedness for tool use in chimpanzees (Pan troglodytes): Implication for theories on theevolution of language. Psychological Science, 18, 971–977. Non-right- Right- handed (N = 27) handed (N = 38) A B 0.00 –0.05 Mean AQ Score (±SE) –0.10 –0.15 –0.20 Planum Temporale –0.25 Fronto-Orbital Sulcus –0.30Figure 29.6 Mean asymmetry quotient (AQ) values (– SE) for the planum temporale and fronto-orbitalsulcus in Non-right handed and Right-handed chimpanzees when classified on the basis of their handednessfor manual gestures. Data partially regraphed from Taglialatela, J. P., Cantalupo, C., & Hopkins, W. D.(2006). Gesture handedness predicts asymmetry in the chimpanzee inferior frontal gyrus. NeuroReport, 17,923–927. Used with permission. 600

THE COMPARATIVE NEUROPSYCHOLOGY OF TOOL USE IN PRIMATES 601TABLE 29.3 Data on Cerebellar Asymmetry and Hand Preference in Capuchin Monkeys andChimpanzees Capuchin Chimpanzees Anterior Posterior Anterior PosteriorMean AQ À.129 .000 À.032 .044R (Handedness) À. 761 À.456 .101 .174R (Tool use) NA NA .321 À.485R AQ Values .231 À.761AQ, asymmetry quotient; R, = correlation value.Bolded values indicated significant correlations (P <.05).either region. Chimpanzees show a significant actions, and manual gestures (Hopkins,inverse correlation between asymmetry quotient 2006); however, it has been suggested that(AQ) values in the anterior and posterior cere- these results may be an artifact of the pri-bellar regions, supporting the view that this mates living in a human environmentbrain area shows a torque asymmetry. This asso- (McGrew & Marchant, 1997a). Thus, dataciation is not found in the capuchin monkeys. from wild subjects are very important forIn addition, handedness for a coordinated evaluating the potential influence of captivebimanual task correlates with asymmetries in rearing on behavioral asymmetries. Shown inthe posterior region of the cerebellum in capu- Table 29.4 are the hand preference distribu-chins but not in chimpanzees. However, like the tions for tool use in apes and monkeys.PT and FO results, in chimpanzees, handedness There are at least two important findings fromfor tool use significantly correlates with the these studies. First, in wild chimpanzees, tool useasymmetries in the anterior and posterior sec- behaviors elicit strong preferences at the indivi-tions of the cerebellum (Table 29.3) but in oppo- dual level in comparison to measures of sponta-site directions. No data on handedness for tool neous daily activities, such as unimanual feedinguse and cerebellar asymmetry are available at this (Marchant & McGrew, 1996; McGrew &point for capuchin monkeys, or any other Marchant, 2001). In other words, tool use is aspecies. better and more sensitive measure of handednessHandedness than behaviors such as unimanual feeding or simple reaching because most animals show aAlthough there are no reports of handedness definitive preference of one hand over the other.for tool use in wild monkeys, this behavior Second, apes show population-level handednesshas been reported in wild chimpanzees (Biro for termite fishing and leaf sponging, although inet al., 2003, 2006; Boesch, 1991; Lonsdorf & opposite directions (Table 29.4). Wild chimpan-Hopkins, 2005; Marchant & McGrew, 2007; zees also show a borderline significant bias to theMcGrew & Marchant, 1996). As noted pre- right hand for nut cracking and ant dipping.viously, most humans are right-handed, andsome have suggested that this might be a In captive apes and monkeys, there are faruniquely human trait related to the evolution fewer data on handedness and tool use com-of complex motor actions such as tool use pared to those data reported from wild indivi-and language. Studies in captive chimpanzees duals, which is somewhat surprising given the(and other primates) have reported evidence theoretical interest in this topic and the commonof population-level right-handedness for a use of tool use devices as behavioral enrichmentnumber of behaviors including throwing, in zoos and research laboratories (Table 29.4).bimanual feeding, coordinated bimanual When one considers probing tool use tasks, it

TABLE 29.4 Handedness for Tool Use in Nonhuman Primates Hand Preference Classification #L #R #A Task Author(s)Wild ApesPan 29 15 10 TF McGrew & Marchant, 1996Pan 12 4 1 TF Lonsdorf & Hopkins, 2005Pan 1 7 8 AD Marchant and McGrew, 2007Pan 36 46 3 NC Boesch, 1991Pan 7 11 2 NC Biro et al., 2003Pan 5 7 2 AV McGrew et al., 1999Pan 5 10 2 LS Boesch, 1991Pan 4 5 0 AG Humle & Matsuzawa, 2009Pan 6 19 0 AD Humle & Matsuzawa, 2009Pan 10 11 1 PP Humle & Matsuzawa, 2009Pan 7 17 0 PP-E Humle & Matsuzawa, 2009Pan 4 11 2 LS Biro et al., 2006Captive ApesPan 6 0 5 STF Fletcher & Weghorst, 2005Pan 5 3 1 STF Hopkins, 1999Pan 9 14 14 STF Hopkins & Rabinowitz, 1994Pan 0 2 2 STF Morange, 1994Pan 74 63 52 STF Hopkins et al. (in press)Pan 0 3 0 MT Morris et al., 1993Pan 2 2 0 TH Colell et al., 1995Pan 4 5 0 TH Marchant, 1983Pan 23 50 16 TH Hopkins et al. 2005Pan 3 1 1 MT Steiner, 1990Pan 0 1 0 NC Fouchart et al., in pressPan p. 5 5 4 UK Harrison & Nystrom, 2008Pan p. 0 1 0 KP Toth et al., (1993Gorilla 0 2 3 TH Shafer, 1993Pongo 2 2 2 STF O’Malley & McGrew, 2006Pongo 2 1 2 STF Phillips, 2008, unpublished dataCaptive MonkeysMacaca 3 1 1 STF Westergaard, 1991Papio 2 3 0 LS Westergaard, 1993Papio 2 1 1 STF Westergaard, 1993Cebus 3 1 1 STF Westergaard, 1991Cebus 8 1 2 STF Lilak & Phillips, 2007Cebus 3 0 1 STF Limongelli et al., 1994Cebus 8 5 0 STF Westergaard & Suomi, 1994aCebus 8 4 2 NC Westergaard & Suomi, 1994bCebus 5 7 2 LS Westergaard & Suomi, 1993Cebus 2 1 0 TH Westergaard & Suomi, 1994cCebus 11 12 2 TH Westergaard et al., 2000Cebus 1 9 4 KP Westergaard & Suomi, 1996L, left; R, right; A, ambiguous. For wild apes: TF, termite-fishing; AD, ant dipping; NC, nut cracking; LS, leaf sponging. Forcaptive apes and monkeys: TH, throwing; STF, simulated termite fishing (or probing tasks); KP, stone knapping; UK,unknown or not reported; NC, nut cracking; LS, leaf sponging; MT, multiple tasks; PP, pestle pounding; PP-E, pestlepounding-extraction, and AG, algae scooping. 602

THE COMPARATIVE NEUROPSYCHOLOGY OF TOOL USE IN PRIMATES 603seems that capuchin monkeys show a left- alone (Hopkins et al., 2007b). The RASP andward bias in hand use, not unlike the data EFG are frequently produced in conjunctionpresented on termite fishing in wild chimpan- with the simultaneous production of a manualzees. For throwing, captive chimpanzees show gesture directed toward a human (Taglialatela &population-level right-handedness, whereas Hopkins, 2005) (Fig. 29.7). As can be seen incapuchin monkeys do not. There are simply Figure 29.7, the onset of the manual gesturetoo few data in other species and for other directed toward a human is temporally verytasks to make any meaningful inferences. close to onset of the orofacial movements associated with the production of the RASP in this specific example. In two separate studies usingTOOL USE, MANUAL MOTOR both between-group and within-subject compar-SKILL, AND OROFACIAL isons, we have found that subjects that simulta-CONTROL neously produce the RASP and EFG sounds while gesturing are significantly more right-handedAs noted earlier, the motor and cognitive processes than those that do not simultaneously produce ainvolved in tool use have been proposed as poten- sound while gesturing (Hopkins & Cantero, 2003;tial neuropsychological preadaptations for the evo- Hopkins & Leavens, 1998). The mean handednesslution of language and speech in humans indices for gestural communication in chimpan-(Bradshaw & Rogers, 1993). One of the challenges zees that do or do not simultaneously produce awith this evolutionary scenario is identifying the vocalization are shown in Figure 29.8.common neurological, cognitive, and motor abil- A second interesting aspect of the RASP andities that would have been selected upon during EFG sounds is that not all chimpanzees produceevolution that allowed for language and speech them. Some chimpanzees use them very reliably,(Arbib, 2005; Gentilucci & Corballis, 2006). In while others are less reliable or simply fail to useshort, at some point in primate evolution, the them. The origin of this difference remains unclear,presumed neurological, motor, and cognitive pro- but if a central motor system controls both manualcesses originally underlying manual actions had to and potentially orofacial movements associatedbe incorporated into motor control of the mouth, with (the learning of) sound production, then itfacial musculature, and peripheral speech organs. can be hypothesized that individual differences inMoreover, cognitive structures underlying tool use motor skill in one domain (manual) might behad to be extended into the social domain. related to motor skill in another domain (orofacial The underlying assumptions of these argu- control). We have at least some evidence to supportments are that if common neural substrates this hypothesis in two sets of data from my labora-underlie both manual and potentially motor tory (Table 29.5). First, my colleagues and I havecontrol of the mouth and facial musculature, previously tested our chimpanzees on a taskthen significant associations should be evident requiring fine motor prehension of small foodfor their coactivation or use in nonhuman pri- items (Hopkins & Russell, 2004; Hopkins et al.,mates. Unfortunately, very little research in non- 2002). In these studies, we recorded the numberhuman primates has focused on this question, of errors made by the chimpanzees when graspingbut several recent studies in my laboratory sug- small food items. When comparing the errorsgest that, at least in chimpanzees, common between chimpanzees that reliably and unreliablyneural mechanisms might be involved in the produce the RASP or EFG vocalization, we findcontrol of manual and orofacial actions. that chimpanzees that produce the RASP and EFG First, in captivity, chimpanzees have been sounds make significantly fewer grasping errorsshown to selectively produce one of two sounds, compared to those who do not. We also find thateither a ‘‘raspberry’’ (RASP) or ‘‘extended food chimpanzees that reliably produce the RASP andgrunt’’ (EFG), to capture the attention of an EFG sounds also perform significantly better on aotherwise inattentive human and suppress these tool use task requiring them to successfully insertsounds in the presence of the food or human small sticks into a hole to extract food. Lastly, we

A B A B C D C DFigure 29.7 Sequential frames representing the co-occurrence of the production of the raspberry andmanual gestures. (A) Start of trial. (B) Onset of raspberry. (C) Offset of raspberry. (D) End of manualgesture. Handedness for Gesture + Vocal Sum-HI * Gesture + No Vocal P< .01 Gesture + Vocal –0.8 –0.6 –0.4 –0.2 0.0 0.2 0.4 0.6 0.8 Left-handed No bias Right-handedFigure 29.8 Mean handedness index values (– SE) for manual gestures that are (Gesture + Vocal) or notaccompanied by a vocalization (Gesture + No Vocal). Sum-HI is the handedness value for all gesture thatwere and were not accompained by a vocalization. 604

THE COMPARATIVE NEUROPSYCHOLOGY OF TOOL USE IN PRIMATES 605TABLE 29.5 Grasping Skill and Hand Preference Differences in Chimpanzees Who are Reliable orUnreliable in their Use of Attention-Getting Sounds Vocalization Group Reliable Unreliable t P NBehavioral measure Grasping skill* 7.77 9.94 2.32 .02 136 Tool use performance† 8.43 12.43 2.52 .02 60Hand preference Tool use .133 À.075 Gesture .421 .331 Reaching .173 .083 Tube 221 .071 Mean .237 .107 3.10 .01 101* Indicates number of errors out of 40 trials (20 trials for each hand).† Indicates average latency to insert stick into pipe. P = level of significance, N = number of subjects tested on the task.find that chimpanzees that reliably produce the Studies in our laboratory and others haveRASP and EFG sounds are significantly more clearly shown that chimpanzees and other greatright-handed for several handedness tasks apes (orangutan, gorilla, and bonobo) useincluding reaching, coordinated bimanual actions, manual gestures referentially and intentionallytool use, and manual gestures. Collectively, these to direct the attention of a human to otherwiseresults suggest that common neural pathways, pos- unattainable objects (usually food but sometimessibly lateralized to the left hemisphere, may to a tool) (Call & Tomasello, 1994, 2007; Leavensunderlie motor control of both manual and orofa- et al., 2004a,b; Russell et al., 2005). The produc-cial movements in chimpanzees. Moreover, it sug- tion of manual gestures is not a frustratedgests that the lateralization for both manual and reaching response as these communicative beha-orofacial control of volitional expressions and viors are selectively and almost exclusively pro-sounds were in place before the split between the duced in the presence of an audience and notcommon ancestor of humans and apes. without one (Leavens & Hopkins, 1998; Leavens et al., 1996; Poss et al., 2006). Moreover, chimpanzees and other great apes can alter their mod-GESTURAL AND VOCAL ality of communication depending on theCOMMUNICATION IN CAPTIVE attentional status of a human or conspecific.APES AS A FORM OF SOCIAL Thus, apes will use primarily visual signals whenTOOL USE the human or conspecific is facing them but willI have principally focused on material tool use in use tactual or auditory signals when the human orthis chapter but want to also draw some parallels conspecific is facing away from them (Cartmill &between tool use as reported in wild apes Byrne, 2007; Hostetter et al., 2001, 2007; Leavens(notably chimpanzees and orangutans) with evi- et al., 2004b; Liebal et al., 2004). This suggests thatdence of referential, gestural communication in the signals are intentional and that the apescaptive apes. As noted previously, historically understand the function of their signaling.scientists have considered social and nonsocial The evidence of gestural communication incognition as separate domains of investigation. I captive chimpanzees stands in contrast to reportsbelieve that the interface between social and in wild apes (Leavens et al., 2005). Feral chim-nonsocial cognition lies at the heart of the panzees no doubt gesture in the context of foodobserved differences reported during intraspe- begging and reconciliation but, as far as we know,cies gestural and vocal communication in cap- they do not seem to referentially point in the waytive apes and tool use in wild apes. that gestures have been reported in captive

606 PRIMATE NEUROETHOLOGYchimpanzees (Pika & Mitani, 2006). As my col- (i.e., try a different signal) on their signals. All ofleagues and I have argued elsewhere, this discre- these behavioral traits reflect a more ‘‘fluid’’ orpancy can in part be explained by what we refer to ‘‘innovative’’ form of intelligence that simply getsas the ‘‘referential problem space’’ (Leavens et al., expressed differently, depending on certain ecolo-2005). Basically, when wild chimpanzees want gical or social factors. It is likely that commonsome food that is in proximity to them, they can cognitive and neurological processes underlielocomote over to the food and take it. The excep- many of these behaviors rather than modules ortion is, of course, food that is not otherwise visible domain-specific ‘‘social’’ and ‘‘nonsocial’’ cogni-to them, such as termites inside a termite mound tive processes, as they are often characterized.or the meat of a nut inside a shell. In these cases, To the extent that other species engage inchimpanzees must fashion a tool in order to similar communicative behaviors remains, inextract the food. Like wild chimpanzees, when essence, unstudied. There are a few short reportscaptive chimpanzees want food that is visible to of ‘‘gestural’’ communication in Old and Newthem and the food is in the cage, then can walk World monkeys, but none of these results are ter-over and grasp the food. In contrast, sometimes ribly robust (Anderson et al., 2007; Hattori et al.,in captivity, foods are available to them but the 2007; Hess et al., 1993; Mitchell & Anderson, 1997).chimpanzees cannot gain access to the food, such In one of the more sophisticated and elaborateas when food is located outside their home cage. studies, Hattori and colleagues (2007) replicated aIn these cases, the chimpanzees have learned to study previously done in chimpanzees on the sen-use their gestures (and other signals) to capture sitivity to the eyes during intraspecies communica-the attention of a human to retrieve the food for tion. In the study, the experimenters recorded thethem. In essence, the chimpanzees use the human frequency of ‘‘gestures’’ in response to the presenceas a tool and, arguably, the apes would use a tool of food when a human had his or her eyes open orin the absence of a human, if one were provided closed. Interestingly, the capuchin monkeys didto them. In other words, if a rake were made look differentially at the human in these conditionsavailable, the chimpanzees would likely use the but did not differentiate the human’s ‘‘gestural’’rake to get the food (see Barrett & Rendall, this behavior. That is, the capuchins were equally likelyvolume, for similar themes). In the absence of a to point to the human whether the human had hisrake (or other tool), they are left only with the or her eyes open or closed. This stands in contrastsporadically present human that they have to reports in chimpanzees, which suggest they dolearned to manipulate through their communi- differentially gesture depending on whether thecative signals in order to obtain the food. eyes of the human are open or closed (Hostetter I would further suggest that the flexibility seen et al., 2007). This would suggest that the capuchinin tool use by wild apes is similarly seen in the monkeys did not understand the communicativecommunicative repertoire of captive chimpanzees. function of their gestures. Hattori and colleaguesRecall that wild chimpanzees use a variety of dif- (2007) also noted that they never observed theferent tools in relation to different ecological and capuchins alternate their gaze between the referentsocial factors (Whiten et al., 1999). Apes in cap- and the human, a behavioral concordance fre-tivity will use a variety of different communicative quently reported in great ape intraspeciessignals to capture and direct the attention of a communication.human or conspecific. For example, captive apes Thus, it seems that ecological factors havewill vocalize, spit, throw, bang on the cage, or clap favored the emergence of tool use in capuchintheir hands as means of getting the attention of a monkeys and apes but, perhaps, only apes mighthuman when food is visible but otherwise unavail- be able to capitalize on this inherent ability andable to the apes (e.g., Leavens et al., 2004b). When a utilize it functionally in captive settings to solve atool fails to work properly in the wild, apes will novel, socio-communicative problem. The com-repair the tool or get new one. In captivity, when a parison here is not limited to Pan versus Cebuscommunicative signals fails, apes will repair it (i.e., either; in captivity, the non–tool-using gorillas dotry again with some modification) or elaborate not show the same patterns of intraspecies

THE COMPARATIVE NEUROPSYCHOLOGY OF TOOL USE IN PRIMATES 607communication that has been reported in chim- different shaping procedures, such as the cane-panzees and orangutans (Cartmill & Byrne, 2007; pulling task employed by many; however, toLeavens et al., 2004b; Poss et al., 2006). Thus, not what extant these abilities would generalize tounlike material tool use, at the heart of the matter (1) new tool-using tasks or (2) new contextsin gestural communication in monkeys and apes remains the challenge. In other words, asare questions of causality and functionality. In recently suggested by Rumbaugh andother words, do monkeys and apes understand Washburn (2003), to what extent there wouldthe function of their communicative signals? This be emergent cognitive processes that extendis much like the question of whether they under- beyond the specific behavioral context in whichstand the function of their tool use. the tool-using abilities were trained remains to be determined. Moreover, to what extent the animals would understand the function of toolSUMMARY AND SOME CLOSING use beyond the immediate context is not clear.THOUGHTS The existing data suggest that apes, notablyWith respect to object manipulation and tool use, chimpanzees and orangutans, would and dothe literature seems to suggest that basic phyloge- generalize, while the data from capuchin mon-netic differences are evident in object manipula- keys remain unclear. Apes and capuchins havetion, prehension, and tool-using abilities in dominated this literature because they acquireprimates. More distantly related primates from and use tools relatively easily, but data fromhumans show less complex object manipulation other species would be particularly useful forskills, no individual control of the digits, and few this research because many of them are consid-tool-using abilities, whereas more closely related ered ‘‘relational’’ learners (Rumbaugh &primates show more sophisticated object manipu- Washburn, 2003) and therefore might do welllation and tool-using skills as well as greater control on tool-related tasks, despite the lack of sponta-of individual digits. The phylogenetic picture, of neous proclivity to use tools in the wild andcourse, is more complicated because different pri- captivity.mates within certain Genera may or may not Neurologically, the comparative data fromexpress certain abilities depending on specific eco- different primate species indicate that therelogical or social adaptations. Cebus are an excellent have been significant grade shifts in the organi-example as it seems that certain ecological factors zation of the cerebellum and in interhemispherichave favored the evolution of tool use in this species connectivity, at least as reflected in the relativecompared to other species from the same family. corpus callosum size. Great apes have relativelyLikewise, among great apes, gorillas are not well small CCs compared to Old and New Worldknown for their tool-using abilities in either cap- monkeys. Similarly, in great apes, there hastivity or the wild, despite the fact that they have been significant expansion of the cerebellum,good object manipulation skills and individual particularly within the hemispheres, comparedcontrol of their digits. Thus, at a global level, to Old and New World monkeys. One of theincreasing brain size and asymmetry has been interesting findings from these data are theselected for in primate evolution, but many adap- within-family variations in cerebellar volumetive radiations have occurred within specific pri- as well as interhemispheric connectivity. Formate families in response to certain ecological and instance, within New World monkeys, there issocial factors. Tool use may simply be one form or considerable variation in the relative size of theexpression of these adaptations, but equally com- cerebellum, and Cebus differ substantially fromplex cognitive and motor processes may underlie other genera, suggesting that there may be spe-different non–tool-use behaviors of apes and mon- cific changes in the organization in this brainkeys (Byrne, 1995). area for the learning of complex motor actions, With specific reference to tool use, I have no such as tool use or object manipulation. In con-doubt that most nonhuman primates can learn trast, within great apes, there is less variability into solve simple tool-using tasks through the relative size of the cerebellum and arguably

608 PRIMATE NEUROETHOLOGYgreater commonality in manipulatory and acquisition and maintenance of throwing inmotor learning abilities, albeit these get chimpanzees and other apes (Hopkins et al.,expressed very differently in chimpanzees and 2005). The use of these communicative beha-orangutans (say, for tool use) compared to gor- viors by apes but not other nonhuman primatesillas (say, for complex manual actions associated is not trivial, because it suggests that funda-with feeding). mental differences might be present in social Neuropsychological data from chimpanzees cognitive abilities between apes and monkeyssuggest that common cortical mechanisms may that are rooted in their understanding of causalunderlie tool use and gestural communication. relationships, of which tool use may represent aSpecifically, regions within the inferior frontal physical domain of this underlying system. Thisgyrus correlate with lateralized hand use for both argument is consistent with the view of Readermanual gestures and tool use, whereas other and Laland (2002), who argue that evolution hasbehavioral measures of handedness do not. It is selected for increasing executive function in pri-important to emphasize that the studies to date mates, which has adaptive value for both socialhave focused on morphology, but whether there and nonsocial cognitive abilities or, more gen-are functional homologs in the chimpanzee (or erally speaking, innovative cognitive behaviors.other nonhuman primates) brain in the use oftools and gestures is not clear and will requirethe application of functional imaging studies ACKNOWLEDGMENTS(Taglialatela et al., 2008). The results of our This work was supported in part by NIH grantsstudies in chimpanzees further suggest that indi- RR-00165, NS-42867, NS-36605, HD-38051, andviduals who appear to exhibit better orofacial HD-56232. Correspondence and reprint requestsmotor control perform better on the fine should be addressed to Dr. William Hopkins,motor tool use tasks and grasping tasks and are Division of Psychobiology, Yerkes Nationalmore right-handed for several handedness tasks. Primate Research Center, 954 Gatewood Road,Collectively, these results point to the conclusion Atlanta,Georgia30322.E-mail:whopkin@emory.eduthat the left hemisphere of chimpanzees is domi- or whopkins@agnesscott.edu.nant for motor learning and control in both themanual and orofacial domain (but seeSherwood, 2005, and Sherwood et al., 2005, forother systems involved in motor control of face). REFERENCESThe need for data from other species cannot be Anderson, J. R., Kuwahata, H., & Fujita, K. (2007).overstated. Gaze alternation during "pointing" by squirrel Lastly, the suite of tool-using abilities found monkeys (Saimiri sciureus)? Animal Cognition,in great apes is paralleled, in some ways, by a 10, 267–271.suite of communicative behaviors that are used Arbib, M. (2005). From monkey-like actionin captivity during intraspecies communication. recognition to human language: AnCaptive chimpanzees, and other great apes, par- evolutionary framework for neurolinguistics.ticularly orangutans, use gestures, vocal signals, Behavioral and Brain Sciences, 28, 105–167.and several attention-getting behaviors such as Beaton, A. A. (1997). The relation of planumthrowing, hand clapping, cage banging, and spit- temporale asymmetry and morphology of theting when communicating with humans (and corpus callosum to handedness, gender and dyslexia: A review of the evidence. Brain andother apes). No other nonhuman primates Language, 60, 255–322.engage in these kinds of behaviors, so far as I Beck, B. B. (1980). Animal tool behavior: The useknow, or at least with any degree of frequency. and manufacture of tools by animals. New York:There are reports of throwing in captive capu- Garland.chin monkeys, but as I have argued elsewhere, Biro, D., Inoue-Nakamura, N., Tonooka, R.,throwing had to be explicitly trained in these Yamakoshi, G., Sousa, C., & Matsuzawa, T.studies, which is quite different than the (2003). Cultural innovation and transmission

THE COMPARATIVE NEUROPSYCHOLOGY OF TOOL USE IN PRIMATES 609 of tool use in wild chimpanzees: Evidence from Thierry, & N. Herrenschmidt (Eds.), Current field experiments. Animal Cognition, 6, 213–223. primatology. Vol III: Behavioral neuroscience,Biro, D., Sousa, C., & Matsuzawa, T. (2006). physiology and reproduction (pp 37–49). Ontogeny and cultural propagation of tool use Strasbourg: Universite Louis Pasteur. by wild chimpanzees at Bossou, Guinea: Case Colell, M., Segarra, M. D., & Sabater-Pi, J (1995). studies in nut cracking and leaf folding. In: T. Hand preferences in chimpanzees (Pan Matsuzawa, T. Tomonaga, & M. Tanaka (Eds.), troglodytes), bonobos (Pan paniscus) and Cognitive development of chimpanzees (pp 476– orangutans (Pongo pygmaeus) in food-reaching 507). New York: Springer. and other daily activities. International JournalBoesch, C. (1991). Handedness in wild of Primatology, 16, 413–434. chimpanzees. International Journal of Corballis, M. C. (1992). The lopsided brain: Primatology, 6, 541–558. Evolution of the generative mind. New York:Boysen, S. T., Kuhlmeier, K., Halliday, P., & Oxford University Press. Halliday, Y. M. (1999). Tool use in captive Crow, T. (2004). Directional asymmetry is the key gorillas. In: S. T. Parker, R. W. Mitchell, & H. to the origin of modern Homo sapiens (the L. Miles (Eds.), The mentality of gorillas and Broca-Annett axiom): A reply to Rogers’ orangutans. Cambridge: Cambridge University review of the speciation of modern Homo Press. sapiens. Laterality: Asymmetries of Body, BrainBradshaw, J. L., & Rogers, L. J. (1993). The and Cognition, 9, 233–242. evolution of lateral asymmetries, language, tool Crow, T. J. (1998). Sexual selection, timing and the use, and intellect. San Diego: Academic Press. descent of man: A theory of the genetic originsByrne, R. W. (1995). The thinking ape: Evolutionary of language. Current Psychology of Cognition, 17, origins of intelligence. Oxford: Oxford University 1079–1114. Press. Cunningham, C. L., Anderson, J. R., & Mootnick,Byrne, R. W., & Byrne, J. M. (1991). Hand A. R. (2006). Object manipulation to obtain a preferences in the skilled gathering tasks of food reward in hoolock gibbons, Bunopithecus mountain gorillas (Gorilla gorilla berengei). hoolock. Animal Behaviour, 71, 621–629. Cortex, 27, 521–536. Deacon, T. (2004). Monkey homologues ofByrne, R. W., & Corp, N. (2004). Neocortex size language areas: Computing the ambiguities. predicts deception rate in primates. Proceedings Trends in Cognitive Sciences, 8, 288–290. of the Royal Society B, 271, 1393–1699. Deacon, T. W. (1997). The symbolic species: TheCall, J., & Tomasello, M. (1994). Production and coevolution of language and the brain. New comprehension of referential pointing by York: W. W. Norton and Company. orangutans (Pongo pygmaeus). Journal of Deaner, R. O., Isler, K., Burkart, J., & Van Schaik, Comparative Psychology, 108, 307–317. C. P. (2007). Overall brain size, and notCall, J., & Tomasello, M. (2007). The gestural encephalization quotient, best predicts cogni- communication of monkeys and apes. Oxford: tive ability across non-human primates. Brain, Psychology Press. Behavior and Evolution, 70, 115–124.Candland, D. (1987). Tool use. In: G. Mitchell & J. Dore, F. Y., & Dumas, C. (1987). Psychology of M. Erwin (Eds.), Comparative primate biology. animal cognition: Pigetian studies. Psycho- Vol 2, Part B: Behavior, cognition and motivation logical Bulletin, 102, 219–233. (pp. 85–103). New York: Alan R. Liss. Dunbar, R. I. M. (1996). Grooming, gossip, and theCantalupo, C., Freeman, H. D., & Hopkins, W. D. evolution of language. Cambridge: Harvard (2008). Patterns of cerebellar asymmetries in University Press. great apes as revealed by MRI. Behavioral Finlay, B. L., & Darlington, R. B. (1995). Linked Neuroscience, 122, 191–198. regularities in the development and evolution ofCartmill, E., & Byrne, R. W. (2007). Orangutans mammalian brains. Science, 16, 1578–1584. modify their gestural signaling according to Fletcher, A. W., & Weghorst, J. A. (2005). Laterality their audience’s comprehension. Current of hand function in naturalistically housed Biology, 17, 1–14. chimpanzees (Pan troglodytes). Laterality, 10,Christel, M. I. (1994). Catarrhine primates 219–242. grasping small objects: Techniques and hand Fouchart, J., Bril, B., Hirata, S., Morimura, M., preferences. In: J. R. Anderson, J. J. Roeder, B. Houki, C., Ueno, Y., et al. (in press). A

610 PRIMATE NEUROETHOLOGY preliminary analysis of nut-cracking movements understands without language. Cognition, 64, in a captive chimpanzee: Adaptation to the 285–308. properties of tools and nuts. In: V. Rouz & B. Hess, J., Novak, M. A., & Povinelli, D. J. (1993). Bril (Eds.), Stone knapping: The necessary "Natural pointing" in a rhesus monkeys but no conditions for a uniquely hominoid behavior. evidence of empathy. Animal Behaviour, 46, Cambridge, MA: McDonald Institute Series 1023–1025. Monograph. Hopkins, W. D. (1999). On the other hand:Fragaszy, D., Izar, P., Visalberghi, E., Ottoni, E. B., Statistical issues in the assessment and & Oliveira, M. G. (2004). Wild capuchin interpretation of hand preference data in non- monkeys (Cebus libidinosus) use anvils and human primates. International Journal of stone pounding tools. American Journal of Primatology, 20, 851–866. Primatology, 64, 359–366. Hopkins, W. D. (2006). Comparative and familialFragaszy, D. M. (1989). Tool use, imitation and analysis of handedness in great apes. insight: Apples, orangs and pea soup. Psychological Bulletin, 132, 538–559. Behavioral and Brain Sciences, 12, 596–599. Hopkins, W. D. (Ed.). (2007). Evolution ofFrost, G. T. (1980). Tool behavior and the origin of hemispheric specialization in primates. Oxford: laterality. Journal of Human Evolution, 9, 447–459. Elsevier.Fujita, K., Kuroshima, H., & Asai, S. (2003). How Hopkins, W. D., & Cantalupo, C. (2004). do tufted capcuhin monkeys (Cebus apella) Handedness in chimpanzees is associated with understand causality involved in tool use? asymmetries in the primary motor but not with Journal of Experimental Psychology: Animal homologous language areas. Behavioral Behaviour Processes, 29, 233–242. Neuroscience, 118, 1176–1183.Galaburda, A. M. (1995). Anatomic basis of Hopkins, W. D., Cantalupo, C., Wesley, M. J., cerebral dominance. In: R. J. Davidson & K. Hostetter, A. B., & Pilcher, D. (2002). Grip Hugdahl (Eds.), Brain asymmetry. Cambridge, morphology and hand use in chimpanzees (Pan MA: MIT Press. troglodytes): Evidence of a left hemisphereGentilucci, M., & Corballis, M. C. (2006). From specialization in motor skill. Journal of manual gesture to speech: A gradual Experimental Psychology: General, 131, .412–423. transition. Neuroscience and Biobehavioral Hopkins, W. D., & Cantero, M. (2003). From hand Reviews, 30, 949–960. to mouth in the evolution of language: TheGibson, K. R., & Ingold, T. (1993). Tools, language influence of vocal behavior on lateralized hand and cognition in human evolution. Cambridge: use in manual gestures by chimpanzees (Pan Cambridge University Press. troglodytes). Developmental Science, 6, 55–61.Grafton, S. T., Fagg, A. H., Woods, M. A., & Arbib, Hopkins, W. D., & Leavens, D. A. (1998). Hand use M. (1996). Functional anatomy of pointing and and gestural communication in chimpanzees grasping in humans. Cerebral Cortex, 6, 226–237. (Pan troglodytes). Journal of ComparativeGreenfield, P. M. (1991). Language, tools, and Psychology, 112, 95–99. brain: The ontogeny and phylogeny of Hopkins, W. D., & Russell, J. L. (2004). Further hierarchically organized sequential behavior. evidence of a right hand advantage in motor Behavioral and Brain Sciences, 14, 531–550. skill by chimpanzees (Pan troglodytes).Hammond, G. (2002). Correlates of human Neuropsychologia, 42, 990–996. handedness in primary motor cortex: A review Hopkins, W. D., Russell, J. L., & Cantalupo, C. and hypothesis. Neuroscience and Biobehavioral (2007a). Neuroanatomical correlates of Reviews, 26, 285–292. handedness for tool use in chimpanzees (PanHarrison, R. M., & Nystrom, P. (2008). Handedness troglodytes): Implication for theories on the in captive bonobos (Pan paniscus). Folia evolution of language. Psychological Science, Primatologica, 79, 253–268. 18, 971–977.Hattori, Y., Kuroshima, H., & Fujita, K. (2007). I Hopkins, W. D., Russell, J., Cantalupo, C., know you are not looking at me: Capuchin Freeman, H., & Schapiro, S. (2005). Factors monkeys’ (Cebus apella) sensitivity to human influencing the prevalence and handedness attentional states. Animal Cognition, 10, 141–148. for throwing in captive chimpanzeesHauser, M. (1997). Artifactual kinds and (Pan troglodytes). Journal of Comparative functional design features: What a primate Psychology, 119, 363–370.

THE COMPARATIVE NEUROPSYCHOLOGY OF TOOL USE IN PRIMATES 611Hopkins, W. D., Taglialatela, J. P., & Leavens, D. A. Leavens, D. A., Hostetter, A. B., Wesley, M. J., & (2007b). Chimpanzees differentially produce Hopkins, W. D. (2004b). Tactical use of novel vocalizations to capture the attention of unimodal and bimodal communication by a human. Animal Behaviour, 73, 281–286. chimpanzees, Pan troglodytes. AnimalHopkins, W. D., Russell, J. L., Schaeffer, J. A., Behaviour, 67, 467–476. Gardner, M. Schapiro, S. J. (in press). Leiner, H. C., Leiner, A. L., & Dow, R. S. (1993). Handedness for toll use in captive chimpanzees Cognitive and language functions of the human (Pan troglodytes). Sex differences, performance, cerebellum. Trends in Neurosciences, 16, 444–447. heritability and comparison to the wild. Lewis, J. W. (2006). Cortical networks related toHostetter, A. B., Cantero, M., & Hopkins, W. D. human use of tools. The Neuroscientist, 12, 211– (2001). Differential use of vocal and gestural 231. communication by chimpanzees (Pan Liebal, K., Pika, S., Call, J., & Tomasello, M. (2004). troglodytes) in response to the attentional To move or not to move: How apes adjust to the status of a human (Homo sapiens). Journal of attentional state of others. Interaction Studies, 5, Comparative Psychology, 115, 337–343. 199–219.Hostetter, A. B., Russell, J. L., Freeman, H., & Lilak, A. L., & Phillips, K. A. (2007). Consistency in Hopkins, W. D. (2007). Now you see me, now hand preference across low-level and high-level you don’t: Evidence that chimpanzees tasks in capuchin monkeys (Cebus apella). understand the role of the eyes in attention. American Journal of Primatology, 69, 1–12. Animal Cognition, 10, 55–62. Limongelli, L., Boysen, S. T., & Visalberghi, E.Humble, T & Matsuzawa. T, (2009). Laterality in (1995). Comprehension of cause-effect hand use across four tool-use behaviours relations in a tool-using task by chimpanzees among the wild chimpanzees of Bossou, (Pan troglodytes). Journal of Comparative Guniea, West Africa. American Journal of Psychology, 109, 18–26. Primatology, 71, 40–48. Limongelli, L., Sonetti, M. G., & Visalberghi, E.Johnson-Frey, S. H. (2004). The neural basis of (1994). Hand preference of tufted capuchins complex tool use in humans. Trends in (Cebus apella) in tool-using tasks. In: J. R. Cognitive Sciences, 8, 71–78. Anderson, N. Herrenschmidt, J. J. Roeder, B.Jordon, C. (1982). Object manipulation and tool use Thierry (Eds.), Current primatology. Vol III: in captive pygmy chimpanzees (Pan paniscus). Behavioral neurosciences, physiology and Journal of Human Evolution, 11, 35–39. reproduction (pp. 9–15). Strasbourg: UniversiteKnecht, S., Drager, B., Deppe, M., Bobe, L., Lohmann, Louis Pasteur. H., Floel, A., et al. (2000). Handedness and Lonsdorf, E. V., & Hopkins, W. D. (2005). Wild hemispheric language dominance in healthy chimpanzees show population level handedness humans. Brain, 123, 2512–2518. for tool use. Proceedings of the NationalLeavens, D. A., & Hopkins, W. D. (1998). Academy of Sciences, 102, 12634–12638. Intentional communication by chimpanzee MacLeod, C. E., Zilles, K., Schleicher, A., Rilling, J. (Pan troglodytes): A cross-sectional study of K., & Gibson, K. R. (2003). Expansion of the the use of referential gestures. Developmental neocerebellum in Hominoidea. Journal of Psychology, 34, 813–822. Human Evolution, 44, 401–429.Leavens, D. A., Hopkins, W. D., & Bard, K. A. Marchant, L. F. (1983). Hand preferences among (1996). Indexical and referential pointing in captive island groups of chimpanzees. New chimpanzees (Pan troglodytes). Journal of Bruswick, NJ: Rutgers, The State University of Comparative Psychology, 110, 346–353. New Jersey.Leavens, D. A., Hopkins, W. D., & Bard, K. A. Marchant, L. F., & McGrew, W. C. (1996). Laterality (2005). Understanding the point of of limb function in wild chimpanzees of Gombe chimpanzee pointing: Epigenesis and National Park: Comprehensive study of ecological validity. Current Directions in spontaneous activities. Journal of Human Psychological Science, 14, 185–189. Evolution, 30, 427–443.Leavens, D. A., Hopkins, W. D., & Thomas, R. Marchant, L. F., & McGrew, W. C. (2007). Ant (2004a). Referential communication by fishing by wild chimpanzees is not lateralised chimpanzees (Pan troglodytes). Journal of Primates, 48, 22–26. Comparative Psychology, 118, 48–57.

612 PRIMATE NEUROETHOLOGYMcGrew, W. C., & Marchant, L. F. (1996). On Parker, S. T., & Gibson, K. R. (1977). Object which side of the apes? In: W. C. McGrew, L. manipulation, tool use and sensorimotor F. Marchant, & T. Nishida (Eds.), Great ape intelligence as feeding adaptations in cebus societies (pp. 255–272). Cambridge: Cambridge monkeys and great apes. Journal of Human University Press. Evolution, 6, 623–641.McGrew, W. C., & Marchant, L. F. (1997a). On the Phillips, K., & Sherwood, C. S. (2005). Primary other hand: Current issues in and meta-analysis motor cortex asymmetry correlates with of the behavioral laterality of hand function in handedness in capcuhin monkeys (Cebus non-human primates. Yearbook of Physical apella). Behavioral Neuroscience, 119, 1701–1704. Anthropology, 40, 201–232. Phillips, K. A. (1998). Tool use in wild capuchinMcGrew, W. C., & Marchant, L. F. (1997b). Using the monkeys (Cebus albifrons trinitatis). American tools at hand: Manual laterality and elementary Journal of Primatology, 46, 259–261. technology in Cebus spp. and Pan spp. Phillips, K. A., & Hopkins, W. D. (2007). Exploring International Journal of Primatology, 18, 787–810. the relationship between cerebellar asymmetryMcGrew, W. C., & Marchant, L. F. (2001). and handedness in chimpanzees (Pan Ethological study of manual laterality in the troglodytes) and capuchins (Cebus apella). chimpanzees of the Mahale mountains, Neuropsychologia, 45, 2333–2339. Tanzania. Behaviour, 138, 329–358. Pika, S., & Mitani, J. C. (2006). Referential gesturalMeador, K. J., Loring, D. W., Lee, K., Hughes, M., communication in wild chimpanzees (Pan Lee, G., Nichols, M., et al. (1999). Cerebral troglodytes). Current Biology, 16, R191–R192. lateralization: Relationship of language and Poss, S. R., Kuhar, C., Stoinski, T. S., & Hopkins, ideomotor apraxia. Neurology, 53, 2028–2031. W. D. (2006). Differential use of attentional andMitchell, R. W., & Anderson, J. R. (1997). Pointing, visual communicative signaling by orangutans withholding information, and deception in (Pongo pygmaeus) and gorillas (Gorilla gorilla) capuchin monkeys (Cebus apella). Journal of in response to the attentional status of a human. Comparative Psychology, 111, 351–361. American Journal of Primatology, 68, 978–992.Morange, F. (1994). Handedness in two Povinelli, D. J. (2000). Folk physics for apes. Oxford: chimpanzees in captivity. In: J. R. Anderson, J. Oxford University Press. J. Roeder, B. Thierry, & N. Herrenschmidt Ramnani, N. (2006). The primate cortico- (Eds.), Current primatology. Vol III: Behavioral cerebellar system: Structure and function. neuroscience, physiology and reproduction Nature Reviews: Neuroscience, 7, 511–522. (pp. 61–67). Strasbourg: Universite Louis Reader, S. M., & Laland, K. N. (2002). Social Pasteur. intelligence, innovation, and enhanced brainMorris, R. D., Hopkins, W. D., & Bolser-Gilmore, size in primates. Proceedings of the National L. (1993). Assessment of hand preference in two Academy of Sciences, 99, 4436–4441. language-trained chimpanzees (Pan Rilling, J. K. (2006). Human and non-human troglodytes): A multi-method analysis. Journal primate brains: Are they allometrically scaled of Clinical and Experimental Neuropsychology, versions of the same design? Evolutionary 15, 487–502. Anthropology, 15, 65–77.Moura, dA. A. C., & Lee, P. C. (2004). Capuchin Rilling, J. K., & Insel, T. R. (1998). Evolution of stone tool use in Caatinga dry forest. Science, the cerebellum in primates: Differences in 306, 1909. relative volume among monkeys, apes andNakamichi, M. (1999). Spontaneous use of sticks as humans. Brain, Behavior, and Evolution, 52, tools by captive gorillas (Gorilla gorilla gorilla). 308–314. Primates, 40, 487–498. Rilling, J. K., & Insel, T. R. (1999). DifferentialO’Malley, R. C., & McGrew, W. C. (2006). Hand expansion of neural projection systems in preferences in captive orangutans (Pongo primate brain evolution. NeuroReport, 10, pygmaeus). Primates, 47, 279–283. 1453–1459.Palmer, A. R. (2002). Chimpanzee right- Rilling, J. K., & Seligman, R. A. (2002). A handedness reconsidered: Evaluating the quantitative morphometric comparative evidence with funnel plots. American Journal analysis of the primate temporal lobe. Journal of Physical Anthropology, 118, 191–199. of Human Evolution, 42, 505–533.

THE COMPARATIVE NEUROPSYCHOLOGY OF TOOL USE IN PRIMATES 613Rizzolatti, G., & Arbib, M. A. (1998). Language Taglialatela, J. P., Cantalupo, C., & Hopkins, W. D. within our grasp. Trends in Neuroscience, 21, (2006). Gesture handedness predicts 188–194. asymmetry in the chimpanzee inferior frontalRumbaugh, D. M., & Washburn, D. A. (2003). The gyrus. NeuroReport, 17, 923–927. intelligence of apes and other rational beings. Taglialatela, J. P., & Hopkins, W. D. (2005). New Haven, CT: Yale University Press. Gesture-vocal synchrony in captiveRussell, J. L., Braccini, S., Buehler, N., Kachin, M. chimpanzees, Pan troglodytes. American J., Schapiro, S. J., & Hopkins, W. D. (2005). Journal of Primatology, 66(Suppl 1), 70. Chimpanzees (Pan troglodytes) intentional Taglialatela, J. P., Russell, J. L., Schaeffer, J. A., & communication is not contingent upon food. Hopkins, W. D. (2008). Communicative Animal Cognition, 8, 263–272. signaling activates ‘‘Broca’s’’ homologue inSantos, L. R., Mahajan, N., & Barnes, J. L. (2005). chimpanzees. Current Biology, 18, 343–348. How prosimian primates represent tools: Tomasello, M., & Call, J. (1997). Primate cognition. Experiments with two lemur species (Eulemur New York: Oxford University Press. fulvus and Lemur catta). Journal of Comparative Torigoe, T. (1985). Comparison of object Psychology, 119, 394–403. manipulation among 74 species of non-humanSemendeferi, K., Armstrong, E., Schleicher, A., primates. Primates, 26, 182–194. Zilles, K., & Van Hoesen, G. W. (2001). Toth, N., Schick, K. D., Savage-Rumbaugh, E. S., Prefronal cortex of humans and apes: A Sevcik, R. A., & Rumbaugh, D. M. (1993). Pan comparative study of area 10. American the tool-maker: Investigations into the stone Journal of Physical Anthropology, 114, 224–241. tool-making and tool-using capabilities of aSemendeferi, K., & Damasio, H. (2000). The brain bonobo (Pan paniscus). Journal of Archae- and its main anatomical subdivisions in living ological Science, 20, 81–91. hominids using magnetic resonance imaging. Van Schaik, C. P., Ancrenaz, M., Borgen, G., Journal of Human Evolution, 38, 317–332. Galdikas, B., Knott, C. D., Singleton, I., et al.Semendeferi, K., Lu, A., Schenker, N. M., & (2003). Orangutan culture and the evolution of Damasio, H. (2002). Humans and great apes material culture. Science, 299, 102–105. share a large frontal cortex. Nature Van Schaik, C. P., Deaner, R. O., & Merrill, M. Y. Neuroscience, 5, 272–276. (1999). The conditions for tool use inShafer, D. D. (1993). Patterns of hand preference in primates: Implications for the evolution of gorillas and children. In: J. P. Ward & W. D. material culture. Journal of Human Evolution, Hopkins (Eds.), Primate laterality: Current 36, 719–741. behavioral evidence of primate asymmetries (pp. Visalberghi, E. (1990). Tool use in Cebus. Folia 267–283). New York: Springer-Verlag. Primatologica, 54, 146–154.Sherwood, C. C. (2005). Comparative anatomy of Visalberghi, E., & Limongelli, L. (1994). Lack of the facial motor nucleus in mammals, with an comprehension of cause-effect relations in analysis of neuron numbers in primates. tool-using capuchin monkeys (Cebus apella). Anatomical Record Part A, 287A, 1067–1079. Journal of Comparative Psychology, 108,Sherwood, C. C., Hof, P. R., Holloway, R. L., 15–22. Semendeferi, K., Gannon, P. J., Frahm, H. D., Westergaard, G. C. (1991). Hand preference in the et al. (2005). Evolution of the brainstem use and manufacture of tools by tufted orofacial motor system in primates: A capuchin (Cebus apella) and lion-tailed comparative study of trigeminal, facial and macaque (Macaca silenus) monkeys. Journal of hypoglossal nuclei. Journal of Human Comparative Psychology, 105, 172–176. Evolution, 48, 45–84. Westergaard, G. C. (1993). Hand preference in theSnyder, P. J., Bilder, R. M., Wu, H., Bogerts, B., & use of tools by infant baboons (Papio Lieberman, J. A. (1995). Cerebellar volume cynocephalus anubis). Perceptual and Motor asymmetries are related to handedness: A Skills, 76, 447–450. quantitative MRI study. Neuropsychologia, 33, Westergaard, G. C., Liv, C., Haynie, M. K., & 407–419. Suomi, S. J. (2000). A comparative study ofSteiner, S. M. (1990). Handedness in chimpanzees. aimed throwing by monkeys and humans. Friends of Washoe, 9, 9–19. Neuropsychologia, 38, 1511–1517. .

614 PRIMATE NEUROETHOLOGYWestergaard, G. C., & Suomi, S. J. (1993). Hand tool use by monkeys: Possible implications for preference in capuchin monkeys varies with age. the evolution of right-handedness in hominids. Primates, 34, 295–299. Journal of Human Evolution, 30, 291–298.Westergaard, G. C., & Suomi, S. J. (1994a). The Whiten, A., Goodall, J., McGrew, W. C., Nishida, use of probing tools by capuchin monkeys: T., Reynolds, V., Sugiyama, Y., et al. (1999). Evidence of increases right hand use with Cultures in chimpanzees Nature, 399, 682–685. age. International Journal of Primatology, 15, Williams, N. A., Close, J. P., Giouzeli, M., & Crow, 521–529. T. J. (2006). Accelerated evolution ofWestergaard, G. C., & Suomi, S. J. (1994b). Hand Protocadherin 11X/Y: A candidate gene-pair preference in the use of nut-cracking tools by for cerebral asymmetry and language. tufted capuchin monkeys (Cebus apella) Folia American Journal of Medical Genetics Part B Primatologica, 61, 38–42. (Neuropsychiatric Genetics), 141B, 623.633.Westergaard, G. C., & Suomi, S. J. (1994c). Aimed Yousry, T. A., Schmid, U. D., Alkadhi, H., Schmidt, throwing of stones by tufted capuchin monkeys D., Peraud, A., Buettner, A., et al. (1997). (Cebus apella). Human Evolution, 9, 323–329. Localization of the motor hand area to a knobWestergaard, G. C., & Suomi, S. J. (1996). Hand on the precentral gyrus. A new landmark. Brain, preference for stone artefact production and 120, 141–157.

CHAPTER 30Evolution of an Intellectual Mind in thePrimate BrainAtsushi Iriki, Yumiko Yamazaki, and Osamu SakuraMacaque monkeys are thought to rarely use tools branch to eat a persimmon at its tip, indicatingin the wild (Tomasello & Call, 1985), but repeated that they can retrieve a distant target utilizingdescriptions of anecdotal observations, as appear another object to bridge the spatial gap betweenin many traditional Japanese folk tales, suggest them. By manifesting (either rapidly by trainingthat Japanese macaques do use primitive tools or gradually through spontaneous evolutionarywhen they have close interaction with human processes) explicit control of this latent cognitivecommunities, although this has not yet been precursor, we believed that monkeys could bescientifically proven. On the other hand, contem- enabled to use tools in a way underpinned byporary neurophysiologists, using monkeys as brain functions analogous to those in humans.experimental animals, tend to think that During the training processes, baits wereJapanese macaques, compared with other placed initially on a long-shafted (about 30-macaque monkey species, are especially rapid cm) spoon that the monkeys could pull in toand skillful at acquiring complex cognitive and retrieve the bait, mimicking the fruit at the tip ofmotor tasks. Although we have reservations a branch, which they could also easily retrieveregarding this particular cleverness, it is true that (Ishibashi et al., 2000). Then the spoon wasthey are quite cooperative with humans, which replaced with a rake-shaped tool, first with theshould make them efficient at learning experi- bait set close to its near side (another easy step,mental tasks, whereas other macaques (such as immediately accomplished), and thereafter, werhesus monkeys) tend to express an aggressive increased the distance gradually along the trajec-response. Taking advantage of these traditions tory needed to pull it straight in. When the baitand backgrounds, we have attempted to train was positioned to the side, the monkeys curvedJapanese macaques with tool use that they nor- the path of the rake rather easily after a fewmally do not exhibit in their natural habitat, and attempts. The last step—that is, when the foodtried to detect neurobiological changes in their was placed at the far side of the rake—appearedbrains accompanying the acquisition of this slightly harder and monkeys had difficultyhumanoid higher cognitive function. pushing the rake forward to position it beyond the food. However, in just a few days, theyNEURAL MECHANISMS learned universal techniques: to swing horizon-SUBSERVING TOOL USE IN tally beyond the rake, stop immediately beyondTRAINED MONKEYS the food, and then pull the food straight in. Thereafter, this stepwise trajectory becameLearning to Use Tools fused and smoother until finally, skillfulOne first indication that monkeys could use wielding of the rake was used to retrieve thetools arose after they were observed pulling at a food in a single action. They had now learned 615

616 PRIMATE NEUROETHOLOGY A B CFigure 30.1 Sequential images of a monkey retrieving food pellets with a rake. When a food pellet wasdispensed beyond its reach (A), the monkey wielded the tool (B) and pulled the food closer to retrieve it withthe other hand (C). Adapted from Iriki, A., Tanaka, M., & Iwamura, Y. (1996). Coding of modified bodyschema during tool use by macaque postcentral neurons. Neuroreport, 7, 2325–2330. Used with permission.to control the rake to accomplish their goal somatosensory (e.g., tactile, joint, deep muscular)(Fig. 30.1). and visual information about spatial configura- All the monkeys so far trained in our tion of the body merge to form such imageslaboratory (more than 50, without a single (Ungerleider & Mishkin, 1982) (Fig. 30.2, inset),exception) have acquired this tool use skill yet its tangible neural mechanisms had remainedthrough the procedures described previously hidden until we discovered, in 1996, the phe-to accomplish the learning process. At the nomenon described here (Iriki et al., 1996).very beginning of this series of studies, the We have recorded bimodal neurons,complete training procedure took more than responding both to tactile stimulation on thea few months to complete. After experience, hand (thus forming a tactile receptive field,however, the process of accomplishment Fig. 30.2A) and to visual stimuli occurringcould be shortened to 10 to 14 days, depending within the space encompassing the tactileon the criteria for completion of learning receptive field (namely, the visual receptive(Hihara et al., 2003; Iriki et al., 1996; field of the same neuron, Fig. 30.2B), regardlessIshibashi et al., 2000). However, the monkeys of the location of the three-dimensional spacecould never complete this rather simple in which the hand was placed. Thus, theselearning task in a shorter time, which led us neuronal response properties were interpretedto think about the molecular genetic mechan- as coding the image of the hand in space (Irikiisms upon which the later series of experiments et al., 1996; Maravita & Iriki, 2004). Immedi-is based. ately after using the rake, this visual receptive field extended along the axis of the tool (Fig. 30.2C), so that it appeared as if it wasCoding of Modified Body Image upon Tool assimilated into the image of the hand, or alter-Use by Parietal Neurons natively, the image of the hand was extended toExtrapolated from our own introspection during incorporate the tool. When this measurementtool mastery, the monkeys’ tool appeared as if was performed immediately after the monkeysincorporated into their body schema as an used their innate hand to retrieve food, onlyextended forearm. This plasticity has been holding the tool as an external object, the visualregarded as an origin of intelligence, which has receptive field was no longer extended, butlong been believed peculiar to human intelli- limited around the hand (Fig. 30.2D). Thisgence. Such a semantic image of the body, condition, although physically identical withwhich is subject to modifications by assimilating that shown in Figure 30.2C, was mentallytools, has been thought from clinical experience identical with that shown in Figure 30.2B;to be formed and stored in the parietal cortex thus, the neuron appeared to code the mon-(Head & Holmes, 1911). In this cortical area, key’s introspective image of its hand.

EVOLUTION OF AN INTELLECTUAL MIND IN THE PRIMATE BRAIN 617 CS IPS A B LS 10 mm A B C DFigure 30.2 Changes in bimodal receptive field properties following tool use. The somatosensoryreceptive fields (sRFs) of cells in this region were identified by light touches, passive manipulation ofjoints, or active hand use. The visual receptive field (vRF) was defined as the area in which cellular responseswere evoked by visual probes (the most effective ones being those moving toward the sRF). (A) sRF (bluearea) of the bimodal neurons and their vRF (pink areas) before (B) and immediately after (C) tool use orwhen just passively grabbing the rake (D). Left inset: The left hemisphere of a monkey brain, with A and Bindicating that somatosensory and spatial visual processing pathways merge at the intraparietal areaindicated by the red square where neurons were recorded. CS, central sulcus; IPS, intraparietal sulcus; LS,lateral sulcus. Adapted from Maravita, A., & Iriki, A. (2004). Tools for the body (schema). Trends inCognitive Science, 8, 79–86. Used with permission. Activation of this area during tool use has The next question to address was if monkeysbeen confirmed by positron emission tomo- could be trained to acquire advanced stages ofgraphy (PET) (Fig. 30.3) (Obayashi et al., body representation.2001). This was necessary because it was not We attempted to train monkeys with a kindpossible to measure the activation of these neu- of ‘‘video game’’; upon playing such a videorons during dynamic tool usage by extensive game, we humans feel as if our own images arescanning of space, which takes a few minutes to projected onto the avatar in the monitor (Irikicomplete, while PET cannot detect receptive et al., 2001). Monkeys were required to collectfields of the neurons. Using these complemen- food with a rake by watching an online videotary methods, we have demonstrated that intra- monitor view captured by a video camera. A rakeparietal neurons flexibly code modified body was necessary because with hands alone, theyimages upon tool use. could easily feel and grope for the food. While The coding of modifiable body image they watched their hand use the rake on thedescribed previously represents ‘‘enactive repre- screen, an artificial dot superimposed on thissentation’’ (Bruner et al., 1966) or an ‘‘internal video image was used to map the visual receptivemodel’’ (Kawato, 2008), which contributes to fields. Parietal neurons with tactile receptiveembedded control of the movement of body fields on the hand (Fig. 30.4G) were nowparts. This comprises the earliest mode of repre- endowed with a visual receptive field aroundsentation that humans could acquire through the video image of the hand (Fig. 30.4A).the course of their developmental processes In addition to the incorporation of the tool on(Bruner et al., 1966), which develop in three the video screen (Fig. 30.4E), this visual recep-stages, which further develop in additional two tive field was also modulated by virtual changesstages into iconic (or visual, 9 to 10 years old) in the size (Fig. 30.4B,C) and locationand symbolic (mature adult) representations. (Fig. 30.4C,D) of the hand, and even to the

618 PRIMATE NEUROETHOLOGY Tool use–related activities by its right hand A D f e d MT cerebellum c b a e IPS IPS insula B TEO preSMA TEO d MT PMv PMv TEO c C IPS) preSMA PMv b preSMA insula a PMv TEO R LFigure 30.3 Brain-activated areas related to tool use (compared with control task). (A) Dorsal view ofmonkey’s brain (a–f correspond to coronal sections in 3-D). (B) Right oblique (50 degrees) view ofcoregistered magnetic resonance image (MRI) with the positron emission tomography (PET) data. Notethat the contralateral intraparietal sulcus (IPS) area is projected onto the paramidline area. (C) Left obliqueview (50 degrees). (D) Coronal sections orthogonal to the AC–PC line. Adapted from Obayashi, S., Suhara,T., Kawabe, K., Okauchi, T., Maeda, J., Akine, Y., et al. (2001). Functional brain mapping of monkey tooluse. Neuroimage, 14, 853–861. Used with permission. IPS, intraparietal sulcus; SMA, supplementary motorarea; MT, middle temporal area; TEO, occipitotemporal transition; PMv, ventral premotor area.simulated disappearance of the hand leaving groping for the food. The space over the screen,only the tool’s end visible (Fig. 30.4F). Thus, hiding the hand from direct view, was scannedthe body images must be visually represented with the probe and the visual receptive field wasby the same group of parietal bimodal neurons. found in the space where the hand resided Parietal neurons were also able to encode the underneath the screen (Obayashi et al., 2000).image of the invisible hand under the table Wherever the hand moved under the screen,

EVOLUTION OF AN INTELLECTUAL MIND IN THE PRIMATE BRAIN 619 B F E A C G DFigure 30.4 Neural correlates of tool use under indirect visual control. Neural responses are recordedwhile monkeys retrieve small pieces of food and observe their actions on a video monitor, as captured by avideo camera. Visual (A—F) and somatosensory (G) neural responses in different viewing conditions.When monkeys use a monitor (experimental setup shown in right inset), a visual receptive field ofrepresentative intraparietal bimodal neurons was formed around the hand in the monitor (A),encompassing its somatosensory receptive field (G). The visual receptive field altered to match themodified appearance of the hand in the monitor (B–D), extending along the rake when used under themonitor (E) and confined around the tip of the rake once the image was blotted out except for the tip (F).Adapted from Maravita, A., & Iriki, A. (2004). Tools for the body (schema). Trends in Cognitive Science, 8,79–86. Used with permission.either actively or passively, the receptive field (Fig. 30.5B) to reach the long rake (Fig. 30.5C),moved to follow the invisible hand. The mon- and then changed the rake (Fig. 30.5D) tokeys and their parietal neurons can thus code retrieve the food (Fig. 30.5E) (Hihara et al.,and modify their body image in their mind 2003). Using PET, we found that, in additionalone, without necessity of actually seeing the to the intraparietal activation seen for single-toolbody. use, activation of prefrontal cortex was detected Having these abilities, we expected monkeys for this combinatorial use of the tools (Obayashito plan and sequentially combine the use of their et al., 2002) (Fig. 30.5H).body parts before actually executing a move-ment in their mind, using those introspective Induction of Gene Expression and Circuitbody images. When monkeys, pretrained to use Reorganization by Tool Use Traininga single rake, were exposed to a condition thatrequired combinations of different tools, they A minimum of about 10 days was required forcould solve the problem facing them within a tool use training to be completed, suggestingfew trials. This was unlike the initial training of that the process is not merely based on func-tool use, which necessitated about 2 weeks. In tional plasticity of existing neural pathways, butthe combination situation, the food was placed may involve anatomical reorganization of theat a distance that could not be reached by the circuitry. Indeed, the expression of immediatehand or a short rake that was within reach. Food early genes (Iriki, 2005) and the elevation ofcould only be reached by a longer rake, but this neurotrophic factors and their receptorsrake was not within reach itself (Fig. 30.5A). The (Ishibashi et al., 2002b) were coincident withmonkeys, without hesitation, took the short rake completion of the cognitive learning processes

620 PRIMATE NEUROETHOLOGY Sequential tooluse A B C D E Applied combination F G HFigure 30.5 Complex tool use in monkeys. (A—E) Experimental setting for the double-rake reachingstudy in monkeys. Experimental setting (F,G) and positron emission tomography (H) brain activation(critical intraparietal, pink arrow, and prefrontal activation foci, blue arrow) for the complex tool useexperiment in monkeys. Combinatory usages (sequentially from A to E) of short and long rake brainactivation pattern for sequential combinatory tool usages, showing prefrontal, in addition to parietal,activation. Adapted from Maravita, A., & Iriki, A. (2004). Tools for the body (schema). Trends inCognitive Science, 8, 79–86. Used with permission.(Ishibashi et al., 2002a). This expression was from the TPJ remained only in the deep layers offocused in the bank of the intraparietal sulcus the sulcus fundus, TPJ axons in trained monkeys(IPS) where the bimodal neurons described pre- extended further, by approximately 1.5 mm,viously are located, and returned to control into the more superficial layers of the bank tolevels once the learning process was completed. form active excitatory synapses with postsy-Thus, we hypothesized that some morphological naptic neurons (Fig. 30.6B). Through this newmodification or reorganization of the intrapar- anatomical connection, induced by a trainingietal neural circuitry should be evident because regimen, macaque monkeys have the capacityof training-induced genetic expression. to enable a novel mode of multimodal integra- To discover such reorganization, retrograde tion in the parietal cortex, and as a result,tracer (Fast Blue) was injected into the IPS area become able to use tools as an extension ofand the whole cerebral cortex was searched to their body parts.locate the neuronal cell bodies sending axons to How can tool use learning drive interactions ¨these areas, comparing naıve monkeys with between the TPJ and IPS? One potential impli-those monkeys that had extensive tool use cation would be that use of the tool as an exten-training (Hihara et al., 2006). In trained mon- sion of innate body parts induces a temporarykeys, two cortical areas (ventral prefrontal and mismatch with an existing body image stored intemporoparietal junction [TPJ]) were labeled the IPS region, and thus requires a recalibration ¨that were never labeled in naıve, control mon- driven by the monkey’s own intention to incor-keys (Fig. 30.6A). Subsequently, anterograde porate the external object (tool) into its internaltracer (biotinylated dextran amine) was injected body self-representation (Hihara et al., 2006;into the TPJ and patterns of axonal arbor and Iriki, 2006). Thus, tool use training may havesynaptic connections were explored in the IPS brought about explicit awareness of the mon-area, both at a light- and electron-microscopic key’s own body and own intentions (or mind)level. Unlike control monkeys, in which axons and eventually induced the subserving neural

Figure 30.6 Sites of biotinylated dextran amine (BDA) injections and distribution of anterogradelylabeled fibers and ultrastructural identification of labeled active synapses. (A) Schematic drawing of themacaque brain from a lateral view. Red dots indicate injection sites. Superior temporal sulcus (STS) ishighlighted green, and the characteristic branch at its posterior end, used as a landmark for identifyinginjection sites (see text), is indicated by an oblique arrow. The vertical bar shows a coronal section shown inB and the oblique bar indicates the sections of postcentral gyrus orthogonal to intraparietal sulcus (IPS)shown in C and D. (B) Coronal sections of injection sites, made into the temporoparietal junction (TPJ)area, in the most caudal part of the upper bank of the posterior portion of the STS where it branches. (C)Distribution of anterogradely labeled fibers in the anterior bank of the intraparietal sulcus, posterior to theSI-forearm regions. Camera lucida-based drawings from serial sections have been superimposed. BT1 andBT2 are tissues from trained monkeys in which sections covering a thickness of 250 mm have beensuperimposed. (D) Distribution of anterogradely labeled fibers in control monkeys (BC1 and BC2) inwhich sections covering a thickness of 500 mm have been superimposed. Note the difference in the thicknessof sections between C and D, indicating much denser labeling of fibers in trained animals. On the left inillustrations C and D (BT1, BC1), anterior portions of the postcentral gyrus that were cut off duringpreparing tissues are supplemented by imaginary dashed lines for better morphological understanding. (E)Ultrastructure of the BDA-positive boutons identified in a trained monkey. Reconstruction of ananterogradely labeled (presumably single) axon through 1,050 mm in the anterior bank of theintraparietal sulcus. A square in the right enlarged inset indicates a portion of the tissue dissected to beprocessed for electron microscopy. (F) Ultramicrograph of an anterogradely labeled terminal (filled arrow)in layer II, making an asymmetric synapse with the dendritic spine (open arrow) of postsynaptic neuron.Adapted from Hihara, S., Notoya, T., Tanaka, M., Ichinose, S., Ojima, H., Obayashi, S., et al. (2006).Extension of corticocortical afferents into the anterior bank of the intraparietal sulcus by tool-use training inadult monkeys. Neuropsychologia, 44, 2636–2646. Used with permission. 621

622 PRIMATE NEUROETHOLOGYconnections. Because of the newly formed TPJ- The Brain’s Latent Potential to ExhibitIPS connections, the subject can objectify its Training-Induced Higher Cognitivebody parts as equivalent to external tools. FunctionsThrough this process, primates are able to ‘‘dis- To date, rather simplistic comparisons haveembody’’ the sense of self from the constraints of been made between the cognitive abilities oftheir own physical body, and thereby this ability humans and various species of nonhuman pri-might have served as a latent precursor of mates. For example, the mental ability of apes is‘‘objectifying’’ themselves—that is, they develop thought to be comparable with 7-year-oldan ability to manipulate their own body as they human children, while a monkey’s is equivalentwould external objects. This might lead to the to a 2-year-old’s mental ability. However, theestablishment of the concept of metaself description of our results given previously hasallowing the development of human-like intelli- indicated that the reality is not so simple—mon-gence, as will be described in the following keys possess latent cognitive abilities that, oncesections. Indeed, in human subjects, activation exposed to the proper environment, can be rea-of this TPJ-IPS circuitry is detected in lized via expression of preprogrammed geneticself-objectification paradigms (Corradi- design, to widen their spectrum of cognitiveDell’Acqua et al., 2008). function. Figure 30.7 illustrates a conceptual Biology of ?3 Metaphysics Language Symbolic ?1 Education Representation Imitation Culture Self-recognition Training Iconic Representation Body image Tool use Development ?2 Enactive Representation Development N W H um a n Maneuver T N /W Monkey VirtualAxisFigure 30.7 Framework for the ‘‘biological science of human intellect.’’ See text for details. Adapted fromIriki, A., & Sakura, O. (2008). The neuroscience of primate intellectual evolution: natural selection andpassive and intentional niche construction. Philosophical Transaction of the Royal Society of London. Series B,Biological Science. 363, 2229–2241. Used with permission.

EVOLUTION OF AN INTELLECTUAL MIND IN THE PRIMATE BRAIN 623comparison of human and monkey spectra contact with a social system and received aunder this view. cultural education. Neural mechanisms for exhi- Various forms of intellectual abilities, biting such social systems might be a clue to thisroughly aligned in order of their complexity, problem (as shown by ‘‘?2’’ in Fig. 30.7); and (3)are shown as a virtual one-dimensional axis on extrapolation of evolutionary change into thethe left of Figure 30.7, from lower (bottom) to future (as shown by ‘‘?3’’ in Fig. 30.7). By under-higher (top) ones. The center and right columns standing neurobiological mechanisms of humanshow the spectrum of abilities of monkeys and intellect and its evolution, we may be able tohumans, respectively. ‘‘N’’ refers to normal con- predict, and may beneficially design, our futureditions, while ‘‘W’’ refers to living in the wild, technologies and social systems to match theand ‘‘T’’ refers to a situation under training. In natural requirements of our biological capacity.other words, it is possible for humans to advancetheir abilities through biological development,and thereafter through education and cultural EVOLUTION OF ANexperiences. This leads to markedly greater cog- ‘‘INTELLECTUAL MIND’’ IN THEnitive abilities, leading us to eventually become BRAIN: A HYPOTHESIS‘‘philosophers.’’ In contrast, the normal abilities Evolution of Primate Brain Capacityof monkeys in the wild develop through normal Through Mere Natural Selectionbiological developmental processes and stop atachieving dexterous movements, but do not A primitive nervous system of the evolutionarilyadvance much further. However, through an early simple animals has evolved into the brain.artificial process of training, they are able to At its highest end, sophisticated informationuse tools, and perhaps attain even higher intel- processing mechanisms, the ‘‘mind,’’is foundlectual abilities. This creates an overlap (shown only in more recently evolved complex animals,by the bold green square) in the cognitive abil- and it comprises the basis of humanistic intel-ities of monkeys who have received the proper lectual functions. Thus, it seems reasonable totraining and the intellectual functions of summarize the initial mechanisms of howhumans. This overlap could provide us with a organisms evolved such a nervous system.platform from which to study human intellec- The situation is clear when compared withtual functions as an empirical biological science. the organisms that have no nervous system. There are at least three questions to be con- Plants, for example, have prospered and adaptedsidered that could be subjects of biological ana- well to their environment, but they have nolyses: (1) the structure of intellectual brain brain. Plants reproduce and grow, but they dofunction itself, which might be revealed by elu- not move. In contrast, animals move—they arecidating how a monkey could acquire human- ‘‘animated.’’ They respond to stimuli from theirlike intelligent brain functions, and the max- environment by manipulating proteins in theirimum extent that could be achieved (as shown bodies to move. The nervous system evolved as aby ‘‘?1’’ in Fig. 30.7); (2) the reason why humans mechanism for processing the informationcan spontaneously advance in educational and acquired by sense organs and thereby for purpo-cultural achievement, whereas in monkeys, it is sefully controlling movement. As animals’necessary to have artificial training guided by bodies became larger and their senses andhumans. In other words, we must think of a system for movement became more complex,human raised in the wild in order to speculate there was a greater need for information proces-on the intellectual capacity of humans as a prising; the brain developed as the center for thismate species that could be studied from the purpose and stabilized through naturalperspective of the natural sciences. However, selection.such human specimens do not exist, and even For most animals, from single-celled organ-if they did, they would probably be able to man- isms to mammals, the basic movement is loco-ifest normal human abilities once they came into motion, or ‘‘intransitive movement’’ of the self.

624 PRIMATE NEUROETHOLOGYAn animal’s appendages for movement are used point. These primates were merely movingfor finding food or escaping danger, and have objects in response to the direct requirementsdeveloped the most efficient way to move the of their environments, thus a part of naturalbody for these purposes. Animals’ sense organs phenomena. These modes of evolution could bedetect the distance or direction for the best move- solely induced through conventionalment, and are adapted to provide the information Darwinian natural selection.necessary for the smooth generation of move-ments into action in an efficient way. In other Precursors of ‘‘Mind’’ Acquired inwords, the movement of an animal has adapted to Tool-Using Primate Ancestorsbe the most efficient way to respond to its envir-onment, as a part of environmentally embedded The situation took on a significant change whennatural phenomena. These situations apply to some of the primate species took up externalmost animals, except for humans and maybe objects in their hands, and used and movedapes, and we cannot perceive in most animals, these objects as an extension of their own bodytherefore, a ‘‘mind’’ that includes such things as (Sakura & Matsuzawa, 1991); this was the begin-will or intention. At times, we anthropomorphize ning of the use of tools. At this point, when toolsthe bodies or movement of animals and regard became a part of the human body, the body wasthem as if having a mind, but given the limits of objectified as a ‘‘thing’’ equivalent with a tool,control over their senses and movement—in and was represented as such within the brain.turn, a limitation of their nervous systems— When one’s own body becomes objectified andthere is no necessity to perceive such movements separate, one must assume a subject with anas indicative of a mind. With regard to the impor- independent status within the functioning oftant role of intransitive movement, as such, there the nervous system that moves the body andis an indivisible unity between the brain and the other objects (Iriki, 2006). Thus, the mindrest of the nervous system, the ‘‘subject’’ that could be regarded as a sort of virtual conceptcontrols the movement of the body, and labeled to a putative functional component tothe physical organs for movement that are the link the subject and the object of the transitive‘‘objects’’ of this control. In other words, movements, the latter of which are induced bythe scope of the nervous system is limited to the the function of the nervous system.body and always unified with it, so it is unreason- Indeed, several recent studies report howable to divide the two into subject and object. laboratory-raised, nonhuman primates exposed When some animals acquired the ability to to tool use can exhibit intelligent behaviors, suchfree their hands from the sole task of moving as imitation and referential vocal control, whichthe body, they were able to manipulate external are never seen in their wild counterparts (Iriki,objects in their environment. Among these spe- 2006) (for further reading on vocalizations, seecies of animals, primates are especially impor- Chapters Miller & Cohen and Romanski &tant to unravel the origin of the human mind. Ghayanfar). Tool use training appears to forgeWhen ancestral species of primates were able to novel corticocortical connections that underlieconfirm the results of their manipulation in this boost in capacity, as described in the pre-detail through their binocular vision, the vious sections, and normally exists only as aneural network in their brains evolved to latent potential in wild nonhuman primates.adapt to these advances, and their situation Although tool use training is patently nonnatur-began to change (Iriki, 2008). The physical alistic, its marked effects on brain organizationmovement of animals began to include the and behavior could shed light on the evolutiontransitive action of moving other objects. This of higher intelligence in humans.led to a separation between the body as the Consequently, how do higher cognitive func-subject that moves something and the physical, tions change once an assumed mind is within theouter object that is moved. However, there is functioning of one’s nervous system? The sub-still no necessity to assume a will or mind at this ject might become aware of the continuity of the

EVOLUTION OF AN INTELLECTUAL MIND IN THE PRIMATE BRAIN 625body across time, gradually establish a concept changes, and thus behaviors should be theof the self, and unavoidably assume the existence ‘‘engine’’ of the evolutionary process itselfof a mind in others as well. Further, the action of (Plotkin, 1988). Apart from some classical phi-moving an object transcends material objects losophical arguments, this kind of argumentand reaches out to the subject of others, and originated from Darwin (1881) himself, andminds would begin to function cooperatively. has recently been re-evaluated as ‘‘niche con-In this way, numerous subjects with a mind struction,’’ which may be equally importantbegin to have mutual recognition, and a during evolutionary processes (Odling-Smee‘‘theory of mind’’ (Premack & Woodruff, 1978) et al., 2003).is born (see Rosati et al., this volume). Then Two points, however, are still open questions.various actions based on the workings of the First, how much behavioral change is importantminds—empathy, judgment, sympathy, com- in contributing to phenotypic evolution? Somepassion, imitation, education—contribute to researchers state that an organism’s reaction tothe development of a culture and society that is its environment should be regarded as of equalrich with humanity, and the effects of the minds importance as natural selection, like the nichedevelop further through the activities of this construction model, while others see this role associety. These effects of the mind eventually negligible. We suppose that the history of thegive birth to a spiritual civilization that is based idea will prove that the role of behavior in evolu-on self-control and overcoming selfishness. On tion is more important. Darwin himself pointedthe other hand, once this phenomenon further out the role, and it has since been neglected foradvances, this might lead to a scientific and more than a hundred years. At last, recenttechnical civilization in which nature is an empirical research has led to its re-evaluation.object to be manipulated. Thus, precursors of This pattern is similar to that of sexual selection,the intellect required to make an important next so we believe the trend will continue.step toward a cultural explosion were present in The second question concerns the physiolo-the primitive primate ancestors of the hominids. gical mechanism within the organism to realize Neural circuitry in the primitive primate such a process. The neurobiological mechanismbrain might have been exposed to these func- as previously depicted may constitute a part oftional processes expressed as behavioral charac- such niche construction processes during theteristics, and some biological changes may have course of evolution. The structure and functionbeen induced to be selected and stabilized of the central nervous system varies among spe-through natural selection. Indeed, we have cies; thus, it is reasonable to assume that the wayshown earlier that demanding pressure from in which the behavior of each species reacts tothe environment (maybe within a range of pre- the environment also varies. However, under-programmed genetic expression in the brain) standing human evolution is difficult becauseresults in reorganization of the neural circuitry, we lack a full understanding of how humanat least by axogenesis and synaptogenesis as intentionality affects the evolutionary process.described earlier, to modify the pattern of beha- We have not succeeded in uniting nonhumanvioral expressions. In these directions, a poten- evolutionary theory with that of humanstial precursor of evolutionary changes, including (Laland & Brown, 2002). Thus, to elucidateneurogenesis, leading to brain expansion could neural mechanisms of human intellectual evolu-be possessed latently, and could guide the tion, some additional mechanisms need to bedirection of evolutionary change. postulated. Following tool use, either through externali- zation of the body parts or alternative internali-Niche Construction—Role of Behaviors in zation (or incorporation) of objects into theEvolutionary Theory body image, mutual interaction betweenA fact that has been repeatedly emphasized is organisms and the environment emerges. Inthat changes in behavior precede morphological this process, tools comprise a cultural trace

626 PRIMATE NEUROETHOLOGYembedded in the environment and thus traced in the structure of the present ‘‘civilizedmodifying it. This is then inherited across environment’’ and in the form of various artifi-generations. This constructed environment cially manufactured tools. These tools serve asthen forces the genetic expression of brain func- ‘‘mental fossils.’’tion to exhibit phenotypes (morphologicalfeatures or neural circuitry) that match the Stepwise Mastery of Higher Classes ofusage of such cultural traces. This process of Tools—Intentional Niche Constructionorganism-environment interaction shouldreach equilibrium after a few cycles and then be Existing studies on tool use have focused solelycomplete. Thus, this possible modification of on those extending only one aspect of our phy-phenotype is a result of fluctuations within the sical functions, that is, producing motor actions.preprogrammed range of genetic codes. Such Indeed, the typical definition of the ‘‘tool’’ per seinteraction is formed passively (or autono- belongs to this class of tool (Asano, 1994; Beck,mously) by interactions between the subject 1980). These tools cover a wide range of com-and the environment. Thus, we can call this plexities, from a rake as a mere spatial extensionprocess ‘‘passive niche construction.’’ of the hand to a technical device or machinery As long as it remains passive (or autono- composed of multistep actions and used by var-mous), once stabilized, an acquired state would ious animals, including both vertebrates andbe sustained without further change, unless the invertebrates (Beck, 1980). In addition to thesenatural environment or mode of activity of motor tools, humans (but not nonhuman ani-organisms changes accidentally. Indeed, the mals) use other categories of tools, that is, toolsfirst primitive stone tool used by our hominid to assist individuals to obtain sensory informa-ancestors did not change for over 1.5 million tion (Goldenberg & Iriki, 2007; Fig. 30.8). Theseyears (Shennan, 2002). They were stabilized in tools include a prism or a mirror to merely shiftthis novel mode of interaction, maybe because the gaze angle and can be advanced to technolo-those hominids lacked the ability to actively (or gical devices for exploring information unde-intentionally) modify the mode of interactions tectable by our own innate sensory organsaccording to insight. This may be because those (e.g., a radio-detector [cf. the similar, butprimate ancestors did not have a human-like slightly different, classification of tools as pro-mind or subjective self and thus could not expli- posed by Asano, 1994]). This clear discontinuitycitly imitate (Iriki, 2006) or intentionally plan. between humans and nonhuman animals in toolEvolutionary traces of successive additional use can be a clue to building a possible pathfactors throughout our history can possibly be of cognitive evolution occurring through Brain Memory Thought Periphery Sensory Mirror/Camera Detector Motor Stick Mechanics Origin of tool Externalized organ Simple extension Combination/CalculationFigure 30.8 Hierarchical structure of various classes of tools. Adapted from Iriki, A., & Sakura, O. (2008).The neuroscience of primate intellectual evolution: natural selection and passive and intentional nicheconstruction. Philosophical Transaction of the Royal Society of London. Series B, Biological Science. 363,2229–2241. Used with permission.

EVOLUTION OF AN INTELLECTUAL MIND IN THE PRIMATE BRAIN 627interaction between users and environments. completely guided by the sensory toolIt could further help explore brain mechanisms (Fig. 30.9C). Thus, the hominoids’ uniquefor sensory tool use, acquired through an accu- degree of utilization of their environment canmulating past history of interactions between be simulated in monkeys by reconstructing theindividuals and environments, which ultimately conditions for tool use with various types ofserve to functionally externalize different parts cognitive load. Humans may have sponta-of the innate body. neously been shaped by similar operant condi- To know whether this evolution of tool use tioning situations. This induction of sensorycan be established in nonhuman primate tool use in a nonhuman primate constitutes aJapanese monkeys by simulating interactions significant modification in cognitive func-between individuals and environments, we tioning that is brought about through a circularaimed to train the monkey to use a sensory interaction between the individual and environ-tool, which, like an endoscope, was a rake with ment, and thus offers a novel paradigm for thea tiny camera mounted at its tip (as an externa- empirical study of human cultural evolution.lized eye). This tool was used to explore unview- Increasingly higher classes of tools should beable places to search for food by watching the successively incorporated into the environmentcaptured image projected onto a viewable and subsequently form a selective pressure formonitor. For the preliminary training to learn evolution, which is accelerated over generations.the sensory tool, we trained monkeys to use Although each step might be a simple associa-rakes to obtain food by watching the camera tion, it might produce something beyond whencapturing the image on the table with a piece of the whole scheme was structuralized. One suchfood (Yamazaki et al., 2009). The monkeys’ task candidate is the concept of the self (and accom-was to associate the placement of the rake with panying ‘‘intention’’ of the subjective self),the food on the monitor and to minimize the which has emerged through the self-objectifica-distance between them in order to reach the tion process, as described in the previousfood. Although akin to the experiment shown section. Thus, niche construction became inten-earlier, the results were disappointing. The mon- tional to accelerate evolution remarkably. Underkeys never learned this task by simply trying to such ‘‘intentional niche construction,’’ theassociate this sensory cue with motor control— direction of evolution is no longer passivelynotwithstanding introducing several supportive determined by the environment (Fig. 30.10,procedures to help monkeys utilize the monitor left), but living organisms themselves canimage. The success rate to obtain food by any of decide, through interactive mechanisms amongthe tasks never exceeded chance level after society members, the direction of environmentaltraining for more than 3 months. Mere sensor- modification, perhaps using various tools, andimotor learning does not lead to the eventually in turn influence their own evolu-development of this function. tionary outcome (Fig. 30.10, right). Subsequently, the training process was Assuming the kinds of physical toolsrestructured. Thinking that an ability to utilize described previously could be integrated, thethe modified body images by motor tools may be next plausible category of tools would be thosea necessary prerequisite, we attached a sensory that extend the functions of the brain itself, orcue the tip of the motor tools, such as a dental ‘‘metaphysical’’ modalities. Painting and writingmirror. Use of this tool was then surprisingly (of pictures and script) as ‘‘external memoryeasy for monkeys to acquire (Fig. 30.9A). devices’’ may have first emerged as a simpleGradually, the mirror and the rake were sepa- form of such tools. This invention accompaniedrated, and eventually the motor tool and the the emergence of various novel concepts in oursensory tool were both used independently: brain, such as the sense of existence across timeTraining separated visual cues from their actual and space. These tools further facilitated theorigins in visuomotor space until exploration development of materials for record keeping,(Fig. 30.9B). Reaching and food retrieval were such as tape recorders, hard disks, or memory

A. Sensorimotor tool like a dental mirror mirror Mirror rake B. Tools for exploration for the food Active localization Passive localization Manual mirror Stand mirror Nonmirror rake RC mirror Monitor C. Sensory tool for exploration and reaching the food Camera rakeFigure 30.9 Structures of sensory tool usage acquisitions. (A) A mirror rake was used to find thefood placed behind the raised apparatus. (B) Tools for exploration for the food. For active searching(left panel), the manual mirror (upper part of the panel) and the remote-controlled mirror (lower)were used to locate the hidden food. For the passive searching (right panel), the stand mirror wasused to reflect the hidden food, and the monitor was used to show images of the hidden foodcaptured by the video camera. In both cases, the food was retrieved using the nonmirror rake aftersearching. (C) Sensory tool use. The tool was a rake mounted with a small camera, with atransparent tip. The camera captured the image in front of the camera under the screen, so thatthe monkeys can search for the food by looking at the monitor showing the captured image. Adaptedfrom Yamazaki, Y., Namba, H., & Iriki, A. (2009). Acquisition of an externalized-eye by Japanesemonkeys. Experimental Brain Research, 194, 131–142. Used with permission. 628

EVOLUTION OF AN INTELLECTUAL MIND IN THE PRIMATE BRAIN 629 Environment Environment Hominin Human 3 Enviromental manipulation Enviromental change Human 2 Enviromental manipulation Tool Primate ancestor Human 1 Environment Environment Intentional Natural selection niche constructionFigure 30.10 Comparison of evolutionary mechanisms before early hominids by ‘‘natural selection’’ (left)and modern human intellect by ‘‘intentional niche construction’’ (right). Adapted from Iriki, A., & Sakura,O. (2008). The neuroscience of primate intellectual evolution: natural selection and passive and intentionalniche construction. Philosophical Transaction of the Royal Society of London. Series B, Biological Science. 363,2229–2241. Used with permission.chips, to pass on and improve new abilities for of relational concepts (brain and mind) andlong periods. At present, modern computer materialistic concrete concepts (physical body)technologies are starting to provide us with not may benefit from insightful philosophical pre-only memory storage but also ‘‘external thinking dictions. As a methodology, study guided bydevices.’’ These externalized brains could be abduction and Eastern philosophy wouldshared and assimilated with each other and increase its importance with the addition ofmodified by other society members, and thus induction and reduction methodology of thepotentially modify modes of their own brain Western philosophical frameworks.functions beyond the range of plasticity that The evolution of a civilized environment bycan be modulated through individual behavioral directed intention described previously might behistories. Thus, current evolution is accelerated of concern because of the chaotic uncertainty offurther than ever before, and we may be entering the resulting state of the environment. However,a critical and novel epoch for human evolution this structure of uncertainty is constrained by(cf. Moravec, 1988). the natural functions of primate prefrontal and Following the introduction of goal-directed parietal cortices. How can we predict thoseintentional niche construction into the evolu- ‘‘future’’ brain functions (‘‘?3’’ in Fig. 30.7)?tionary processes, our ‘‘civilized environment’’ As mentioned, we could extrapolate our futureacquired a structure composed of the intentional from deep understandings about neurobiolo-concept and goal-directed behavior, along with gical mechanisms subserving human intellectualits artificial products (such as tools, machineries, functions of the present and the past through theand architectures). All of these are controlled by evolutionary processes, as described in thisa human society composed of self-established chapter. Once the mind that dwells in ourindividuals. This would mean that biological- brains emerges, what will it create next, andgenetic processes and literary-cultural processes where is it headed? Humanity faces a new situa-became formally integrated and interacted. This tion, one that living organisms have never yetsuggests the necessity of integrated scientific stu- experienced: that of the possibility of numerousdies of those multiple and diverse fields. In this ‘‘minds’’ existing in an external thinking devicesense, scientific studies that allow manipulation (Fig. 30.8, top right) linked simultaneously

630 PRIMATE NEUROETHOLOGYthrough electronic communication in an Duhamel, M. D. Hauser, & G. RizzolattiInternet society. In such a situation, the wills of (Eds.), From monkey brain to human brainindividual subjects become separate from their (pp. 253–271). Cambridge, MA: MIT Press.bodies and act mutually through the indepen- Iriki, A. (2006). The neural origins anddent functioning of the Internet, in which the implications of imitation, mirror neurons and tool use. Current Opinions in Neurobiology, 16,shards of a thousand selves form the community 660–667.of a virtual society. In such situations, some Iriki, A. (2009). Posterior parietal cortex and tooladditional virtual concept of an advanced form usage and hand shape. In: Squire LR (ed.)of ‘‘multiselves’’ might emerge that would exist Encyclopedia of neuroscience, volume 7, pp.in future extensions of the neurobiological 797–802. Oxford: Academic Press.mechanisms depicted here. Iriki, A., & Sakura, O. (2008). The neuroscience of primate intellectual evolution: natural selectionREFERENCES and passive and intentional niche construction. Philosophical Transaction of the Royal Society ofAsano, T. (1994). Tool using behavior and London. Series B, Biological Science. 363, language in primates. In: S. C. Hayes, 2229–2241. L. J. Hayes, M. Sato, & K. Ono (Eds.), Iriki, A., Tanaka, M., & Iwamura, Y. (1996). Behavior analysis of language and cognition Coding of modified body schema during tool (pp. 145–148). Reno: Context. use by macaque postcentral neurons.Beck, B. B. (1980). Animal tool behavior. New York: Neuroreport, 7, 2325–2330. Garland. Iriki, A., Tanaka, M., Obayashi, S., & Iwamura, Y.Bruner, J. S., Olver, R. R., & Greenfield, P. M. (2001). Self-images in the video monitor coded (1966). Studies in cognitive growth. New York: by monkey intraparietal neurons. Neuroscience Wiley. Research, 40, 163–173.Corradi-Dell’Acqua, C., Ueno, K., Ogawa, A., Ishibashi, H., Hihara, S., & Iriki, A. (2000). Cheng, K., Rumiati, R. I., & Iriki, A. (2008). Acquisition and development of monkey tool- Effects of shifting perspective of the self: An use: Behavioral and kinematic analyses. fMRI study. NeuroImage, 40, 1902–1911. Canadian Journal of Physiology andDarwin, C. (1881). The formation of vegetable Pharmacology, 78, 958–966. mould through the action of worms, with Ishibashi, H., Hihara, S., Takahashi, M., Heike, T., observations on their habits. London: John Yokota, T., & Iriki, A. (2002a). Tool-use Murray. learning selectively induces expression ofGoldenberg, G., & Iriki, A. (2007). From sticks to brain-derived neurotrophic factor, its receptor coffee-maker: Mastery of tools and technology trkB, and neurotrophin 3 in the intraparietal by human and non-human primates. Cortex, cortex of monkeys. Cognitive Brain Research, 43, 285–283. 14, 3–9.Head, H., & Holmes, G. (1911). Sensory Ishibashi, H., Hihara, S., Takahashi, M., Heike, T., disturbances from cerebral lesions. Brain, 34, Yokota, T., & Iriki, A. (2002b). Tool-use 102–254. learning induces BDNF in a selective portionHihara, S., Notoya, T., Tanaka, M., Ichinose, S., of monkey anterior parietal cortex. Molecular Ojima, H., Obayashi, S., et al. (2006). Brain Research, 102, 110–112. Extension of corticocortical afferents into the Kawato, M. (2008). From ‘‘Understanding the anterior bank of the intraparietal sulcus by brain by creating the brain’’ toward tool-use training in adult monkeys. manipulative neuroscience. Philosophical Neuropsychologia, 44, 2636–2646. Transactions of the Royal Society B, 63, 2201–Hihara, S., Obayashi, S., Tanaka, M., & Iriki, A. 2204. (2003). Rapid learning of sequential tool use by Laland, K. N., & Brown, G. R. (2002). Sense and macaque monkeys. Physiological Behavior, 78, nonsense: Evolutionary perspectives on human 427–434. behaviour. Oxford: Oxford University Press.Iriki, A. (2005). A prototype of Homo faber: A silent Maravita, A., & Iriki, A. (2004). Tools for the precursor of human intelligence in the toolbody (schema). Trends in Cognitive Science, using monkey brain. In: S. Dehaene, J.-R. 8, 79–86.

EVOLUTION OF AN INTELLECTUAL MIND IN THE PRIMATE BRAIN 631Moravec, H. (1988). Mind children. Cambridge, Premack, D., & Woodruff, G. (1978). Does the MA: Harvard University Press. chimpanzee have a theory of mind? BehavioralObayashi, S., Suhara, T., Kawabe, K., Okauchi, T., and Brain Sciences, 4, 515–526. Maeda, J., Akine, Y., et al. (2001). Functional Sakura, O., & Matsuzawa, T. (1991). Flexibility of brain mapping of monkey tool use. wild chimpanzee nut-cracking behavior using Neuroimage, 14, 853–861. stone hammers and anvils: An experimentalObayashi, S., Suhara, T., Nagai, Y., Maeda, J., analysis. Ethology, 87, 237–248. Hihara, S., & Iriki, A. (2002). Macaque Shennan, S. (2002). Genes, memes and human prefrontal activity associated with extensive history: Darwinian archeology and cultural tool use. Neuroreport, 13, 2349–2354. evolution. London: Thames & Hudson.Obayashi, S., Tanaka, M., & Iriki, A. (2000). Tomasello, M., & Call, J. (1997). Primate cognition. Subjective image of invisible hand coded by Oxford: Oxford University Press. monkey intraparietal neurons. Neuroreport, 16, Ungerleider, L. G., & Mishkin, M. (1982). Two 3499–3505. cortical visual systems. In: D. J. Ingle, et al.Odling-Smee, F. J., Laland, K. N., & Feldman, M. (Eds.), Analysis of visual behavior W. (2003). Niche construction: The neglected (pp. 549–586). Cambridge: MIT Press. process in evolution. Princeton, NJ: Princeton Yamazaki, Y., Namba, H., & Iriki, A. (2009). University Press. Acquistion of an externalized-eye by JapanesePlotkin, H. C. (Ed.). (1988). The role of behavior in monkeys. Experimental Brain Research, 194, evolution. Cambridge, MA: MIT Press. 131–142.

AUTHOR INDEXAbbott, R. J., 164 Anderson, A. W., 125Adams, S. R., 72 Anderson, D., 109Addessi, E., 146 Anderson, J. R., 104, 121, 122, 123, 125, 126, 128,Adenzato, M., 576 535, 606Adolphs, R., 300, 308, 540 Anderson, M. L., 396Aerts, P., 32 Andreoni, J., 167, 168, 170Aflalo, T. N., 457, 458, 460 Andy, O., 20Agnetta, B., 533 Annese, J., 433Aharon, I., 528, 542 Ansari, D., 398Ahn, A. N., 48, 50, 51 Antinucci, F., 8Ahrens, E. T., 427, 429 Antonarakis, S. E., 439Aiello, L. C., 65, 205 Aosaki, T., 321Ainslie, G., 107 Aou, S., 375Akine, Y., 618 Apicella, P., 320Akkal, D., 195 Appenzeller, O., 297Albert, M., 152 Arathorn, D. W., 491Albrecht, D. G., 206 Arbib, M. A., 88, 93, 265, 505, 588, 603Albright, T. D., 350 Argyle, M., 538Alcock, J., 163, 166 Armstrong, E., 65Alexander, G. E., 213, 257, 309, 320, 398, 407, 563 Armstrong, K. M., 230Alexander, M., 305 Arnason, U., 13Alexander, R. D., 167 Arnold, K., 88, 92, 244Alexander, R. M., 33, 50 Arrow, K. J., 167Alexander, R. McN., 48, 51 Asaad, W. F., 247, 356, 408, 418Algom, D., 148 Asano, T., 626Allais, P. M., 99 Asanuma, H., 454, 455Allan, L. G., 110 Ashby, F. G., 242, 247Allison, T., 476, 540, 542 Ashe, J., 257, 260Allman, J. M., 165, 407, 424, 427, 429, 434, 526, 531 Assad, J. A., 243, 245, 301, 324, 551Allmann, J., 202 Atick, J. J., 205, 206Altheide, T. K., 439 Attneave, F., 205Amador, N., 321 Augustine, J. R., 298Amaral, D. G., 297, 298, 339, 373, 374, 375, 542 Averbeck, B. B., 395, 513, 514, 551, 560, 561, 566Amiez, C., 414 Axelrod, R., 167Amit, D. J., 348Amunts, K., 433 Bachevalier, J., 300, 303, 373, 374, 376Anapol, F. C., 33 Bachorowski, J. A., 91Ancrenaz, M., 593 Baddeley, A., 442Andersen, R. A., 189, 213, 276, 281, 289, 467, 543 Badre, D., 247, 413, 415, 416 633

634 AUTHOR INDEXBailey, J. A., 439 Bechara, A., 302Bailey, J. N., 367 Beck, B. B., 588, 590, 626Bailey, J. T., 106 Beck, D. M., 214Bailey, P., 339, 478 Beck, P. D., 183Baillargeon, R., 118, 130 Beck, R. M. D., 20Baizer, J. S., 300, 506 Beer, J. S., 165Baker, R., 281 Beevor, C., 454Baker, S. N., 264 Behar, I., 105Bakin, J. S., 207 Behne, T., 118Baldwin, D., 92 Behrens, T. E., 430Balhoff, J., 17 Bell, A. J., 211Ballard, D. H., 205 Bendor, D., 245Balter, M., 432 Benefit, B., 26Bandler, R., 295 Benevento, L. A., 509, 514Banks, W. P., 148 Bennett, A. T. C., 67Bao, S., 333 Bennett, J., 369, 371Bar, M., 484 Benson, D. A., 445Barbas, H., 299, 407, 506, 507 Bentley, D. R., 439Barbour, D. L., 245 Bentley, R. A., 164Bar-Haim, Y., 536 Beran, M. J., 110, 144, 146, 147, 149Barkow, J. H., 100, 110 Bercovitch, F. B., 305, 536Barks, S. K., 432 Berezikov, E., 440Barlow, H. B., 205, 208, 209 Berg, A., 492Barnes, C. L., 511 Berg, E. A., 412Barnes, J. L., 595 Bergman, A., 86Baron, J., 101 Bergman, T. J., 243, 572Baron-Cohen, S., 533, 537, 539, 542 Berman, K. F., 413Barone, P., 398, 561 Berns, G. S., 163Barr, C. S., 362, 371 Bernstein, L. E., 510Barraclough, D. J., 551, 557 Berntson, G. G., 103, 149, 151, 396Barraclough, N. E., 300, 508, 509, 515, 516, 517 Berretty, P. M., 242Barrett, L., 7, 570–584, 606 Berson, D. M., 177Barrey, E., 50, 51 Bertelson, P., 234Bartels, A., 295 Bertram, J. E. A., 32Barth, H., 121, 122, 147, 150, 152, 156, 504 Besle, J., 510Barth, J., 124 Bhatnagar, K. P., 184Barto, A. G., 163, 317, 551, 555, 559 Bialek, W., 210Barton, R. A., 64, 65, 76, 201, 202, 526 Bicca-Marques, J. C., 72Basbaum, A. I., 177 Bickel, W. K., 108Basmajian, J. V., 32 Bickerton, D., 88Basso, M. A., 551 Biederman, I., 483, 484, 490Bates, L., 68 Biewener, A. A., 51, 53Bateson, M., 105, 108 Biknevicius, A. R., 48Batista, A. P., 289 Biro, D., 601, 602Bauer, H. R., 501 Bishop, P., 50, 53Bauman, M. D., 305, 306, 307, 374 Bisley, J. W., 213, 301Baxter, M. G., 302, 303 Biswal, B., 431Bayer, H. M., 551 Bizzi, E., 281Bayer, R. H., 148 Black, F., 163Baylis, L. L., 407 Blakeslee, S., 414Bayliss, A. P., 536, 539, 542 Blank, C. C., 244Beard, K. C., 12, 23 Blanke, O., 230Beaton, A. A., 587 Blauert, J., 280

AUTHOR INDEX 635Blessing, W. W., 294 Bridge, H., 428, 593Blickhan, R., 49, 51, 53 Brincat, S. L., 481, 482Bloch, J. I., 16 Britten, K. H., 213Blond, O., 333 Brodmann, K., 428, 433, 563Bloom, P., 92 Brody, C. D., 392Boatright-Horowitz, S. L., 526, 528, 530 Brokx, J. P., 504Boch, R., 226 Brooks, R., 118, 572, 573, 574, 576Bocxlaer, I., 24 Brooks, S. A., 440Bodner, M., 513, 514 Brosch, M., 246Boesch, C., 69, 88, 92, 131, 601, 602 Brosius, J., 14Boesch, H., 69 Brosnan, S. F., 102Boettiger, C. A., 418 Brothers, L. A., 373, 375Boinski, S., 65 Brown, E. N., 352, 354, 355Boisvert, M. J., 147 Brown, G. R., 625Boitani, L., 202, 203 Brown, J. H., 164Boldrey, E., 454 Brown, S., 373Boles, T. L., 169 Brown, V. J., 321Bombardieri, R. A., 179 Bruce, C. J., 300, 457, 487, 509, 511Bongard, J., 572, 573, 574, 576 Bruce, V., 533Bongiolatti, S. R., 416 Bruner, J. S., 617Bonner, J. T., 164 ¨ Buchel, C., 302Borgen, G., 593 Buckley, M. J., 338Bornstein, R. F., 532 Bucy, P. C., 373, 374Bosseler, A., 518 Budescu, D. V., 553Botchin, M. B., 369, 370 Bugnyar, T., 135, 533Botvinick, M., 399 Bukach, C. M., 208Boussaoud, D., 472 Bullier, J., 507Bowles, S., 165, 166 Bulthoff, H. H., 484Bowman, E. M., 320, 562 Bunge, S. A., 247, 409, 414, 415Bown, T., 23 Bunsey, M., 338Boyd, R., 164, 168 Burkhalter, A., 178Boye, M., 500 Burkhart, J., 121, 122, 125, 126, 128Boynton, G. M., 476 Burks, S. V., 169Boysen, S. T., 103, 149, 151, 396, 590 Buschbaum, G., 206Boyton, R. M., 242 Bush, E. C., 407Bradley, B. P., 536 Bush, G., 299Bradshaw, J. L., 587, 588, 603 Bushnell, M. C., 227Brady, D. R., 500 Bussey, T. J., 204, 417Brady, J. V., 504 Butler, R. A., 530Braesicke, K., 303 Buttelmann, D., 92Braeuer, J., 121, 122, 124, 127, 130, 131 Butterworth, B., 391Brandt, K. L., 89 Buzsaki, G., 509Brannon, E. M., 5, 102, 144–156, 385, 392 Byrd, L. D., 376Brass, M., 413 Byrne, J. M., 597Brasted, P. J., 349, 352, 356 Byrne, R. W., 64, 65, 66, 67, 68, 69, 70, 72, 78, 588,Brauer, J., 533 597, 605, 607Braver, T. S., 413, 416Breazeal, C. L., 579 Cabel, D. W., 257Brecht, M., 268, 466 Cabeza, R., 300Bregman, A. S., 239, 240 ´ Caceres, M., 438, 440, 441Breitmeyer, B. G., 229 Cadieu, C., 7, 471–495Bremmer, F., 389 Caggiula, A. R. 457Breukelaar, J. W. C., 150 Cahusac, P. M., 352, 353, 354

636 AUTHOR INDEXCaine, N. G., 73, 526, 528, 530, 531 Chawarska, K., 539Calabresi, P., 333 Chechik, G., 247Calarco, J. A., 440 Chelazzi, L., 212Calderone, J. B., 203 Chemnick, L. G., 18Caldwell, C. A., 124, 127 Chen, L. L., 356Call, J., 112, 117, 119, 120, 121, 124, 126, 127, 135, Chen, M. K., 106 136, 551, 560, 588, 594, 605, 615 Cheney, D. L., 5, 84–93, 118, 132, 207, 243, 304,Calton, J. L., 189 505, 506, 510, 527, 531, 532, 571, 572Calvert, G. A., 508 Cheney, P. D., 257, 455Camerer, C. F., 553, 556, 557 Cheng, K., 472, 473Cameron, E. L., 226 Cheng, Z., 439Camille, N. 165 Chiarelli, A., 23Caminiti, R., 260, 261 Chieffi, S., 265Campbell, F. W., 206 Christel, M. I., 596Candland, D. K., 528, 588, 590 Christoff, K., 414Cant, J. G., 43 Christopherson, K. S., 440Cantalupo, C., 599, 600 Churakov, G., 14Cantero, M., 603 Church, R. M., 145, 146, 152, 154, 387, 391, 392Cantlon, J. F., 144, 145, 146, 147, 148, 149, 150, 152 Churchland, M. M., 257Capitanio, J. P., 31 Churchland, P., 573Caraco, T., 105, 163 Ciochon, R. L., 23Carello, C., 576 Cisek, P., 550Carey, S., 152, 472 Clare, S., 428Carlson, K. J., 33 Clark, A., 573, 574, 576Carmi, R., 529, 540 Clark, C. J., 39, 53Carosi, M., 307 Clark, J. J., 533Carpenter, A. F., 399 Clark, V. P., 428Carpenter, M., 93, 118, 119, 121, 124, 127, 136 Clarke, H. F., 412Carr, C. E., 182 Clarke, S., 434Carrasco, M., 229 Clayton, N. S., 135Carter, A. N., 134 Cleveland, A., 367, 370Carter, C. S., 376 Clower, W. T., 398, 563Cartmill, E., 605 Clutton-Brock, T. H., 65Cartmill, M., 4, 10–27, 39, 40, 45, 49, 52, 55, 64, 202 Cohen, J. D., 389, 396, 398Casagrande, V. A., 184, 186, 424 Cohen, J., 425, 440Castelli, F., 393 Cohen, L., 396Catmur, C., 576 Cohen, N. J., 337, 349, 351, 355Cavagna, G. A., 49, 51, 52, 53 Cohen, Y., 526Cavalli-Sforza, L. L., 164 Cohen, Y. E., 233, 237–249, 301, 513Cavanaugh, J., 229, 230 Cola, M. G., 433Cave, K. R., 224 Colby, C. L., 387, 389, 467, 543Cenzato, M., 259 Cole, C. E., 296Chafee, M. V., 205, 388 Colell, M., 602Chakrabarty, S., 455 Coleman, G. Q., 434, 435, 436, 437Champoux, M., 369 Coleman, J. S., 163Chance, M., 531 Collins, C. E., 184Chandrasekaran, C., 9, 508, 509 Collins, P., 410Chang, Y. H., 32, 38 Colombo, J. A., 434Changeux, J. P., 391, 392 Connor, C. E., 471, 481, 482, 484Changizi, M. A., 202, 481 Conrad, B., 439Chapman, C. A., 64, 70, 75, 76 Constantinidis, C., 213, 225, 407Chauvin, C., 65, 72 Cook, J. M., 13Chavis, D. A., 506, 507 Cook, M., 538

AUTHOR INDEX 637Cook, R. G., 76 Davis, P. J., 295Cooke, B., 365 Daw, N. D., 550Cooke, D. F., 189, 456, 457, 459, 462, 463, 467 Dawes, R. M., 170Cooper, E. E., 483, 484 Dawkins, R., 164, 165, 167Corballis, M. C., 587, 603 de Faria, D. S., 109Corbetta, M., 526 de Gelder, B., 296Cordes, S., 147, 387 De Martino, B., 528Corkin, S., 337 De Santis, L., 244Cornell, R., 373 De Valois, R. L., 206, 207Corp, N., 64, 588 de Waal, F. B. M., 131, 132, 133, 307, 532, 538, 570,Corradi-Dell’Acqua, C., 622 571Cossart, R., 581 Deacon, T. W., 398, 587Courtney, S. M., 213 Dean, P., 9, 296, 466, 472Cowey, A., 230, 296, 306 Deaner, R. O., 103, 301, 526, 530, 534Crabtree, J. W., 178 Debreu, G., 167Craig, A. D., 181, 182, 191, 295 Decety, J., 166Craighero, L., 543 deCharms, R. C., 263Cramer, N. P., 466 DeFelipe, J., 423, 434Crammond, D. J., 464 Dehaene, S., 146, 156, 391, 392, 396Creutzfeldt, O. D., 423 Dekowska, M., 208Crick, F., 424, 428 Della-Maggiore, V., 418Crockford, C., 88, 89, 92 Deluty, M. Z., 146Crompton, R. H., 33 Demb, J. B., 513Cromwell, H. C., 320, 321, 551 Demes, A. B., 42Crow, T. J., 587 Demes, B., 33, 44, 46, 47, 50, 54, 55Croxson, P. L., 430 Dennett, D., 570Crutcher, M. D., 257, 320 Deputte, B., 304Cullen, K. E., 281, 283 Desimone, R., 207, 212, 213, 224, 225, 351, 474, 487Cunningham, C. L., 596 Desmond, A. J., 444Cunningham, E. P., 65, 66, 68, 71 Deubel, H., 227Cusick, C. G., 193 Devlin, J. T., 241Czelusniak, J., 18 Dewey, J., 573, 576Cziko, G., 580 DeYoe, E. A., 391 Di Bitetti, M. S., 65, 69D’Amato, F. R., 370 Di Fiore, A., 359D’Arceuil, H. E., 428 di Pellegrino, G., 413, 543D’Esposito, M., 414, 415, 418 Diamond, I. T., 424Dagosto, M., 40, 42 Diamond, R., 472Dally, J. M., 135 Dias, R., 412, 418Dalrymple-Alford, J. C., 150 Diaz-Perez, J. C., 64Dalton, K. M., 208 Dickinson, A. R., 317, 328, 329, 330, 331Damasio, A. R., 241 Dicks, D., 307, 373Damasio, H., 587 Diester, I., 388, 396, 397, 398Dan, Y., 206 Diltz, M. D., 514Darlington, R. B., 587 DiMaggio, P., 168Darwin, C., 3, 10, 39, 84, 85, 90, 117, 244, 625 Ding, L., 225Datla, K. P., 327 Dinstein, I., 543Daugman, J. G., 211 Disotell, T., 18David, S. V., 9, 111, 209, 211 Dittus, W. P. J., 73Davies, N. B., 102, 166 Divac, I., 318Davis, E. R., 146 Dixson, A. F., 202, 305, 366, 367, 375, 538Davis, H., 150, 152 Dobzhansky, T., 111Davis, M., 296, 302 Dong, D. W., 206

638 AUTHOR INDEXDonoghue, J. P., 194, 263, 264 Engell, A. D., 298Donohue, S. E., 418 Engh, A. E., 87, 92Dore, F. Y., 594 Enroth-Cugell, C., 206Dorris, M. C., 551 Erev, I., 553, 556Dostrovsky, J. O., 181 Erickson, C. A., 351, 474Doupe, A. J., 242 Erickson, C. J., 20Downing, P. E., 208, 540, 542 Erikson, G., 24Doya, K., 556 Eriksson, O., 64Draaisma, D., 572 Erwin, J. M., 443Dreher, J. C., 413 Eshel, I., 166Dreyfus, H., 572, 573 Ethier, C., 467Driver, J., 233, 234, 526, 533, 535 Etzioni, A., 168Drury, H. A., 201 Evans, M. E., 418Dubner, R., 225 Evans, T. A., 110, 146, 503Dufour, V., 71, 108, 110 Evarts, E. V., 257, 258Duhamel, J. -R., 389, 393, 467 Everitt, B. J., 302, 303Dum, R. P., 459 Everling, S., 393Dumais, S. T., 580 Ewert, J. P., 466Dumas, C., 594Dunbar, R. I. M., 43, 46, 65, 117, 133, 134, 359, 360, Faber, T. L., 432 526, 570, 588 Fabre-Thorpe, M., 243Duncan, J., 212, 224 Fagot, J., 76Durham, W. H., 165 Fahrenholz, F., 364 Fairbanks, L. A., 367, 368, 369, 370Eacott, M. J., 318 Fallah, M., 227, 229, 230Easterbrook, J. A., 300 Fanardjian, V. V., 297Ebbesen, E. B., 110 Fang, P. C., 189, 194Ebner, T. J., 257 Fanselow, M. S., 302Edds-Walton, P. L., 182 Farah, M. J., 413, 489Eddy, T. J., 122, 124, 126, 127, 129 Farley, C. T., 48, 50, 51Edwards, D. H., 539 Farris, H. E., 237Egeth, H. E., 212, 224, 526 Fecteau, J. H., 225Egnor, S. E. R., 89, 90 Fecteau, S., 516Eichenbaum, H., 337, 338, 349 Fehr, E., 166, 168, 169, 170Eichler, E. E., 439 Feigenson, L., 146, 152Eickhoff, S. B., 428, 433 Feldman, J., 237Eifuku, S., 489 Feldman, M. W., 164, 165, 166, 239Eimas, P. D., 242 Felleman, D. J., 201, 203, 204, 205, 207, 224, 300,Eisenberg, J. F., 76 342, 471, 478Eizirik, E., 13, 22 Fellows, L. K., 413Elftman, H., 32 Feltovich, N., 553, 556Ellsberg, D., 99 Ferrari, P. F., 543Elowson, A. M., 243 Ferrera, V. P., 213Elowson, M., 89 Ferrier, D., 454Elsley, J. K., 230 Fetterman, J. G., 146Elston, G. N., 423, 434 Fetz, E. E., 257, 264, 455Embry, S., 17 Fias, W., 156, 391, 393Emerson, R. W., 532 Field, D. J., 205, 206, 207, 208, 209, 210, 211Emery, N. J., 123, 125, 135, 307, 374, 532, 533, Finkenstadt, T., 466 538 Finlay, B. L., 587Emmerton, J., 145, 146 Fiorentino, I., 17Enard, W., 91, 438 Fischer, B., 226Engbert, R., 533 Fischer, J., 84, 85, 86, 89

AUTHOR INDEX 639Fischer, M. S., 33 Gabarini, F., 576Fish, E. W., 368 Gabriel, M., 417Fish, J. L., 65 Gabrieli, J. D., 337Fisher, C., 92, 93 ¨ Gachter, S., 166, 168, 169, 170Fisher, R. A., 166 Gaffan, D., 298, 318, 338, 349, 352, 410Fishman, Y. I., 245 Gaffan, E. A., 319Fitch, W. T., 91, 92, 243, 501, 504, 505 Galaburda, A. M., 193, 587Fitts, P. M., 259 Galambos, R., 237Fitzpatrick, D., 9 Galdikas, B., 593Flack, J. C., 307 Gale, G. D., 302Flavell, J. H., 118 Galfano, G., 533Fleagle, J. G., 32, 43 Gallagher, M., 297, 302, 303Fletcher, A. W., 602 Gallant, J. L., 211, 212, 550Flombaum, J. I., 92, 129 Gallese, V., 576Foerster, O., 465 Gallistel, C. R., 102, 146, 149, 153Fogassi, L., 462, 543 Galuske, R. A., 434Foldiak, P., 487 Gambaryan, P. P., 49, 50Ford, K. R., 32 Gandhi, S., 462Fortna, A., 439 Gankiewicz, E., 33Fouchart, J., 602 Ganz, L., 229Foundas, A. L., 194 Garber, P. A., 65, 66, 69, 71Fowler, C. A., 501 Gardenfors, P., 88Fox, M. D., 431 Gardner, R., 174Foxe, J. J., 510, 511 Garland, T., 434Fragaszy, D. M., 24, 134, 594, 595 Gasc, J. P., 33Francolini, C. M., 224 Gatesy, J., 13Frank, M., 149 Gatesy, S. M., 53Franz, T. M., 33, 48 Gauthier, I., 208, 241, 542Frazor, R. A., 210 Gearing, M., 440Frederick, S., 107, 563 Gebo, D. L., 18, 42, 64Freedman, D. J., 243, 247, 301, 386, 389 Gehlsen, G., 32Freeman, W. H., 491 Geisler, W. S., 209, 210Freese, J. L., 298 Gelade, G., 223Freiwald, W. A., 477, 479, 480, 487, 488 Gelman, R., 146, 149, 153Fries, W., 542 Gemba, H., 90Friesen, C. K., 526, 533 Genovesio, A., 412Fritsch, G., 256, 454, 455 Gentilucci, M., 265, 462, 603Fritz, J. B., 505 Gentner, T. Q., 249Frost, G. T., 588 Genty, E., 104, 128Frost, S. B., 183, 194 George, G., 10Fu, K. M. G., 193, 510 Georgopoulos, A. P., 257, 259, 260, 550Fu, Q. G., 257 Gergely, G., 118, 136Fuchs, A. F., 281, 457 Gershon, M., 573Fudenberg, D., 167, 553, 556 Geschwind, N., 501Fujii, N., 8 Geyer, S., 433Fujita, I., 249, 474, 477 Ghazanfar, A. A., 3–9, 87, 193, 237, 243, 248, 300,Fujita, K., 104, 128, 596 301, 500–518, 550Fukushima, K., 490 Ghiselin, M. T., 167Full, R. J., 49, 51, 53 Giambrone, S., 135Funahashi, S., 213, 320, 324, 398, 407 Giard, M. H., 234Furey, M. L., 475 Gibson, J. J., 209, 573, 576Fuster, J. M., 213, 308, 324, 388, 398, 407, 414, 514, Gibson, K. R., 587, 595, 598 581 Gifford, G. W., 246, 513, 514

640 AUTHOR INDEXGigerenzer, G., 98, 100, 166 Gregg, T., 295Gil da Costa, R., 87 Gregory, W. K., 13Gilad, Y., 527 Grice, H. P., 92Gilbertson, T., 264 Grice, P., 571Gilby, I. C., 105 Grieve, K. L., 541Gilissen, E. P., 427 Griffin, D., 570, 571Gil-White, F., 164 Griffin, T. M., 49, 50, 51, 53Gimpl, G., 364 Griffiths, T. D., 237, 244Gingerich, P., 14 Grigorescu, C., 209Gintis, H., 131, 160–172 Grill-Spector, K., 208Giszter, S. F., 466 Groh, J. M., 273–289Giurfa, M., 410 Grohrock, P., 518Glasser, M. F., 429 Gros-Louis, J., 89Glazko, G. V., 24 Grosof, D. H., 207Gleitman, L. R., 92, 93 Gross, C. G., 208, 249, 300, 472Glimcher, P. W., 163, 214, 225, 541, 543, 551 Grossberg, S., 581Gnadt, J. W., 213, 543 ¨ Grusser, O. J., 194Gneezy, U., 166, 168, 169 Gu, J., 438Gobbini, M. I., 208, 475 Guard, H. J., 363Goldberg, M. E., 226, 227, 301, 389, 581 Guenther, F. H., 242, 246Goldman-Rakic, P. S., 193, 205, 212, 213, 388, 513, Guigon, E., 487 514, 515, 516 Guilfoyle, D. N., 428Goldstein, J. M., 542 Guillery, R. W., 186Golla, H., 69 Guitton, D., 466Goodale, M. A., 203, 224, 265 Guldin, W. O., 194Goodall, J., 88, 590 Gundersen, H. J., 434Goodman, M., 21 Gunnell, G., 23Goossens, B. M. A., 123, 125 Gunther, M. M., 33, 42, 43, 50Gopnik, A., 118 Guo, K., 528Gordon, J. A., 406 Gurden, H., 333Gordon, P., 149Gothard, K. M., 6, 292–309, 375, 541 Hackett, T. A., 182, 191, 193, 244, 248, 433, 507,Gottlieb, G., 8 510, 513Gottschalk, A., 206 Hadj-Bouziane, F., 306Gould, H. J., 194, 195 Haenny, P. E., 213Gould, S. J., 8 Hafed, Z. M., 533Gould, S. J. V. E., 396 Haggard, P., 265Goursaud, A. S., 373, 374 Hahn, F., 167Gouzoules, S., 89 Haiss, F., 466Goy, R. W., 365 Haldane, J. B. S., 166Gradwell, P. B., 370 Hall, G., 318, 332Graf, W. M., 182 Hall, W. C., 424Grafen, A., 171 Hallinan, E. V., 108Grafton, S. T., 588 Halsband, U., 414Graham, E. A., 64 Hamann, S. B., 542Grandin, T., 539 Hamilton, W. D., 167Grant, K. A., 367 Hammerschmidt, K., 84, 89, 90Graves, F. C., 363 Hammerstein, P., 102, 166Gray, D., 46 Hammock, E. A. D., 366Graziano, M. S. A., 188, 194, 266, 268, 454–467 Hammond, G., 599Green, L., 108 Hampson, M., 431Greenfield, P. M., 587 Hamrick, M. W., 40Greenlee, M. W., 213 Han, K., 18

AUTHOR INDEX 641Hannon, B., 66, 68 Higley, J. D., 367–372Hare, B., 92, 101, 117–136, 538 Hihara, S., 616, 619, 620, 621Harlow, H. F., 319, 406, 581 Hikosaka, K., 509Harries, M. H., 511 Hikosaka, O., 225, 318, 320, 509Harrison, R. M., 590, 602 Hildebrand, M., 32, 46, 50Harting, J. K., 424 Hilgetag, C. C., 204Hartline, P. H., 277 Hinde, R. A., 3, 503Hartwich-Young, R., 274, 276, 296 Hinkley, L., 425Hartwig, W., 16, 24 Hirasaki, E., 32Haruno, M., 550 Hirata, S., 127Harvey, P. H., 64, 65 Hirsch, P., 168Hassani, O. K., 321 Hitzig, E., 256, 454, 455Hasselmo, M. E., 207, 306 Hladik, C. M., 65Hatsopoulos, N. G., 256–268 Ho, T. H., 557Hattori, Y., 606 Hocherman, S., 257, 464Haude, R. H., 538 Hoebel, B. G., 457Hauser, M. D., 9, 70, 73, 86, 89, 90, 91, 102, 103, Hof, P. R., 423, 434 121, 122, 126, 149, 151, 237, 239, 243, 248, 500, Hoff, B., 265 501, 503, 506, 510, 571, 596 Hoffman, G. E., 437Hawkins, J., 414 Hoffman, J. E., 227Haxby, J. V., 204, 207, 208, 475, 476, 481, 540, 542 Hoffman, K. L., 292–309, 541Hayakawa, T., 439, 440 Holden, C. J., 164Hayden, B. Y., 5, 105, 201–215, 528 Holekamp, K. E., 134, 135Hayhoe, M., 527, 529 Holland, J. H., 161Head, H., 616 Holland, P. C., 302, 303Hecht, E., 430 Hollerman, J. R., 320, 321, 328Hechter, M., 163 Holloway, R. L., Jr., 422Hedges, S. B., 13 Holman, S. D., 365Heeger, D. J., 213 Holmes, A., 536, 542Hegde, J., 204, 205, 207 Holmes, G., 616Heglund, N. C., 53 Holsapple, J. W., 263Heilbronner, S. R., 9, 105 Holt, L. L., 242Heiligenberg, W., 3 Holyoak, K. J., 148Heinrich, B., 135 Honig, W., 319Heinz, R. D., 500, 504 Hopfield, J. J., 581Henik, A., 393 Hopkins, W. D., 427, 587, 599, 600, 601, 603, 605,Hennig, W., 11 608Henrich, J., 164, 168 Horel, J. A., 300Henzi, S. P., 573, 576, 577 Hori, E., 536, 542Hepp-Reymond, M. C., 257, 260 Horner, V., 119Herbert, J., 160–172 Horowitz, B., 526, 528, 530Herbinger, I., 89 Horton, K. E., 127Herculano-Houzel, S., 184 Horton, N. J., 124Herman, L. M., 87, 93, 406 Horvath, J., 20, 21, 23Herrmann, E., 120, 121, 124, 127, 133 Horvath, S., 20, 21, 23Herrnstein, R. J., 106, 107 Hoshi, E., 408, 416, 514, 557Hertwig, R., 106 Hostetter, A. B., 126, 127, 605, 606Heschl, A., 121, 122, 123, 125, 126, 128, 130 Houle, A., 64, 70, 72Hess, J., 606 Houston, A. I., 102, 111, 112Hess, W. R., 457 Howell, A. B., 49, 50, 51Heyes, C. M., 118 Howell, L. L., 431Heywood, C. A., 300, 306 Howell, S., 368, 369, 370Hietanen, J. K., 536, 539 Hrdy, S. B., 538, 542

642 AUTHOR INDEXHu, X. T., 429 Ji, Q., 13Huang, C. S., 18, 194, 465 Jiang, J. T., 501Hubel, D. H., 206, 225, 490 Jin, Y., 492Huerta, M. F., 185, 193, 195 Jog, M. S., 321Humphrey, N. K., 117, 570 Johansen-Berg, H., 430Hung, C. P., 395, 486, 487, 491 Johnson, C., 577, 578Hunter, M. W., 406 Johnson, D. M., 147Huntley-Fenner, G., 238 Johnson, M. E., 439Huxley, J. S., 164 Johnson, M. H., 541, 542 Johnson, M. W., 108Ichinose, S., 621 Johnson, P. B., 257, 261, 464Ifrah, G., 396 Johnston, T. A., 443Ignashchenkova, A., 227 Jolly, A., 117, 529Ikeda, T., 225, 541 Jolly, C., 531Ingold, T., 587 Jones, E. G., 298Inoue, M., 121, 399 Jones, F. W., 39, 54, 202Insel, T. R., 362, 364, 368, 370, 376, 427, 596, 597 Jonides, J., 526, 533Iriki, A., 615–630 Jonker, L., 166Ishai, A., 475, 542 Jordan, K. E., 144–156, 504Ishibashi, H., 615, 616, 619, 620 Jordon, C., 590Ishida, H., 32, 33, 47 Joseph, J. P., 398, 561Isler, K., 65, 78 Jouffroy, F. K., 33Isoda, M., 399 Jouffroy, M. H., 42, 43Itakura, S., 121, 123, 124, 125, 533, 535 Judge, P. G., 144, 146Ito, M., 472 Jungers, W. L., 32, 33Ito, S. I., 513, 551, 557 ¨ Jurgens, U., 88Itti, L., 224, 529, 540Iwai, E., 472 Kaas, J. H., 177–196, 244, 248, 424, 433Iwamura, Y., 616 Kacelnik, A., 98, 99, 105, 106, 108, 112Iyengar, S., 191 Kadia, S. C., 245Izar, P., 594 Kaffka, K., 238Izquierdo, A. D., 303 Kagel, J. H., 550Izumi, A., 502, 503, 504 Kahneman, D., 99, 100, 101, 102, 104, 105, 106, 112, 528, 550Jablonka, E., 164 Kakei, S., 257Jacob, S. N., 400 Kalaska, J. F., 257, 260, 261, 464, 551Jacobs, G. H., 202, 203 Kalin, N. H., 302, 305, 306, 307, 362, 363Jacobsen, C. F., 318, 407 Kamil, A. C., 406Jakobson, L. S., 265 Kamin, L. J., 327, 332James, J. K., 164 Kaminski, J., 86, 126, 127, 130, 134, 135, 533James, W., 19, 164, 526, 573, 574, 576 Kamitani, Y., 476Janik, V. W., 87, 88 Kandel, E., 208Janmaat, K. R. L., 64, 66, 70, 71, 72, 73, 75, 76, 77 Kanwisher, N., 208, 476, 477Janson, C. H., 65, 66, 67, 68, 69, 70, 71, 72, 76 Kaplan, J. R., 367, 368, 369, 370Janssen, P., 204, 341 Kappeler, P., 17, 131Jay, M. F., 274, 276 Karaman, M. W., 438Jaynes, E. T., 163 Karin-D’Arcy, R., 131Jeannerod, M., 265, 266 Kastak, D., 406Jeffress, L. A., 280 Katz, C., 21Jenkins, F. A., Jr., 32, 40, 54 Kaufman, J. A., 427, 428, 429Jensen, J. N., 132 Kaufmann, L., 398Jensen, K., 132 Kawabe, K., 618Jerison, H. J., 64, 422 Kawagoe, R., 321, 551

AUTHOR INDEX 643Kawashima, R., 541 Koechlin, E., 414, 415, 416Kawato, M., 550, 551, 617 Kohler, R. E., 445Kay, R. F., 12, 17, 78, 201 Kohonen, T., 395, 460Kayaert, G., 489 Kohshima, S., 123, 126Kayser, C., 510 Kojima, S., 500, 502, 503Keating, C. F., 528 Kollock, P., 163Keay, K. A., 296 Kondo, H., 473Keil, F. C., 136 Kondo, S., 32Keller, A., 466 Konner, M., 8Keller, E. L., 225, 281 Konorski, J., 208Kellman, P. J., 240 Korstjens, A. H., 64Kelly, K. A., 282 Kosar, E., 181Kendrick, K. M., 375, 540 Kotrschal, K., 135Keramatian, K., 414 Kotter, R., 396Kermadi, I., 398 Kouh, M., 491Kerns, J. G., 413 Koyama, N. F., 132Kerr, J. N., 333 Koyama, Y., 375Kersten, D., 209 Kozorovitskiy, Y., 365Keverne, E. B., 304, 362, 363, 526, 528, 530, 531 Krachun, C., 127, 130, 135Khaitovich, P., 438 Kraemer, G. W., 362, 372Khera, A. V., 9, 530 Kralik, J. D., 104Kiani, R., 485 Kram, R., 53Kiefmann, M., 14 Kravitz, E. A., 539Killackey, H. P., 178 Krebs, J. R., 64, 102, 106, 108, 166Kimura, T., 32, 44, 47, 51, 55 Kreiman, G., 208King, M. B., 457 Kremers, J., 184King, M. C., 438 Kreps, D. M., 162, 165Kingstone, A., 526, 533 Krieger, K., 93Kinnaird, M. F., 74 Kriegs, J., 14Kirchner, H., 540 Krubitzer, L. A., 178, 424Kiritani, S., 500 Kubota, K., 213, 407Kirk, E. C., 201 Kubovy, M., 240Kirsh, D., 583 Kuhl, P. K., 242, 500, 501, 503Kitchen, D. M., 85, 153 Kulikowski, J., 242Klein, J. T., 214, 301, 532, 541 Kumar, S., 13Klein, M. E., 440 Kummer, H., 128Kliegl, R., 533 Kunzle, H., 178Klier, E. M., 276 Kuraoka, K., 306Klin, A., 539 Kuroshima, H., 119, 120, 128Kling, A., 373 Kurylo, D. D., 189Kluver, H., 373, 374 Kustov, A., 514Knecht, S., 587 Kusunoki, M., 225Knickmeyer, R. C., 542 Kuypers, H. G. J., 507Knight, F. H., 104 Kwan, H. C., 464Knight, R. T., 418 Kyes, R. C., 369, 528Knott, C. D., 593 Kylliainen, A., 539Knowlton, B. J., 318Knudsen, E. I., 233 LaBar, K. S., 300, 302Kobatake, E., 472, 481, 486, 487 Lacquaniti, F., 259Kobayashi, H., 123, 126 Laidlaw, D. H., 427, 429Kobayashi, K., 538 Lakatos, P., 510Kobayashi, S., 321 Lake, J. A., 164Koch, C., 224 Lakshminaryanan, V., 102

644 AUTHOR INDEXLaland, K. N., 596, 608, 625 Lizabeth, M. R., 500–518Lamb, M. J., 164 Ljungberg, T., 327, 328Land, M. F., 527, 529 Lloyd, S. A., 305Landauer, T. K., 147, 580 Lockwood, C. A., 65Langton, S. R. H., 533 Logan, C. A., 423, 444Larney, E., 47, 49 Logothetis, N. K., 9, 87, 208, 241, 345, 347, 431,Larson, S. G., 31, 32, 33, 43, 45, 46, 47, 48, 49, 54, 55 484, 503, 540Lashley, K. S., 337, 422 Lomolino, M. V., 164Lasky, R. E., 242 Lonsdorf, E. V., 601, 602Latimer, B., 44 Lorincz, A., 484Latto, R., 230 Lotto, A. J., 242Laubach, M., 395 Lovejoy, A. O., 444Laughlin, S. B., 205 Loveland, D. H., 106Le Gros Clark, W. E., 201, 202, 424 Lowe, D. G., 492Le Prell, C. G., 504 Lowe, M. J., 431Leavens, D. A., 603, 605, 606, 607 Lu, X., 263Lebedev, M. A., 213 Luce, R. D., 99LeDoux, J. E., 298, 302 Lumsden, C. J., 164, 165Ledyard, J. O., 170 Luppino, G., 195, 263, 459, 465, 467Lee, C. R., 52 Luschei, E. S., 281Lee, D., 550–566 Lutrell, L. M., 132, 133, 134Lee, P. C., 553, 554, 594 Lyman, R. L., 164Lehky, S. R., 204 Lyon, B. E., 151Leiner, A. L., 598 Lyon, D. C., 178, 185, 186Leiner, H. C., 598 Lyons, D. E., 119, 136Lemelin, P., 32, 33, 40, 45, 46, 47, 48, 49, 50, 54, 55Lennie, P., 204, 205 Ma, X., 429Leon, M. I., 541, 551 MacDonald, A. W., 413Leonard, C. M., 487 MacDonald, J., 500Leppanen, J. M., 536 Mace, R., 164Lesch, K. P., 371 Machado, C. J., 303Levin, D. T., 214 Machens, C. K., 213Levine, D. K., 553, 556 Mackintosh, N. J., 318, 332Lewis, D. A., 437 MacLean, E., 155Lewis, J. W., 189, 389, 467, 588, 599 Maclean, P. D., 298, 300Lewis, K. P., 152 MacLeod, C. E., 78, 427, 597, 598Lewkowicz, D. J., 503, 518 MacPhee, R. D. E., 12Lewontin, R. C., 162 Maddox, M., 441Li, Y., 33, 49, 55 Maeda, J., 618Li, Z., 224 Maestripieri, D., 6, 101, 307, 359–376Liberman, A. M., 242, 501 Mahajan, N., 595Liberman, U., 166 Mahon, L. E., 207Liddell, B. J., 296 Maier, A., 207Liebal, K., 126, 127, 605 Maier, J. X., 9, 207, 509, 512Lieberman, P., 92 Malamut, B. L., 318Likhtik, E., 299 Malkova, L., 296, 300, 303Lilak, A. L., 602 Maloiy, G. M., 33Lima, D., 181 Mandeville, B., 167Limongelli, L., 595, 602 Mangun, G. R., 9, 526Lindell, S. G., 370 Mann, V. A., 242Linden, A., 318 Mansouri, F. A., 412, 413Liu, J., 509 Mante, V., 210Livingstone, M. S., 7, 471–495 Manter, J., 32

AUTHOR INDEX 645Manuck, S. B., 369, 371 McCrossin, M., 26Manvelyan, L. R., 297 McGaugh, J. L., 300Maravita, A., 616, 617, 619, 620 McGraw, S. W., 70Marchant, L. F., 587, 594, 601, 602 McGreevy, B. P., 566Marcus, D. S., 207 McGrew, W. C., 587, 590, 594, 601, 602Margaret, S. L., 471–495 McGuire, M. T., 369, 370Margoliash, D., 248, 249 McGurk, H., 500, 518Marino, L., 427 McIntosh, A. R., 418Maris, M., 574, 575 McKiernan, B. J., 265Markson, L., 92 McKone, E., 208Marler, P., 90, 503, 510 McMahon, T. A., 49Marr, D., 111 McNelis, N. L., 526, 528, 530Marra, S. L., 526, 528, 530, 531 McPeek, R. M., 225Marschark, M., 241 Meador, K. J., 588Marsh, B., 106 Mech, L. D., 202, 203Martel, F. L., 363 Meck, W. H., 145, 146, 152, 153, 154, 387, 391, 392Marteniuk, R. G., 265 Medawar, P. B., 495Martensz, N. D., 362, 363 Mednick, S. A., 165Martin, A., 241 Mehlman, P. T., 367, 369, 370Martin, J. H., 455 Mehta, A. D., 212Martin, R. D., 33, 45, 46, 50, 64, 78, 190, 424 Mel, B. W., 491Martinez Trujillo, J. C., 212 Melchner, M. J., 223Marzke, M., 265 Meldrum, D. J., 33, 46Maskin, E., 167 Melis, A. P., 127, 129, 130, 131, 132, 134Mason, C. R., 261 Meller, R. E., 362, 363Mason, W. A., 304, 374 Melnick, D., 18Massaro, D. W., 518 Meltzoff, A. N., 118, 136, 166, 501, 503Massopust, L., 22 Menzel, C. R., 76Masterton, R. B., 183 Menzel, E. W., Jr., 67Matelli, M., 195, 459, 462, 563 Meredith, M. A., 276, 277Mathews, A., 536, 542 Merigan, W. H., 204Matsumoto, K., 321 Merleau-Ponty, M., 582Matsumoto, M., 557 Merten, K., 385, 390, 392, 393Matsuzaka, Y., 459, 563 Mesoudi, A., 164, 172Matsuzawa, T., 119, 127, 396, 624 Messier, J., 464Mattiello, J. A., 256–268 Messinger, A., 349, 350, 351, 355Mattingly, I., 501 Mesulam, M. M., 506Maunsell, J. H., 204, 211, 213, 229, 245, 471 Metzger, R. R., 275, 278, 279May, M. L., 76 Meunier, M., 474Maynard Smith, J., 102, 166, 167 Michael, L. P., 3–9, 525–544Mayr, E., 133 Michael, S. A. G., 454–467Mazer, J. A., 211, 550 Michaels, C. F., 576Mazur, J. E., 106, 107 Micheyl, C., 244, 245Mazzei, K. C., 72 Miczek, K. A., 363, 368McAdams, C. J., 212 Middleton, F. A., 263McAndrews, M. P., 399 Mikami, A., 213, 399McCarthy, G., 476 Miklosi, A., 134McClearn, D., 46 Miller, B. L., 165McCollum, G., 259 Miller, C. T., 237–249, 500McComb, K., 87, 88, 153, 393 Miller, E. K., 144, 156, 213, 321, 356, 385, 389, 390,McCormack, K., 371 393, 395, 396, 398, 416, 417McCoy, A. N., 9, 105, 551 Miller, G. A., 572McCrea, R., 281 Miller, J. D., 500

646 AUTHOR INDEXMiller, J. H., 167 Muakkassa, K. F., 263Miller, L. E., 257 Mufson, E. J., 434Miller, M. H., 165 Muller, H. J., 533Miller, W., 23 Muller, J. R., 229Milner, A. D., 203, 224 Muller, M. N., 131Milner, B., 337, 349, 399 Mullette-Gillman, O. A., 276, 277Milton, K., 65, 66, 68, 70, 76, 78 Munoz, D. P., 225Mink, J. W., 65 Murphey, D. K., 229Mirenowicz, J., 325, 327, 328 Murphy, W. J., 13Mischel, W., 110 Murray, E. A., 190, 204, 302, 303, 338, 349, 352,Mishkin, M., 186, 190, 203, 204, 224, 244, 245, 337, 355, 356, 410 339, 406, 540, 616 Murray, M. M., 244Mitani, J. C., 89, 131, 132, 606 Murthy, V. N., 264Mitchell, R. W., 123, 125, 535, 606 Musallam, S., 214Miyachi, S., 263 Mushiake, H., 400Miyashita, Y., 339, 340, 341, 342, 344, 346, 347, 348, Muybridge, E., 46, 49, 50 349, 354, 398, 474 Myowa-Yamakoshi, M., 119Miyawaki, K., 242Mizumori, S. J. Y., 321 ¨¨ ¨ Naatanen, R., 246Moeller, S., 477, 479, 480 Nadel, L. 66Moll, H., 118 Nagell, K., 136Moll, J., 165 Nahon, J. L., 439Mollon, J. D., 64, 202 Nakahara, H., 327, 412Monsell, S., 412 Nakamichi, M., 590Montague, P. R., 163, 333 Nakamura, K., 306, 308Moody, S. L., 213 Namba, H., 628Mookherjee, D., 553, 556 Napier, J., 43, 76Mooney, R., 9 Nash, J. F., 552Moore, B., Jr., 167 Nasir, S. M., 510Moore, D. R., 274 Navalpakkam, V., 224Moore, M., 501 Naya, Y., 337–356Moore, T., 223–234, 457, 458 Nei, M., 24Morales, J., 18 Neisser, U., 206Moran, D. W., 257 Neiworth, J. J., 121, 123, 125, 126Moran, J., 212 Nelken, I., 244, 245, 246Morange, F., 602 Nelson, E., 363Moravec, H., 629 Nelson, R. J., 258Morel, A., 186 Newman, M., 164Morgenstern, O., 99, 166, 550 Newman, T. K., 307Mori, S., 428 Newsome, W. T., 163, 213, 509Morris, G., 327 Ni, X., 23Morris, J. S., 296, 541 Nichols, S., 119Morris, R. D., 602 Nicolelis, M. A. L., 9Morrison, J. F., 72 Nieder, A., 144, 155, 156, 243, 385–401, 514Morrow, M. M., 257 Nieuwenhuys, R., 274Morton, E. S., 90 Niki, H., 407Morton, T. E., 90 Nimchinsky, E. A., 434Moschovakis, A. K., 283 Ninokura, Y., 399, 400Moscovitch, M., 476 Nishida, T., 590Mountcastle, V. B., 226, 276, 478 Nishijo, H., 308Moura, dA. A. C., 594 Nishizaki, M., 475Moyer, R. S., 147, 148 Nissen, H. W., 318, 406 ¨Muller, A. E., 65 Noble, A. C., 72

AUTHOR INDEX 647Noda, H., 281 Owren, M. J., 85, 88, 90, 91, 501, 505, 571, 579, 580 ¨Noe, R., 78 Oxnard, C. E., 34, 38, 42, 54Noesselt, T., 512Nooteboom, S. G., 504 Paciulli, L. M. 69Norgren, R., 181 Packard, M. G., 300, 318Norman, K. A., 476 Padberg, J., 8, 425Noser, R., 67, 68, 69, 72 Padden, D. M., 242Notoya, T., 621 Padoa-Schioppa, C., 103, 324, 551Nottebohm, F., 3, 248 Pagel, M., 164Noudoost, B., 223–234 Pages, S., 18Nudo, R. J., 183, 194, 455 Pagnoni, G., 432Numan, M., 370 Pakkenberg, B., 434Nystrom, P., 590, 602 Palmer, A. R., 587 Palombit, R. A., 510O’Brien, M. J., 164 Pan, W. -X., 327O’Brien, T. G., 74 Pandya, D. N., 300, 407, 478, 506, 507, 511O’Connor, D. H., 212 Paninski, L., 257, 259, 261O’Keefe, J., 66 Panksepp, J., 304, 361, 363O’Leary, J. G., 264 Papez, J. W., 300O’Malley, R. C., 602 Pare, D., 299O’Regan, J. K., 214 Park, M. C., 464O’Scalaidhe, S. P., 507, 513, 514, 515 Parker, A. J., 163, 298Obayashi, S., 617, 618, 619 Parker, G. A., 102Obeid, I., 8, 518 Parker, K. J., 366Oden, D. L., 406 Parker, S. T., 587, 595Odling-Smee, F. J., 165, 625 Parr, L. A., 432, 503, 518Ohno, S., 439 Parsons, L. M., 164Ojima, H., 621 Partan, S. R., 304, 501Okamoto, S., 121 Pashler, H. E., 212Okamoto-Barth, S., 121, 122, 123, 124, 533 Passingham, R. E., 405, 414Okauchi, T., 618 Pasternak, T., 213Oldham, M. C., 438 Pasupathy, A., 321, 356, 417, 481, 482, 484Oliveira, M. G., 594 Paton, J. J., 302, 303, 308, 551Oliver, D. L., 274 Patrick, M., 32Olshausen, B. A., 205, 206, 207, 208, 209, 210, 211, Patterson, M. L., 500 491 Patterson, R. D., 504Olson, C. R., 551 Paulignan, Y., 265Olson, C. X., 242 Pawitan, Y., 555Olson, M., 167 Payne, J. W., 100Olszewski, J., 263 ˜ Paz-y-Mino, G., 135Olupot, W., 74 Pearce, J. M., 318, 332Onishi, K. H., 118, 130 Peck, C. K., 277, 278Ono, T., 308 Pelphrey, K. A., 539Oram, M. W., 487, 511, 517 Peltier, S. J., 431Orbell, J. M., 170 Penfield, W., 339, 454, 456, 459, 465Oribe, E., 297 Peng, X., 204Orlov, T., 243 Penn, D. C., 570, 571Osorio, D., 202 Penny, G. R., 181Ostrom, E., 168, 169, 170 Pepper, J. W., 135Ostry, D. J., 510 Pepperberg, I. M., 93, 406Ottoni, E. B., 594 Perachio, A. A., 305Owen, A. M., 416 Perin, C. T., 562Owings, D. H., 90 Perner, J., 118

648 AUTHOR INDEXPeronnet, F., 234 Poss, S. R., 126, 127, 605, 607Perret, E., 408 Postle, B. R., 213Perrett, D. I., 207, 208, 249, 300, 306, 477, 487, 511, Potts, R. 64 517, 533, 537 Poundstone, W., 166Pert, C. B., 573 Povinelli, D. J., 118, 121, 122, 123, 124, 126, 127,Perusse, R., 150 128, 129, 130, 131, 135, 136, 596Pesenti, M., 150 Povinelli, R. J., 131Pessoa, L., 296, 301 Prather, J. F., 543Peterhans, E., 207 Prather, M. D., 374Peters, R. J., 529, 540 Pratt, W. E., 321Petersen, M. R., 500 Premack, D. W., 118, 122, 625Peterson, B. S., 105 Preuschoft, H., 50Petit, O., 133 Preuss, T. M., 8, 177, 194, 195, 422–445, 459, 516,Petkov, C. I., 245, 246, 249 587Petkov, N., 245, 246, 249 Previc, F., 471Petrides, M., 213, 399, 414, 416, 418, 507 Price, J. L., 473Petter, A., 33 Pridmore, P. A., 46Pettigrew, J. D., 202 Pritchard, T. C., 181Pfeifer, R., 572, 573, 574 Procyk, E., 398Phelps, E. A., 302, 303 Pruessner, J. C., 300Phillips, K. A., 599, 602 Pryce, C. R., 370Phillips, M. L., 298 Puce, A., 476Piazza, M., 393, 396 Purves, D., 9, 201, 203, 208, 212Pica, P., 149 Purvis, A., 436Picard, N. 195 Putman, P., 536, 542Pierce, J. 89 Pylyshyn, Z. W., 237, 239Pietrini, P., 475Pigarev, I. N., 514 Qi, H. -X., 190, 192Pika, S., 126, 127, 606 Quintana, J., 388Pilbeam, D., 25 Quiroga, R. Q., 208, 209, 489Pinel, P., 393Pinker, S., 92, 93, 505, 506 Rabbitt, P. M., 413, 533Pinsk, M. A., 472 Rachlin, H., 105Platt, J. R., 146, 147 Rader, K. A., 445Platt, M. L., 3–9, 105, 123, 125, 214, 225, 392, 525– Raghanti, M. A., 434 544, 551 Raichlen, D., 33, 46Ploog, D., 88 Raiffa, H. 99Plotkin, H. C., 625 Rainer, G., 213, 398Poeppel, D., 246 Rakic, P. S., 213Poggio, T., 491 Raleigh, M. J., 369, 370, 373, 539Poldrack, R. A., 513 Ramanathan, D., 466Polk, J. D. 33, 47 Ramnani, N., 416, 428, 430, 598Pollard, K. S., 440 Ramsden, B. M., 207Polyak, S. L. 64 Ramseyer, A., 71, 108Pomerantz, S. M., 366, 370 Ranzato, M., 492Poole, J., 88 Rao, R. P., 205Popesco, M. C., 439 Rao, S. C., 407Popper, K., 164 Rapcsak, S. Z., 308Populin, L. C., 276, 277, 278, 279 Rapoport, A., 553Poremba, A., 244, 246, 249 Rasmussen, D., 18, 23Porter, C., 18 Rauschecker, J. P., 191, 193, 244, 245, 246, 507Porter, K. K., 275, 283 Ravel, S., 321Posner, M. I., 229, 526, 533, 539 Ray, D., 18

AUTHOR INDEX 649Raymond, J. E., 532 Rogers, L. J., 587, 588, 603Read, D., 107 Rohrer, T., 573Reader, S. M., 596, 608 Roitman, J. D., 213, 243, 391, 392Real, L. A., 163 Roitman, M. F., 327Reaux, J., 127, 129 Rollinson, J., 33, 45, 46, 50Recanzone, G. H., 242 Rolls, E. T., 193, 207, 300, 303, 308, 324, 407,Redgrave, P., 296 491Redlich, A. N., 206 Romanski, L. M., 193, 247, 248, 298, 301, 407, 500–Reed, E., 576 518, 542Reed, K. E., 64 Romo, R., 213, 392, 407, 514Rehkamper, G., 406 Roos, C., 17, 21Reinagel, P., 210 Rosa, M. G., 540Rempel-Clower, N. L., 299 Rosati, A. G., 101, 108, 109, 110Remple, M. S., 181, 183, 186, 187 Rosch, E., 241Rendall, D., 7, 85, 90, 91, 243, 359, 501, 504, 505, Rose, K., 23 570–583, 606 Rose, M. D., 8, 38, 40, 42Renner, J. C. 146 Rosen, I., 454Rensink, R. A., 214 Rosen, R. F., 440Repacholi, B. M., 118 Rosenblum, L. A., 365Rescorla, R. A., 317, 318, 330, 332 Rosenblum, L. D., 500Reser, D., 514 Rosenkilde, C. E., 514Restle, F., 406 Ross, C. F., 78, 190, 202, 424Reynolds, J. H., 212 Rossie, J., 23Reynolds, J. N. J., 318 Rosvold, H. E., 307, 373Reynolds, T. R. 47, 55 Roth, A. E., 550, 553, 556 ´Riba-Hernandez, P., 64 Rouiller, E. M., 183Richards, J. B., 107 Rowell, T. E., 503Richards, R. J., 444 Roy, A. C., 265, 266Richmond, B. G., 44 Rubino, D., 264Richmond, B. J., 321, 551 Ruderman, D. L., 210Rickert, J., 509 Rumbaugh, D. M., 87, 396, 581, 607Riedel, J., 134 Rushworth, M. F. S., 407, 413, 563, 566Rieke, F., 205 Russ, B. E., 246, 247, 248Riesenhuber, M., 491 Russell, J. L., 600, 603, 605Rieskamp, J., 99, 112 Rylands, A. B., 109Rilling, J. K., 361, 376, 423, 427, 429, 430, 431, 432, 587, 596, 597, 598 Sabbagh, M. A., 542Ristic, J., 539 Sackett, G. P., 530Rivera, M. C., 164 Sakai, K., 339, 341, 349, 354, 398, 414Rivera, S. M., 398 Sakai, S. T., 263Rizzolatti, G., 166, 263, 459, 462, 543, 588 Sakata, H., 454Roberts, W. A., 145, 146 Sakura, O., 615–629Robinson, B. W., 305 Saleem, K. S., 339, 472, 473, 474, 479, 506Robinson, D. A., 276, 457, 466 Saleh, M., 256–268Robinson, D. L., 225 Salzman, C. D., 302, 457Robson, J. G., 206 Samejima, K., 550, 551Rockel, A. J., 423 Samuelson, P. A., 107Rockland, K. S., 474 Sanes, J. N., 263, 264Rockwell, W. T., 572 Sanghera, M. K., 308Roe, A. W., 186 Santos, L. R., 92, 101, 117–136, 151, 237, 239, 508,Roeder, J. J. 128 595, 596Roesch, M. R., 551 Santos, S., 129Rogers, J., 367, 371 Sapolsky, R. M., 300

650 AUTHOR INDEXSato, K., 170 Selezneva, E., 246Satoh, T., 327 Seligman, R. A., 427, 587Sauther, M., 529 Selten, R., 166Savage-Rumbaugh, E. S., 87 Seltzer, B., 300, 478, 506, 507, 542Sawamura, H., 387, 393 Semendeferi, K., 407, 416, 427, 587Scerif, G., 126, 128 Semple, S., 87, 88Schafer, R. J., 223–234 Seo, H., 551, 555, 557, 559, 560, 566Schall, J. D., 163, 195, 213, 214, 225 Sereno, A. B., 204, 245Scheier, C., 573, 574 Sergent, J., 249, 476Schein, S. J., 225 Sergio, L. E., 261Schenker, N. M., 427 Seron, X., 150Scherberger, H., 214 Setlow, B., 321Scherer, K. R., 91 Sewards, M. A., 541, 542Schiller, P. H., 204, 276, 457, 466 Sewards, T. V., 542Schino, G., 363, 372 Seyfarth, R. M., 84–93, 112, 118, 132, 207, 243, 304,Schlack, A., 467 505, 506, 510, 527, 531, 532, 571, 572Schleicher, A., 598 Shadlen, M. N., 213, 541, 551Schmidt, K. M., 170 Shafer, D. D., 602Schmidt, M., 33 Shafir, S., 101Schmitt, D., 21, 31–55 Shaki, S., 148Schmitz, C., 434 Shallice, T., 418Schmitz, J., 17, 22 Shannon, C. E., 205, 372Schmolesky, M. T., 204, 207 Shannon, R. V., 500Schneider, W. X., 227 Shapiro, L. J., 32, 33, 46Schneidman, E., 247 Shedlock, A. M., 13Schneirla, T. C., 8 Sheese, B. E., 367Schoenbaum, G., 321, 324, 325 Sheinberg, D. L., 208, 345, 346Schoenemann, P. T., 427 Sheliga, B. M., 227Scholes, M., 163 Shen, L., 257Scholl, B. J., 237, 239 Shennan, S., 164, 626Schouten, J. L., 475 Shenoy, K. V., 268 ¨Schrodinger, E., 162 Shepherd, G. M., 123, 125Schroeder, C. E., 500, 510, 511 Shepherd, S. V., 123, 125, 307, 525–544Schuck-Paim, C., 112 Sherwood, C. S., 599, 608Schulkin, J., 165 Sheth, B. R., 207Schultz, A., 26 Shidara, M., 321, 551Schultz, R. T., 208, 542 Shik, M. L., 465Schultz, S., 134 Shima, K., 398, 563Schultz, W., 316–333 Shimojo, E., 532Schusterman, R. J., 87, 93, 406 Shinoda, Y., 265Schwandt, M. L., 365 Shively, C. A., 367, 369Schwartz, A. B., 21, 257 Shoham, S., 261Schwartz, J. -L., 510 Shoshani, J., 21Schwarz, C., 466 Shulman, G. L., 526Schweighofer, N., 556 Shultz, S., 65Scott, S. H., 260, 261 Siegel, A., 295Scott, S. K., 244 Sigg, J., 68, 69, 72Scott-Kelso, J. A., 581 Sikela, J. M., 439Sears, C. R., 239 Silberberg, A., 103, 104Seeley, W. W., 431 Silk, J. B., 85, 577Segerstrale, U., 167 Simmons, J. A., 3Seiffert, E., 19, 23 Simon, H. A., 100, 166Sejnowski, T. J., 211 Simoncelli, E. P., 205, 209, 210

AUTHOR INDEX 651Simons, D. J., 214 Squire, L. R., 337Simons, E. L., 23 Stanford, T. R., 286Simonson, I., 101 Stark, E., 257, 263, 266Simpson, G. G., 11, 13 Stefanacci, L., 298Singer, T., 298 Stein, B. E., 276, 277, 508Singleton, I., 593 Steiner, S. M., 602Sinnamon, H. M., 297 Steinmetz, M. A., 213, 225Sinnott, J. M., 500, 504 Steinschneider, M., 246Sivic, J., 491 Steiper, M. E., 85, 134Skarda, C. A., 581, 582, 583 Steklis, H. D., 373Skavenski, A. A., 189 Stephan, H., 20Skibo, J. M., 164 Stephen, V. S., 525–544Skuse, D., 371, 539 Stepniewska, I., 188, 189, 194, 433, 467Slater, P. J. B., 87 Sterck, E. H. M., 70Slocombe, K., 89 Stern, J. T., Jr., 32, 33, 34, 44, 49, 54, 55Smiley, J. F., 510 Stern, M., 76Smith, A. P. R., 542 Stevens, J. R., 98–112, 151Smith, B. R., 144, 146 Stevenson, M. F., 109Smith, D. R. R., 504 Stoerig, P., 296Smith, E. A., 164 Stoet, G., 389, 413Smith, E. E., 241 Stolba, A., 68, 69, 72Smith, E. O., 376 Storm, R. W., 239Smith, F. H., 16 Streeter, L. A., 242Smith, G. E., 39 Strick, P. L., 195, 263, 459Smith, H. J., 88 Strum, S. C., 578, 581Smith, T. D., 184 Stryker, M., 276, 457, 466Smith, Y., 299 Stubbs, D. A., 146Smuts, B. B., 65, 359 Suarez, S. A., 69Sneath, P. H. A., 11 Subramaniam, B., 227Snowden, R. J., 285 Suga, N., 3Snowdon, C. T., 89, 243, 361 Sugihara, T., 301, 508, 515, 516, 550Snyder, A. Z., 413 Sugiyama, Y., 590Snyder, L. H., 189, 301, 389, 413 Sugrue, L. P., 163, 225, 541, 551Snyder, P. J., 599 Suhara, T., 618Sohn, J. -W., 551, 562, 563, 566 Summerfield, Q., 504Sokal, R. R., 11 Sumner, P., 64, 202Soligo, C., 64, 65 Sun, S. W., 428Soltani, A., 556 Suomi, S. J., 307, 581, 602Sommers, M. S., 504 Super, H., 213Somogyi, P., 478 Suri, R., 333Sopher, B., 553, 556 Surian, L., 118Southgate, V., 118 Surridge, A. K., 202Sparks, D. L., 274, 276, 277, 279, 285, 286, 287, Susman, R. L., 32, 44, 54, 55 288 Sussman, E. S., 244Spayd, S. E., 64 Sussman, R. W., 32, 40, 64Spelke, E. S., 102, 237, 238, 239 Sutter, M. L., 245Spence, C., 234 Sutton, R. S., 163, 551, 555, 559Sperling, G., 223 Suwyn, C., 441Spiering, B. J., 247 Suzuki, D. A., 281Spies, H. G., 373 Suzuki, W. A., 339, 473Sporns, O., 396 Swartz, S. M., 32Sprague, J. M., 206 Swettenham, J., 539Sprengelmeyer, R., 308 Swisher, J. D., 190

652 AUTHOR INDEXSylvestre, P. A., 281, 283 Tomita, H., 398Szalay, F. S., 21, 40 Tomonaga, M., 146, 533Szalda-Petree, A. D., 108 Tong, F., 476 Tootell, R. B. H., 433, 479, 540Taglialatela, J. P., 599, 600, 603, 608 Torigoe, T., 596Taira, M., 257 Torralba, A., 492Takahashi, K., 264 Tosi, A., 18Takahashi, T., 110 Toth, L. J., 245Tanaka, J. W., 489 Toth, N., 602Tanaka, K., 300, 339, 341, 472, 473, 474, 481, 487, Trefilov, A., 371 489 Treisman, A. M., 223, 224Tanifuji, M., 475 Tremblay, L., 321, 322, 323, 324, 325, 326, 551Tanila, H., 513 Tremoulet, P., 239Tanji, J., 195, 257, 398, 399, 400, 416, 561, 563 Treue, S., 212Tardieu, C., 32 Treves, A., 248Tardif, E., 434 Trippenbach, T., 295 ´Tavare, S., 64 Tristem, M., 13Taylor, C. R., 64 Trivers, R. L., 167Taylor, C. S. R., 457, 458, 459 Troisi, A., 307, 363, 370Taylor, P., 166 Trout, J. D., 500Teather, L. A., 300 Tsao, D. Y., 207, 208, 300, 306, 471–494, 542Tenenbaum, J. B., 491, 493 Tsao, F. T., 7, 540Terborgh, J. W., 76 Tschudin, A., 533Terrace, H. S., 87, 88, 144, 146 Tsunoda, K., 474, 475Terzuolo, C., 259 Tu, M. S., 49, 51, 52Thach, W. T., 257 Tudusciuc, O., 388, 393, 394, 395Thaler, R., 170 Turesson, H. K., 9, 510Theall, L. A., 127 Turnquist, J. E., 32, 42Thewissen, J., 14 Tuttle, R., 32Thier, P., 189, 467 Tversky, A., 99, 100, 101, 104, 105, 106, 112, 528,Thierry, B., 65, 72, 307, 365 550Thomas, E., 301 Tweedale, R., 540Thomas, O. M., 207 Tzelgov, J., 393Thompson, K. G., 163, 214, 227Thompson, P., 285 Uddin, M., 438Thompson, R. K. R., 319 Ulanovsky, N., 244Thorndike, E. L., 554 Uller, C., 103, 119Thorpe, S. J., 206, 243, 507, 540 Ullman, S., 224Tian, B., 191, 244, 245, 246, 514 Underhill, A. M., 279, 282Tiitinen, H., 246 Ungerleider, L. G., 203, 204, 224, 244, 245, 301, 472,Tinbergen, N., 3, 66, 76 540, 616Tinklepaugh., O. L., 101 Uvnas-Moberg, K., 366Tobin, H., 108 Uyeda, A. A., 308Tobler, P. N., 327, 329, 331Todd, M. P., 422–445 Vaadia, E., 513Todorov, E., 257 Valero, A., 66, 68Toga, A. W., 428, 433 Van de Kragt, J. C., 170Tolhurst, D. J., 210 van der Geest, J. N., 539Tolias, A. S., 204 Van Essen, D. C., 189, 201, 203, 204, 206, 207, 224,Tolman, E. C., 66 300, 342, 389, 391, 467, 471, 478Toloczko, D. M., 364 Van Gelder, T., 573, 582Tomasello, M., 92, 118, 119, 121, 135, 533, 560 Van Gisbergen, J. A., 283Tomassini, V., 430 van Hateren, J. H., 211

AUTHOR INDEX 653van Hoof, J. A., 531, 538 Walter, H., 3, 76, 581Van Hooser, S., 186 Walters, N. B., 428Van Marle, K., 393 Walton, M. E., 182Van Opstal, A. J., 283 Wandell, B. A., 201, 203, 206Van Roosmalen, M. G. M., 65 Wang, G., 478Van Schaik, C. P., 65, 75, 78, 131, 588, 593, 596, Wang, Q., 178 597 Wang, W., 281Van Valkenburg, D., 240 Wang, X., 281van Wassenhove, V., 510 Wang, Y., 478Vangor, A. K., 32 Ward, C. V., 44Vanni, S., 204 Ward, J. E., 454Varki, A., 440 Ward, R., 296Varki, N. M., 439 Warneken, F., 119, 132, 136Vasconcelos, M., 112 Warren, J. D., 237, 244Verguts, T., 156, 391 Waser, P. M., 66Vick, S. J., 121, 122, 123, 125, 126, 128 Washburn, D. A., 396, 581, 607Victor, J. D., 207 Washburn, S. L., 88Vigilant, L., 292 Wassink, T. H., 539Vilensky, J. A., 31, 32, 33, 45, 46, 47, 50, 51 Watanabe, M., 321, 513, 551Vincent, J. L., 431 Watanabe, T., 399Vinje, W. E., 550 Watanabe, Y., 300Viranyi, Z., 134 Watson, K., 9Visalberghi, E., 134, 594, 595 Watson, R. E., Jr., 437Viviani, P., 259 Watts, D. P., 131Vlamings, P., 104 Watts, H. E., 135Vogels, R., 243, 483, 484 Weaver, W., 205Vogt, B. A., 298 Webster, M. J., 473, 507Vogt, C., 433 Weghorst, J. A., 602Vogt, O., 433 Weisrock, D., 17Vokey, J. R., 583 Weiss, D. J., 243Von Bonin, G., 339, 455, 478 Welch, K., 465von Cramon, D. Y., 413 Weller, R. E., 184von der Heydt, R., 207 Wellman, H. M., 118von Euler, C., 295 Wellman, L. L., 303Von Neumann, J., 99, 162, 550 Wells, J. P., 32, 33, 51 ¨Von Uexkull, J., 575 Wendland, J. R., 371Vonk, J. 130 Werker, J. F., 500, 503Vorobyev, M., 202 Werner-Reiss, U., 276, 280, 281, 282Vroomen, J., 234 West, M. J., 434Vuilleumier, P., 296, 540, 541, 542 Wester, P., 368Vyas, D. K., 154 Westergaard, G. C., 369, 602 Whalen, J., 147, 387Waddell, J., 296 Whalen, P. J., 297, 302, 306, 308Waelti, P., 327, 328, 329, 330, 331, 333 Wheeler, D. L., 445Wagner, A. D., 317, 318, 330, 332–333, 413 Wheeler, M., 573, 576Wagner, A. R., 317, 318, 330, 332 Wheeler, P., 65Waite, T. A., 101 Wheller, P., 205Walker, A., 38, 43 Whishaw, I. Q., 190Wallace, M. T., 46, 49, 277, 296 White, I. M., 408Wallen, K., 375 White, T. 45Wallis, G., 491 Whitehead, P. F., 33Wallis, J. D., 389, 405–418, 491 Whiten, A., 119, 551, 577, 588, 606Walsh, V., 242, 393, 395 Wickens, J. R., 333

654 AUTHOR INDEXWiese, H., 385, 396 Yaguramaki, N. 55Wiesel, T. N., 206, 225, 490, 494 Yakovlev, V., 347, 348Wildman, D. E., 17 Yamagishi, T., 169, 170Wilke, M., 509 Yamane, S., 300William, D. H., 587–608 Yamane, Y., 475Williams, G. C., 167 Yamazaki, N., 7, 32Williams, J. H. G., 166 Yamazaki, Y., 7, 32, 615–630Williams, N. A., 587 Yang, T., 213Williams, Z. M., 353, 356 Yantis, S., 212, 237, 526Wilson, D. A., 183 Yarbus, A. L., 211, 527, 529Wilson, E. O., 164, 165 Ybarra, M. S., 91, 501Wilson, F. A., 300, 308 Yehia, H. C., 501Wilson, M. L., 151, 393 Yeshurun, Y., 229Wimmer, H., 118 Yeterian, E. H., 407Wing, A. M., 265 Yoder, A., 20, 21Winkowski, D. E., 233 Yodyingyuad, U., 369Winslow, J. T., 364, 365, 368, 370 Yoshor, D., 212Winston, J. S., 298 Young, A. M. J., 327Winter, S. G., 166 Young, A. W., 489Winterhalder, B., 164 Young, H. P., 166Wise, R. J., 244 Young, L. J., 360, 364, 366, 376Wise, S. P., 257, 318, 338, 349, 352, 356, 408, 425, Young, M. P., 294 464 Young, N. M., 134Wiskott, L., 490 Yousry, T. A., 599 ¨Witthoft, W., 410 Yovel, G., 208Wittig, R. M., 87 Yukie, M., 472Wolf, G., 181Wolfe, J. M., 224 Zador, A. M., 210, 263Wolin, L., 22 Zajonc, R. B., 164, 531, 532Wood, E. J., 166, 167 Zald, D. H., 308Wood, G. A. 33, 51 Zangehenpour, S., 504, 518Wood, J. N., 121, 122 Zatorre, R. J., 244Woodruff, G., 393, 625 Zeki, S. M., 225, 295Woodward, A., 118 Zelazo, P. D., 414Woolley, C. S., 300 Zella, J. C., 277Woolley, S. M. N., 249, 550 Zhang, J., 428Woolsey, C. N., 464, 465 Zhang, R., 440Wordon, F. G., 237 Zhang, S., 427Worman, C., 76 Zhao, L., 206, 208Wrangham, R. W., 70, 72, 73, 86, 105 Zhao, T., 429Wright, S., 166 Zhao, Z., 296Wu, G., 99, 100 Zhivotovsky, L. A., 165Wu, W., 264 Zhou, Y. D., 213Wunderlich, R. W., 33 Ziegler, T. E., 536Wurtz, R. H., 226, 229, 230, 551 Zilles, K., 406, 433, 598Wynn, K., 149, 151 Zink, C. F., 541Wynne, C. D. L., 406 Zipser, D., 276Wyttenbach, R. A., 242 Zischler, H. 17, 22 Zisserman, A., 491Xia, L., 396 Zola, S. M., 349, 350Xing, Y., 18 Zorzi, M., 391Xiong, J., 431 ¨ Zuberbuhler, K., 4–5, 86–88, 89, 92, 244, 571Xu, X., 184, 186 Zwiers, M. P., 275, 280

SUBJECT INDEXNote: Page Numbers followed by f denotes figures and t denotes tables.A36, 341–344, 346f, 347f Analog magnitude representations, 153–56Abstract categories, 242–43 Animal learning theory, 317fAbstraction, 406–6, 407–10, 415f Aniston, Jennifer, 208Acetylcholine, 295 ANS. See Approximate number systemAcetylcholinesterase, 432–3 Anterior cingulate cortex, 297, 413, 560fAcoustic size, 504–5 in emotional circuits, 298–99Acoustic stream segregation, 504 lesions, 414fAction zones, 457f, 460–65 Anterior inferior temporal cortex, 387Adapidae, 18 Anterior inferotemporal cortex, 204Adapiformes, 17–18 Anterior pulvinar, 190Adaptive resonance theory, 581 Anterior temporal lobe, 477Adaptive toolbox, 100 Anthropoidea, 12, 17, 23Addition Antibody production, 433 parametric studies of, 149 Anxiety disorders, 536 stimuli, 150f Aotus, 24, 436f task design used to test, 150f Aperiodic crystals, 162Adrenocorticotropic hormone, 366 Aphasia, 588Affect induction, 579–80 Appetitive emotional states, 303–4Affective signals, 90–91 Approximate number system (ANS), 144, 145–46Affiliate structures, 299–300 in human adults, 152Affiliation, 304, 305, 307 shared, 147–49Afrotheria, 13 Apraxia, 588Aggression, 304, 305, 359 Arabic numerals, 149, 396, 397fAkebia trifoliate, 76 Archaeoindris, 21Alarm, 304–5 Archaeolemur, 21Alexander, R.D., 167 Archaeolemuridae, 21Alouatta, 24, 65 Archistriatal gaze field, 233Altruism, 167 Archonta, 13Amble, 49, 50f Arctocebus, 21Amino acid, 258 Area V1, 206–7Amodal completion, 240f Area V2, 207Amphipithecidae, 23 Arithmetic operations, 149–50Amygdala, 374, 551 Artificial intelligence, 572, 573 in emotional circuits, 297 Artiodactyla, 13 lesions to, 373–74 Association neurons, 396 subcortical nuclei of, 297 Associative learning signalsAmygdalectomy, 373 during CMA tasks, 352–534 655

656 SUBJECT INDEXAssociative learning signals (Continued ) Autonomic responses, 296 during VPA tasks, 349–52 Avahi, 20Associative memory, 337–56 Aversive emotional states, 302 forward processing of, 341–44 Avoidance, 304, 305 in limbic cortex, 341–44 Awareness, visual, 214 neuronal organization of, 339–49 neuronal signals underlying, 349–56 Babakotia, 21 in temporal cortex, 339–41 Band-pass filter, 206 in visual cortex, 341–44 Baron-Cohen, Simon, 539Associative task, 408f Basal ganglia, 551Ateles belzebuth, 69 Baseline sustained changing cells, 353Ateles geoffroyi, 65 Basic neural activity, 320–21Attention, 212, 539 Basic uniformity, 423 approaches to, 525–26 Basolateral complex, 298 autism and, 539 Bats, 3–4 circuits involved in, 543f Bauplan, 44 coordination of, 538 Bed nucleus of the stria terminalis, 297 covert, 225–27 Behavior, 5, 6 evolutionary biology of, 531–32 complex, 405–6 feature-integration theory, 224 evolutionary theory and, 625–26 information about, 123–29 goal-directed, 119–20 multisensory, 233–34 measuring revisiting, 72f in natural settings, 528–29 neural activity and, 355f overt, 224–27 numerosity-selective neurons and, 389–90 saccades and, 231–33 of Papio ursinus, 67 shifting overt, 224–25 patterns, 589t, 591t top-down, 224–25 social, 359–76Attentional set, 412–13 Behavioral game theory, 166Attenuation, 5 Behavioral goals, 527–28Audio-visual modalities, 153 social context, 527fAuditory-category processing, 245–47 Behavioral signatures, 144–56Auditory cortex, 191, 509–11 Beliefs, 119, 129–30 core, 276 Beliefs, preferences, and constraints (BPC), reference frame results for, 277f 161, 168Auditory modality, 129, 151 Biederman, Irving, 483Auditory objects Bilophodont, 26 analysis, 244 Bimodal neurons, 616 processing, 245–47 Bimodal receptive field properties, 617f vocalizations as, 237–49 Biological face recognition, 492–94Auditory parabelt, 193 Biological object recognition, 490–91Auditory pathway Biology of Moral Systems (Alexander), 167 coding format in, 280, 283–86 Biomechanical studies, 31 coordinate transformations, 286 Bipedalism, 33, 43 modeling, 283–86 ancestral, 44 motor commands, 281–83 Bisection tasks, 146 reference frame, 273–79 Bitotinylated dextran amine, 621f representational format, 279–81 Black-Scholes options pricing equation, 163Auditory-scene analysis, 239 Blindness, 214Auditory systems, 182–83 Blindsight patients, 296Auditory-visual trial, 155f Blocking test, 331fAustralopithecus, 25–26 Blood oxygen level-dependent (BOLD), 212,Autism, 539 302, 306Autonomic center, 294–95 Body, 8

SUBJECT INDEX 657BOLD. See Blood oxygen level-dependent Cebus capucinus, 88–89Bonobos, 590 Cell biology, 431–37Border collies, 87–87 Center of mass, 50–54Boreotheria, 13 Center-surround inhibition, 206Bottom-up guidance, 224 Central timing mechanism, 265Bounded rationality, 100, 102 Centre Suisse de Recherches Scientifiques, 78BPC. See Beliefs, preferences, and constraints Centripetal surveillance, 531Brachiation, 43 Cercocebus atys atys, 66, 67fBrain-body state, 292 Cercopithecids, 25, 26–27Brain lesion studies Cercopithecoidea, 25, 36t infant development and, 373 Cercopithecus ascanius, 73 social behavior and, 372–75 Cercopithecus mitis, 73Brainstem, 294–95 Cerebellar asymmetries, 599–601, 601tBroca’s area, 599 Cerebellar flocculus, 281Brodmann’s numbers, 428, 433 Cerebellum, interhemispheric connectivity and,Bycanistes subcylindricus, 74 596–98 Cetacea, 13Cacajao, 24 Character virtues, 168–69Caching, 135 Cheirogaleidae, 17f, 21Caipora, 24 Cheirogalues medius, 19Cajal-Retzius cells, 440 Chimpanzee, 406, 501–2, 590Calbindin, 433 cooperation of, 132Callicebus, 24 silent approach, 130Callimico, 24 social cognition, 133Callithrix jacchus, 47, 105, 123, 437 theory of mind in, 118Callitrichidae, 16, 36t Chiropotes, 24Calretinin, 434 Chlorocebus pygerethrus, 500Caluromys philander, 46 Choice consistency, 162Campbell’s monkeys, 244 Church-Turing thesis, 572Cane-pulling task, 595–96 Cingulum bundle, 429fCapuchin, 145f, 594f Civilized environent, 629Cardinality, 385–98 Clades, 11 representation of, 386f Haplorhini, 18fCat-301, 436f phylogenetic relationships of, 14fCatarrhini, 23, 24–25, 424 Cladistics systematics, 11–13 locomotor modes, 36t Cladograms, 11Categories, 241–43 Classifications, 10–11 abstract, 242–43 Clever Hans, 144, 525 auditory, 245–47 Climbing, 465 perceptual, 242–43 Coding format processing, 246–47 auditory pathway in, 280 subordinate, 241 transformations of, 283–86Caudal cingulate motor areas, 187f Codon, 258Caudate nucleus, 417, 429f Cognition, 5, 6, 574Caudomedial area, 193 evolutionary theory of, 64–66Causality, 258 Cognitive revolution, 570CC. See Corpus callosum Colliculo-pulvinar pathway, 296Ceboidea, 23–24, 35t Colobidae, 26Cebuella, 24 Colobinae, 38tCebuella pygmaea, 89 Colobus, 27Cebus, 24, 597, 607 Colugos, 424Cebus apella, 102, 120, 503 Common accounts, 169Cebus apella nigritus, 68 Communicative cues, 120–22

658 SUBJECT INDEXComparative genomics, 437–43 Crocutta crocutta, 134Comparative histology, 431–37 Cross-modal representations, 152–53Comparative Mammalian Brain Collection, 443 Cue-holding index, 345Comparative method, 133–34 Cultural transmission, 164Comparative molecular approaches, 437–43 Cytochrome oxidase, 432–433, 435fComparative Neurobiology of Aging Resource, 443Comparative neuropsychology, 587–608 DA. See DopamineCompetition, 131–32 Damasio, Antonio, 539Competitor behavior, 72–75 Darwin, Charles, 3, 10–11, 84, 90, 117Complex adaptive systems, 161–62 Daubentonia madagascariensis, 427f, 429fComplex behavior, 405–6 Daubentoniidae, 17f, 20, 35tComplex movements, 465–67 Dawkins, Richard, 164, 165Complex systems, 161 Decision making, 213–14, 550–66Computer-assisting tomography, 426 in competitive games, 557–60Computer metaphor, 572 multistage, 560–65Computer vision, 491–92 Declarative memory, 338fConceptual primitives, 92 Defensive movements, 462–63Conditional discrimination, 148f Deictic gestures, 532, 538Conditional motor association, 338, 349 Delayed gratification, 110, 340 associative learning signals during, 352–54 Delayed matching-to-sample (DMS), 339, 347, changing cells, 352f 386f, 388fConditioned appetitive responses, 303–4 Dendrite model, 286, 288fConditioned fear, 302 Denny-Brown, Derek, 31Conditioning, 316 Dermoptera, 437 excitatory, 316 Descartes’ Error (Damasio), 539 Pavlovian, 316, 331 Design-based stereology, 434Conflict, 412 Diagonal sequence, 45–46Containing, 540 Diaminobenzidine, 433Contiguity, 316 Didabots, 574Contingency, 316 Differential responses, 327, 329fContinuous quantity, 393–96 Diffusion-tensor imaging (DTI), 428, 429f neuronal coding of, 394f Digital Fish Library, 444Convergence, 134–35 Digit morphology, 44Convergent vision, 201–202 Dimensionality reduction, 459–60Cooperation, 131–32 Discharge patterns, 283Coordinate transformations, 286 Discounted utility model, 107Coos, 305, 503, 510 Discrete quantityCoots, 151 coding of, 393–96Corpus callosum (CC), 429f, 596–97 neuronal coding of, 394fCorrelated parameters, 258–60 Distal arm muscles, 267Correlation, 257–58 Distractor stimulus, 230Cortical asymmetries, 598–99 DMS. See Delayed matching-to-sampleCortical motor areas, 194–95 Dolphins, 406Cortical motor system, 459f Dominance rank, 306–7, 529–31Cortical-thalamic-cortical loops, 186 Dominance relations, 89Corvids, 406 Dopamine (DA), 294 caching of, 135 background, 325–27 social cognitive skills of, 135 differential responses of, 329fCotton-top tamarins, 151 neurons, 325–31Count words, 145–46 reward-predicting stimulus, 330fCovariation, 257–58 in social behavior, 366Covert attention, 225–27 Dopaminergic neurons, 366Cows, 13–16 Dorsal premotor cortex, 261

SUBJECT INDEX 659Dorsal processing stream, 245 mediodorsal nucleus of thalamus in, 298Dorsal thalamus, 183 orbitofrontal cortex in, 299Dorsolateral prefrontal cortex, 324, 557, 558f, organization of, 294–301 559f, 562f PAG in, 295Dorsolateral substantia nigra, 325 processing, 301–308Dorsolateral view of brain, 178f, 187f superior colliculus in, 296Dorsolateral visual area, 186 vPFC in, 301Dot displays, 147f Encephalogram, 296007 principle, 574 b-endorphin, 362Douc langur, 42f Endogenous opioids, 361–64DRD4, 367 Energy landscape, 581Dryopithecus, 25 Entrez Gene web site, 445DTI. See Diffusion-tensor imaging Eosimiidae, 23DUF1220 domain, 439 Equipotential, 422Duke Lemur Center, 529 Erwin, J.M., 443 Ethylene glycol, 437Early mammals Euarchontoglires, 13 auditory systems, 182–83 Eulemur fulvus, 104 motors systems of, 177–84 Eulemur macaco, 104 olfactory systems, 183–84 Eulemur mongoz, 20 pain, 181–82 Euoticus, 21 sensory systems of, 177–84 Event-related potential, 476 somatosensory system of, 178–81 Evolution taste, 181 convergence and, 134–35 temperature, 181–82 human, 135 vestibular systems, 182–83 Evolutionary game theory, 160–61, 166 visual system of, 177–78 Evolutionary perspective, 160, 162Early visual areas, 205 Evolutionary systematics, 10Ecological intelligence, 65, 66 Excitatory conditioning, 316Ecological rationality, 100–101, 105, 109–10 Executive control circuits, 405–18Economic theory, 163 sets, 410–13Egg counting, 151 strategies, 410–13Electric fish, 3–4 Expected utility theory, 99Electroencephalography, 426 Experimental game theory, 167–71Electromagnetic fields, 426 Expert systems, 572Electromyographic data, 55, 265 Exponential model, 107Emergent properties, 161 Expression of the Emotions in Animal and ManEmerson, Ralph Waldo, 532 (Darwin), 90Emotional states Expressions, 304 appetitive, 303–4 evaluating, 305 aversive, 302 generating, 304Emotion circuits, 292–309 neural basis for, 305 affiliate structures, 299–300 perceiving, 305 anterior cingulate cortex in, 298–99 produced with vocalizations, 502f basolateral complex, 298 Extended food grunt, 603 core structures, 294 Extended Phenotype (Dawkins), 165 hippocampus in, 300 External thinking devices, 629 hypothalamus in, 296–97 External validity, 444 insula in, 298 Extinct species, 424 intermediate structures of, 297–98 Extrastriate body area, 542 lateral temporal lobe in, 300 Eye-centered target location, 275f LIP in, 301 Eye movements, 538 medial pulvinar in, 301 Eye trackers, 553

660 SUBJECT INDEXFace cells, 249 Frontal lobe circuitry, 506–7Face encoding, 508 face encoding, 508Face-patch system, 478 voice encoding, 508Face processing system, 476–77 Frontal pole, 416 in macaques, 477–80 Fronto-insular cortex, 434 model, 494f Fronto-orbital sulcus, 600fFace recognition, 492–94, 503 Fruit, 77–78Face representation, 487–89 coevolution of primates and, 64Face selectivity, 479f finding, 66–67Facial identities, 208 localization strategies, 70–76Facial movement, 501–502 FSTd, 186False-belief tasks, 130 Functional connectivity MRI (fcMRI), 431, 443Fast Blue, 620 Functional magnetic resonance imaging (fMRI),Fayum Depression, 19 207, 361, 409, 430–31, 479, 480f, 513, 540fcMRI. See Functional connectivity MRI Functional reference, 243Fear, 302, 304 Fusiform face area, 208, 542FeatureGate model, 224 Fusiform gyrus, 208Feature-integration theory, 224FEF. See Frontal eye field GABAergic interneurons, 299Ficus capensis, 75f Gabor wavelet-style decomposition, 211Ficus sansibarica, 70, 71f Gaits, 49fField playback studies, 153 inverted pendulum, 52, 52fFigs, 74f visual classification, 51Filtering, 501 Galagidae, 17fFine branch niche hypothesis, 526 Galago, 21Fisher’s Fundamental Theorem, 166 Galagoides, 21Fitness, 99, 162 Galagonidae, 35tFitness-relevant response, 160–61 Game theory, 165–67Fitt’s law, 259 behavioral, 1665-HT. See Serotonin evolutionary, 160, 166Flexible vocal production, 91–93 experimental, 167–71Fluorodeoxyglucose, 431 history of, 166fMRI. See Functional magnetic resonance imaging individual payoffs in, 165Food calls, 513 information in, 165Footfall pattern, 45–46 players in, 165Foraging behavior, 4–5 strategies in, 165Forelimb, 33 Gaussian curve fits, 281fForelimb-dominated locomotion, 42f Gaussian functions, 280Forests, 64 Gaze, 92Formants, 501 control, 233–34Fourier descriptors, 340 cues, 126Fourier energy analysis, 206 direction, 306Fractal patterns, 340f following, 122–23, 534f, 535–37, 535fFractional movements, 256 mirror network and, 542–44Free-floating rationale, 570 sex differences in, 536fFrontal cortex, 183 as strategic social signal, 537–39 reward functions, 318–19 studies of, 124t–125tFrontal eye field (FEF), 187f, 194, 195, 204, 214, 225, tracking, 528f 398, 541 using information about, 123–29 electrical microstimulation of sites within, 228f, Gene-culture coevolution, 160, 163–65 231f Gene-environment coevolution, 165 projections from, 232f Gene expression, 439, 619–22 stimulation, 230 General equilibrium theory, 167

SUBJECT INDEX 661Geon theory, 483, 484f Hominoidea, 25, 38tGestalt psychology, 238 Hominoidea, 25Gestural communication, 605–7 Homo economicus, 167–68Ghiselin, Michael, 167 Homoplasies, 11Gibson, J.J., 576 Homo sapiens, 147f, 160, 444Glial cell morphologies, 434 Homo spp., 25–26Glires, 13 Honesty, 168Goal-directed behavior, 119–20 Honeybees, 410Goal emulation, 119 Horizontal cascades, 348–49Go/no-go tasks, 351, 351f Huffman encoding, 205Gorilla gorilla, 103, 590 Human evolution, 135Gorilla spp., 26 Human infants/adults, 152Grandmother cells, 208 Human language, 193–94, 406Grasp, 265 Human ventral pathway, 476Great Ape Aging Project, 443 Humphrey, Nicholas, 117Greebles, 208 Hyaena hyaena, 134–35Grimace, 304, 501 3-hydroxy-4-methoxyphenylglycol (MHPG), 367Grooming, 304, 359, 363, 365 5-hydroxyindoleacetic acid (5-HIAA), 367, 369Grunts, 75f, 85, 510 Hylobatidae, 25 extended food, 603 Hyperbolic discounting model, 107 infant, 85, 87, 91 Hypothalamic-pituitary-adrenal axis, 297 move, 85, 91 Hypothalamus, 296–97, 375 threat, 86Gustatory cortex, 191 IC. See Inferior colliculus Ignorance, 130Habitats, 64, 67–70 Illusory contours, 209fHadropithecus, 21 Impulse control, 371Handedness, 599, 601–603, 602t, 604f Inclusive fitness, 167Hand movements, 458f, 464 Incompatible models, 160Hand path, 259f Individual decision making, 101Hand-to-mouth movements, 460–62 Individual payoffs, 165Hapalemur, 20 Indriidae, 17fHaplorhini, 17, 18f, 21–27, 35t locomotor modes, 35tHAR1F, 440 Indriidae, 20Head-centered receptive fields, 274f Infant grunts, 85, 87, 91Head-centered target location, 275f Inferior colliculus (IC), 275fHead direction, 306 representational format in, 282fHeiligenberg, Walter, 3 responses of, 275f5-HIAA. See 5-hydroxyindoleacetic acid Inferior temporal cortex (IT), 186, 474fHidden-Markov model, 513–14 Inferior temporal lobe, representations in, 480–90High-level information Inferotemporal cortex, 249, 471, 486f functional organization of, 413–17 Inferotemporal gyrus, 300 implementation of, 417–18 Inflection points, 285Hindlimb-dominated locomotion, 42f Inhibitory control, 103–4Hippocampus, 179, 300, 353 Inhibitory synapses, 285Hippos, 13–16 Insula, 298H.M., patient, 337 Intellectual mind, 623–30HMAX model, 491 Intentional actions, 119–20Holistic face detection, 486f Intentional agents, 118Holistic processing, 489 Intentional niche construction, 627, 629fHolloway, Ralph, 422 Intentions, 532–35Holophyletic, 11 Interhemispheric connectivity, 596–98Hominidae, 16, 25–26, 38t Interim summary, 480, 489–90

662 SUBJECT INDEXIntermediate structures, 297–98 Leaping, 465Internal validity, 444 Learning, 164Intertemporal choice, 107 attentional set and, 412–13 building blocks, 110 as change in predictions, 316–18 ecological rationality of, 109–10 changes in selectivity with, 354fIntracortical microstimulation, 266 neural activity and, 355fIntransitive movement, 623–24 reward expectation during, 323f, 324fIntraparietal sulcus, 387, 388f, 618f, 620 situations, 331–32Invariance, 258 synaptic mechanisms, 333Invariant choices, 99 tasks, 319–20, 320fInverted pendulum gait, 52–53, 52f theory, 317fInvisible hand, 167 Left forefoot, 45Irrelevant alternatives, 101 Lemur catta, 20, 528IT. See Inferior temporal cortex Lemuridae, 17f, 20Ivory Coast, 78, 89 locomotor modes, 34t Lemuriformes, 19–20Jamming avoidance, 3 locomotor modes, 34tJays, 135 Lemuroidea, 17f, 19, 20Jerison, Harry, 422 Lepilemuriade, 20Jitters, 91 Lepilemuridae, 17f Lexical syntax, 92Kanisza Triangle, 207 LGN. See Lateral geniculate nucleusK-cell class, 178 Limbic cortexK-cell pathway, 184 associative memory in, 341–44Kibale National Park, 78 horizontal cascades of information in, 348–49Kin, 307 memory retrieval from, 344–48Kinematics, 267f Limbic system, 298Kinetic energy, 51–52 Limb position, 46Kirsh, David, 583 Limb stiffness, 48–49Klipsch horns, 574 Line of decussation, 185fKnowledge, 119, 129–30 LIP. See Lateral intraparietal cortexKnuckle and fist walking, 43 Lip smack, 304Koboldmaki, 22 Listeners, 87 Living systems, 161Labeled-line code, 391 Local field potentials, 264, 509Lagothrix poeppigii, 69 Locke, 572Language, 193–94, 406 Locomotion, 4Larson, Susan, 54 amble in, 49Lashley, Karl, 422 center of mass in, 50–54Last common ancestor, 11 diversity, 34–44Lateral belt auditory cortex, 511–12 evolution, 31–54Lateral geniculate nucleus (LGN), 184, 203, 205–6, experimental studies of, 32t–33t 212, 228f, 424 extremes, 42fLateral intraparietal cortex (LIP), 204, 213–14, 225, footfall pattern, 45–46 276, 391, 541 forelimb-dominated, 42f in emotional circuits, 302 hindlimb-dominated, 42f reference frame results for, 277f limb stiffness, 48–49Lateral prefrontal cortex (LPFC), 386f, 407, 408f, pathway of evolution, 53–54 409, 412, 416, 417–18 primate v. nonprimate, 44–54 lesions, 414f reduced oscillations, 50–54Lateral sequence, 45f running trot in, 49Lateral temporal lobe, 300 touchdown in, 46Laurasiatheria, 13 weight distribution in, 46–47

SUBJECT INDEX 663Locomotor control, 4 Medial temporal lobe, 337–56Long-term memory, 338f posttraining lesions to, 352 backward spreading of retrieval, 344–48 Mediodorsal nucleus of thalamus, 298 neuronal organization of, 339–49 Megaladapis, 21 in temporal cortex, 339–41 Megalapidae, 35tLophocebus albigena johnstonni, 65, 66–67 Meissner corpuscles, 190Lorenz, Konrad, 3 Meme, 164Loris, 21 MemoryLorisoidea, 17f, 19 associative, 337–56 locomotor modes, 35t declarative, 338fLorsidae, 17f long-term, 338f, 339–49Loss aversion, 106 nondeclarative, 338fLost caps, 434 retrieval, 344–48Low-frequency electrical stimulation, 232 working, 212–13LPFC. See Lateral prefrontal cortex Mesopropithecus, 21 Messenger RNAs (mRNAs), 441f, 442Macaca, 27, 436f Meteorological cues, 71–72Macaca fuscata, 75, 86, 103, 500 MHPG. See 3-hydroxy-4-methoxyphenylglycolMacaca mulatta, 86, 122, 147f, 500, 530 Microcebus, 21Macaca nemestrina, 441f Microstimulation, 229, 231fMacaca sinica, 73 FEF, 228fMacaca tonkeana, 72 SEF, 228fMacaques, 477–80 Midbrain, 465–66Machiavellian intelligence, 117, 578 Mind Blindness (Baron-Cohen), 539Machiavellianism, 169 Mind reading, 122–23Magnetic resonance imaging (MRI), 426, Miopithecus, 27 598–99, 618f Mirror network, 542–44 structural imaging, 426–30 Mirror neurons, 166Magnitude perception, 102–3 Mirza coquereli, 21Magnocellular nucleus, 182 Mode-control model, 154, 391Mammals, 10 Model-animal paradigm, 423 early, 177–84 Modeling, 283–86 vocal production of, 87–88 Modified body image, 616–19Mandeville, Bernard, 167 Mongoose lemurs, 152Mandrillus, 27 Monodelphis domestica, 45, 46Manipulation movements, 463–64 Monogamy, 360Manual fovea, 464 Morpheus family, 439Manual motor skill, 603–5 Morphine, 363Masking, 229 Motor center, 294–95Matching pennies game, 551–54 Motor commands, 281–83 reinforcement learning in, 555–56 Motor cortex, 6–7, 188fMatching-to-sample task, 406, 409f action zones in, 457f, 460–65Maternal care, 360, 370 encoding in, 256–68Mating habits, 98 inputs to, 263–64M-cell pathway, 184 movement and, 454–67Medial dorsal nucleus, 191 outputs of, 264–65Medial forebrain eggs, 293f Motor cortical encoding, 257tMedial frontal cortex, 432f, 551 Motor homunculus, 456fMedial geniculate complex, 182 Motor system, 6, 177–96Medial intraparietal cortex, 276 Mouth movements, 464 reference frame results for, 277f Move grunts, 85, 91Medial posterior cortical, 432f MovementMedial pulvinar, 301 defensive, 462–63

664 SUBJECT INDEXMovement (Continued ) Neuronal coding distance, 260f continuous quantity, 394f facial, 501–502 discrete quantity, 394f fragment, 256 Neuronal organization hand, 464 of associative memory, 339–49 hand-to-mouth, 460–62 of long-term memory, 339–49 intransitive, 623–24 Neuroscience technologies, 425–43 manipulation, 463–64 Neurotransmitters, 294 motor cortex and, 454–67 Newton’s second law, 259 mouth, 464 NIH. See National Institutes of Health number of remaining, 563–64 Nobel Prize, 3 reach-to-grasp, 464 Nobuo Suga, 3 stimulation-evoked, 455f Non-accidental property, 483Movement parameters Nondeclarative memory, 338f encoding of, 260–63 Nonmonotonic response functions, 274f motor cortical encoding of, 257t Nonoptimal stimuli, 245MRI. See Magnetic resonance imaging Nonswitch trials, 413Multidimensionality, 308 Norepinephrine, 294Multidimensional scaling, 484 in social behavior, 367–68Multisensory attention, 233–34 Notebook M (Darwin), 3Multisensory vocal communication, 505–6 Notharctidae, 18Multistage decision making, 560–65 Nottebohm, Fernando, 3Music, 241 Nucleus of solitary tract, 295Myelin, 429f, 433 Number of remaining movements, 563–64Naloxone, 363 Numerical approximation, 5Nash, Stephen, 39f Numerical cognitionNash equilibrium, 552, 553 behavioral signatures of, 144–56National Center for Biotechnology Information of capuchins, 145f (NCBI), 445 neural bases of, 385–401National Committee for Research Resources of rhesus monkeys, 145f (NCRR), 425 Numerical distance effect, 389National Institutes of Health (NIH), 425 Numerical ordering tasks, 145fNational Science Foundation, 444 accuracy in, 146fNatural images, 209–10, 211f reaction time in, 146fNaturalistic saccades, 210f Numerical quantity, 385–98Natural selection, 623–24, 629f neurons encoding, 385–89Natural vision, 209–14 Numerical rank, 398–400NCBI. See National Center for Biotechnology Numerical representations, 151NCRR. See National Committee for Research analog magnitude, 153–56 Resources cross-modal, 152–53Neocortex, 196 scaling of, 393Neural coding, 205 semantic associations in, 397fNeural correlates, 245–47 symbolic, 396–98Neural encoding Numerical size effect, 389 beyond, 263 Numerical taxonomy, 11 defining, 256–58 Numerosity of movement parameters, 260–63 behavioral significance and, 389–90Neural filtering model, 391 detectors, 390–93, 392fNeural network model, 156 filters, 390Neural response, 328f sequentially presented, 388fNeurofilament protein, 435f, 436f Nutritional requirement of brain, 65Neuroimaging, 426 Nycticebus, 21

SUBJECT INDEX 665OBF. See Orbitofrontal cortex Panksepp’s hypothesis, 362Object-choice paradigm, 120 Pan paniscus, 67, 103, 105Object identification, 207–8 Pan spp., 26Object recognition, 490–94 Pant-hoot, 75f biological, 490–91 Pan troglodytis, 69, 89, 102, 119, 147f, 435f, 441f, computer vision and, 491–92 500, 501Object representation, 481–87 Pant-threat, 304Objects, 237–40 Papio, 27, 598 auditory, 237–49 Papio ursinus, 67, 69 boundaries, 238f Parabrachial nuclei, 295 categorical structure of, 241f Parahyaena brunnea, 135 constancy, 242 Paraphyletic, 11 perceptions of, 238 Parapithecidae, 23 processing, 245–47 Parcellate, 433Object selectivity, 479f Parietal ventral area, 179Object task, 408f Parietal insular vestibular cortex, 194Occipital face area, 476 Parietal neurons, 616–19Oculomotor pathway, 281, 561f Parrots, 406Olfactory systems, 183–84 Partner treatment, 170Oligopithecidae, 23 average contributions in, 171fOmomyoidea, 22 Passive niche construction, 626Open-mouth stare, 304–5 Pathlets, 262Opisthocomus hoazin, 78 Pavlovian conditioning, 316, 332Optimal foraging theory, 108 P-chlorophenylalanine, 369Optimal stimuli, 245 Penfield, 454Orbital convergence, 201–2 Perceptions, 119Orbitofrontal cortex (OBF), 192f, 297, 324–25, of objects, 238f 542, 551 understanding, 127t in emotional circuits, 299 visual, 126 population responses in, 326f Perceptual categories, 242Ordinal continua, 149 Perceptual control theory, 580Ordinality, 398–401 Perfect Stranger treatment, 170Orientation tuning, 206 average contributions in, 171fOrienting, 527–28 Periaqueductal gray, 295Orofacial control, 603–5 Perigenual cingulate cortex, 371Ostrom, Elinor, 169 Perirhinal cortex, 179Otolemur, 21 Perisoreus canadensis, 101Overt attention, 225–27 Peri-stimulus time histogram, 342f, 344f, 346fOwl monkey, 188f Perodicticus, 21Oxytocin, 364–66 Perrett, David, 537Oxytocin receptor antagonist, 364 Person coefficients, 493f Perspective taking, 129–30, 135Pacemakers, 154 strategic, 131PACPNT, 78 PET. See Positron emission tomographyPain, 181–82, 191 PFC. See Prefrontal cortexPair-coding, 340, 342f, 398 Phaseolus vulgaris, 474f comparison of, 343f Phosphenes, 229, 230 index, 350 Photoreceptor cells, 205Pair-recall, 340, 342f, 346f Phylogenetic regression techniques, 434 index, 345 Phylogenetic systematics, 11, 14fPaleobiology, 164 Phylogeny, 4Paleopropithecidae, 21 Piagetian tasks, 595Paleopropithecus, 21 Pigeons, 106, 406

666 SUBJECT INDEXPiggybacking, 500–501 Premotor cortex, 407Pithecia, 24 Premotor cortical neuron, 264Pithecia pithecia, 71 Presaccadic enhancement, 226Pithecoid gait, 45 Presbytis melalophos, 41fPlace code, 273, 280, 284f, 285 Pre-supplementary motor area, 563, 565fPlanum temporale, 600f Primary motor cortex, 183, 399Platyrrhini, 23–24, 35t, 424 Private accounts, 169Players, 165 Proconsuloidea, 25Plesiadapiformes, 16 Promise-keeping, 168Plesions, 12 Propithecus, 20Pliopithecoidea, 25 Prosimian groups, 16, 151–52Pluripotency, 308 Prosimii, 16–17Pogoniulus bilineatus, 74 Proteopithecidae, 23Polyandrous systems, 360 Protopithecus, 24Polygynous systems, 360 Proximal arm muscles, 267Polysensory zone, 462 Psychological states, 118–32Pongidae, 16, 25 Public goods game, 169–71Pongo, 436fPongo pygmaeus, 103, 120 Quadruped, 40Pop out, 223 Quadrupedalism, 40fPose coefficients, 493f Quadrupedal leapers, 40Positron emission tomography (PET), 361, 426, Quadrupedal runners, 40 431, 476, 513, 599, 617, 618fPosterior cingulate cortex, 551 Rake, 616f, 627, 628fPosterior inferotemporal cortex, 204 Random target pursuit task, 259f, 262fPosterior parietal cortex, 187–90, 385–87, 386f, Raphe nuclei, 363 432f, 551 RASP, 603, 604f caudal zone, 187 Rate, 108–9 organization of, 189f Rate codes, 284f, 285 rostral zone, 187 Rational actor model, 161, 162Postorbital septum, 12 Rationality, 99–101Postpartum, 370 bounded, 100–101Potential energy, 51–52 choice and, 99Power spectrum, 210 ecological, 100–101, 105, 109–10 natural image and, 211f irrational choice and, 99–100Preattentive processing, 224 Ravens, 135Predictions Reach, 265 error coding, 328–31, 331f Reach-to-grasp movements, 464 errors, 317–18, 327–28, 332–33 Reaction time, 146f learning as change in, 316–18 Receptive field, 224–25, 284fPredictive go/no-go tasks, 351f bimodal, 617fPreference, 101–104 head-centered, 274fPreferential looking paradigm, 154f somatosensory, 617fPreferred directions, 260 Reciprocal altruism, 167 systematic shifts in, 262f Recycling hypothesis, 396 temporal shifts in, 261 Redeployment hypothesis, 396Preferred numerosity, 385 Reduced oscillations, 50–54Preferred trajectories, 261 Reference dependence, 106Prefrontal cortex (PFC), 155, 165, 386f, 397f, 400f, Reference frame 506–7, 514 auditory cortex, 277f abstract information and, 407–10 auditory pathway, 273–79 right, 416f hybrid, 276Prehension, 265–67 LIP, 277f

SUBJECT INDEX 667 MIP, 277f Saccade-related bursts, 278–79 superior colliculus, 278f Saccades, 211, 223Reflected escape, 531 attention and, 231–33Regular-random numerosity illusion, 147 eye position and, 279fReinforcement learning theory, 550 naturalistic, 210f in matching pennies game, 555–56 preparation, 227–33 in rock-paper-scissors game, 552f, visual, 155–56 556–57 Saguinus fusciocollis, 72Repeat-stay, change-shift strategy, 411 Saguinus imperator, 72Replicator dynamic, 166 Saguinus labiatus, 73Replicators, 162, 166 Saguinus oedipus, 89, 122, 500Representational format, 279–81, 282f Saimiri, 24, 436f, 597Rescorla-Wagner learning rule, 330 Saimiri sciureus, 104, 435fResonant frequencies, 426 Salience, 224Resting-state cognitive activity, 432f Sameness, 406Reticular nucleus, 180f Sanguinus fuscicollis, 69Retina, 205–6 Sanguinus mystax, 69Revealed preferences, 163 Savage, Leonard, 162Reverberation circuits, 348f SC. See Superior colliculusReverse transcriptase polymerase chain reaction Scandentia, 437 (RT-PCR), 441f Scoville, Dr., 337Revisiting behavior Screams, 86, 305 measuring, 72f Secondary cues, 69–70 temperature and, 73f Second motor area, 183Reward expectation SEF. See Supplementary eye field adaptation of, 321–24 Selective convergence model, 344f, 348–49, 348f changes, 332 Self-control experiments, 108f during learning, 323f, 324f Selfish Gene (Dawkins), 164, 167Reward functions Self-organizing maps, 460 frontal cortical, 318–20 Semantic associations, 397f striatal, 318–19 Sensorimotor transformation, 187–90Reward-predicting stimuli Sensory center, 294–95 acquisition of responses to, 321, 327, 332 Sensory cortex, 188f caudate neuron in, 322f Sensory system, 6, 177–96 differential responses, 327 Sensory tool usage, 628f dopamine in, 330f Sequentially presented numerosity, 388fReynolds’ mechanism, 48 Sequential protocol, 387Rhesus monkeys, 406 Serial order, 385 numerical cognition of, 145f Serotonergic neurons, 364 preferential looking paradigm, 154f Serotonin (5-HT), 294, 307 visual cardinality in, 386f in social behavior, 368–72Ricochetal brachiation, 43 transporters, 371Right hindfoot, 45 Sex, 529–31Right prefrontal cortex, 416f Sheba, 149Risk, 104–6 Short Interspersed Elements. See SINEs framing of, 105–6 Shrill bark, 304 gambles, 104–5 SIFT algorithm, 492 preferences, 105 SIGLEC16, 440Rock-paper-scissors game, 552f, 556–57 Sigmoidal curve fits, 281fRT-PCR. See Reverse transcriptase polymerase Signalers, 87 chain reaction Simple call sentences, 85–87Rules, 406 Simultaneous protocol, 387Running trot, 49 SINEs, 13, 17

668 SUBJECT INDEXSivaladapidae, 18 Stimulus onset asynchrony, 533Sivapithecus, 25 Stranger treatment, 170, 171f6-degree-of-freedom robot, 266 Strategic behavior, 570–84SMA. See Supplementary motor cortex Strategic interaction, 160Smith, Adam, 167 Strategic punishment, 170Social affordances, 576–79 Strepsirrhini, 17–21, 18f, 34t, 424Social behavior, 359–76 Striatum, 318–24 brain lesion studies and, 372–75 reward functions, 318–19 components of, 360–61 Strombosia schefferi, 76 dopamine in, 366–67 Strong reciprocity, 168 endogenous opioids in, 361–64 Subcortical nuclei of amygdala, 297 neural substrates of, 372–75 Subcortical pathway, 541–42 neurochemical control of, 361–72 Submission, 359 norepinephrine in, 367–68 Substantia nigra, 325 oxytocin in, 364–66 Sub-threshold currents, 230 serotonin in, 368–72 Sugrivapithecidae, 25 vasopressin in, 364–66 Summing localization, 280Social cognition, 588 Superior colliculus (SC), 185f, 225, 228f, 541 of baboons, 86 cat, 279 chimpanzee, 133 electrical microstimulation of, 228f cooperative behavior and, 131–32 in emotional circuits, 296 in corvids, 135 reference frame, 278f evolution of, 132–36 Superior temporal gyrus, 507 forces shaping, 133–34 Superior temporal sulcus, 165, 207, 293f, human evolution and, 135 300, 432f, 473f, 476, 506–12, 512f,Social-cue, 121t 542, 621fSocial envelope, 304 Supplementary eye field (SEF), 195, 228f, 399Social intelligence hypothesis, 65, 580–82 Supplementary motor cortex (SMA), 187f, 194–95,Social referencing, 92 398, 563, 564fSocial salience, 529–31 Swiss robots, 574Social tools, 605–7 Switch trials, 413Society for Neuroscience, 9 Symbolic signals, 90–91Socrates, 572 Symplesiomorphies, 11Somatic responses, 292 Synapomorphies, 11Somatosensory area, 179 Synchronicity, 75–76Somatosensory cortex, 179, 281 Systema Naturae, 10Somatosensory receptive fields, 617fSomatosensory system, 178–81, 190 Tactical deception, 577 organization of, 180f Tai Monkey Project, 78Spatial location, 280, 458f Target location, 275fSpatiotemporally bounded units, 237 Tarsiliformes, 22–23Spatio-temporal mental representations, 67–70 locomotor modes, 35tSpeed measurement, 68f Tarsius, 22, 22fSpike-rate measure, 395 Tarsoidea, 22Spike-triggered averaging, 455 Task sets, 412Spinal cord, 466 Taste, 181Spinal injury, 31 TE, 345, 346f, 347f, 471, 473Squirrel monkey, 192f Temperature, 73f, 181–82, 191States, 307–8 Temporal cortex, 339–41Steering, 44 Temporal discounting, 107–8Stem groups, 12 Temporal lobe circuitry, 506–8, 508Stereoscopic vision, 201–2 Temporal ordering task, 400fStimulation-evoked movements, 455f Temporal-prefrontal connectivity, 516f

SUBJECT INDEX 669Temporal profiles, 264f Umwelt, 575Temporoparietal junction, 620, 621f Unable, 120TEO, 471, 473 Uncertainty, 104–6Territorial defense, 151 Unconditioned fear, 302Thalamus, 190 Unique specificity, 258Theory of mind, 92, 93, 118, 131, 625 Unwilling, 120Thinking, 573 Utility, 101–4Thorndike’s law of effect, 554 building blocks, 102Threat calls, 503 functions, 163Threat grunts, 86 Uvariopsis congensis, 76, 77fThreats, 305–6Thrombospondin 2, 440, 442 Valuation, 103Thrombospondin 4, 440, 442 van Osten, Wilhelm, 525Time, 106–10 Varecia, 17fTime frequency plots, 512f Vasopressin, 364–66Tinbergen, Niko, 3 Vector averaging models, 285T-maze learning, 321 Vector subtraction model, 286, 287fTonically active striatal interneurons, 321 Ventral intraparietal area (VIP), 387, 467Tools, 7, 588–94 Ventral medial nucleus, 191, 192f cognitive mechanisms governing, 594–96 Ventral processing stream, 245 complex, 620f Ventral regions, 458 individual differences in, 598–603 Ventral tegmental area, 325, 364 manual motor skill and, 603–5 Ventral visual pathway, 472–77, 473f neural correlates of, 596 Ventrolateral prefrontal cortex (vPFC), 247f, 248f, neural mechanisms subserving, 615–23 513–17, 514f sensory, 628f emotional circuits in, 301 social, 605–7 multisensory integration in, 517f stepwise mastery of, 626–30 response of, 515f stone, 594f Ventroposterior inferior nucleus, 181, 190Toothcomb, 17, 19 Ventroposterior medial nucleus, 181Top-down attention, 224 Ventroposterior superior nucleus, 179, 180fTopographic organization, 459–60 Vertebrae, 12–13Touchdown, 46 Vertical clinging and leaping, 42–43, 44Trachypithecus obscura, 41f Vertical peak, 46, 47fTraining process, 615, 622–23 Vervet monkeys, 49fTraits, 307 Vestibular cortex, 194Transcranial magnetic stimulation, 426 Vestibular systems, 182–83Transdisciplinary behavioral science, Video games, 617 160–72 VIP. See Ventral intraparietal areaTransient response, 322f Vision, 5Transition durations, 346f Visual areas, 187fTransition times, 346f Visual attention circuits, 223–34Transitive choices, 99 Visual awareness, 214Transitive preferences, 101 Visual cortex, 178Trap-tube task, 595 associative memory, 341–474Trees, 70–71 memory retrieval from, 344–48Tree shrews, 424 Visual modality, 152Trichromatic vision, 202–3 Visual paired associate (VPA), 338, 340, 341fTriggering events, 292 associative learning signals during,True altruism, 167 349–42Twitches, 462 schematic representation of, 350fTwitch-evoking sites, 454 Visual posterior sylvian area, 194Two-streams hypothesis, 205 Visual predator hypothesis, 526

670 SUBJECT INDEXVisual processing streams, 203–5 Vocal tract resonance, 501Visual saccades, 155–56 Voice encoding, 508Visual size, 504–5 Voice recognition, 503Visual specializations, 526–27 Von Economo cells, 434Visual systems, 177–78, 184–90 Von Frisch, Karl, 3 evolutionary influences on, 201–3 von Neumann, John, 162 organization of, 203–15 VPA. See Visual paired associateVisual-visual modalities, 153 vPFC. See Ventrolateral prefrontal cortexVisuo-oculomotor axis, 227Vocal acoustics, 501–2 Wastebasket taxon, 10, 12Vocal communication, 5, 84, 605–7 Weber’s law, 102, 103f, 106, 146, 154, 390Vocalizations Weight distribution, 46–47 amodal completion for, 240f Wernicke’s area, 193, 599 auditory-object analysis, 244 Whales, 10, 13–16 as auditory objects, 237–49 White noise, 239 categorical structure of, 243–44 Win-stay-lose-switch strategy, 554, 556 facial expressions produced with, 502f Wisconsin Card-Sorting Task, 407 pathway for, 244–45 Wisteria floribunda, 433 species-specific, 246 Words, 92, 406Vocal perception, 85–87 Working memory, 212–13Vocal production, 87–88 constrained, 88–90 Xenarthra, 13 fixed, 88–90 flexible, 91–93 Yamagishi, Toshio, 169 involuntary, 88–90 Yarbus, Alfred, 527 of mammals, 88 Y-maze, 410

Primate neuroethology

by Mauricio Madrid on Feb 07, 2011

Accessibility

Upload Details

Usage Rights

1 Embed 1

Statistics

Primate neuroethology — Document Transcript