Reading images the grammar of visual design -- van leeuwen and kress

25,919 views

Published on

4 Comments
30 Likes
Statistics
Notes
  • Hi All, We are planning to start Hadoop online training batch on this week... If any one interested to attend the demo please register in our website... For this batch we are also provide everyday recorded sessions with Materials. For more information feel free to contact us : siva@keylabstraining.com. For Course Content and Recorded Demo Click Here : http://www.keylabstraining.com/hadoop-online-training-hyderabad-bangalore
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Do you have the first edition of this book? Published in 1996
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • interesantísimo. Muchas gracias
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Me gusta y es muy útil
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
25,919
On SlideShare
0
From Embeds
0
Number of Embeds
87
Actions
Shares
0
Downloads
2,071
Comments
4
Likes
30
Embeds 0
No embeds

No notes for slide

Reading images the grammar of visual design -- van leeuwen and kress

  1. 1. r e a d a g e s Thissecondeditionof the landmarktextbookReadingImages buildsonitsreputationastheirst systematicand comprehensiveaccountof thegrammarof visualdesign. Drawingonanenormousrangeof examplesfromchildren,s drawingsto textbookillustrations,photo-journalismto fineart, aswellasthree-dimensionalformssuchassculptureandtoys, theauthorsexaminethewaysinwhichimagescommunicate meantng. Featuresof thisfullyupdatedsecondeditioninclude: . newmaterialonmovingimagesandoncolour . a discussionof howimagesandtheiruseshavechanged throughtime . websitesandweb-basedimages . ideasonthefutureof visualcommunication. ReadingImagesfocusesonthestructuresor'grammar'of visual design- colour,perspective,framingandcomposition- and providesthereaderwithaninvaluable'tool-kit'forreading images,whichmakesit a mustfor anyoneinterestedin communication,themediaandthearts. GuntherKressis Professorof Englishat theInstituteof Education,Universityof London.Theovan Leeuwenhas workedasa film andtelevisionproducerinthe Netherlanosano AustraliaandasProfessorintheCentrefor Language& CommunicationResearchat CardiffUniversity.Heiscurrently Deanat theFacultyof HumanitiesandSocialSciences, Universityof Technology,Sydney.Theyhavebothpublished widelyinthefieldsof languageandcommunicationstudies. i n g i m
  2. 2. P r a i s e f o r t h e r s t e d 'ReadingImagesisthemostimportantbookin visual communicationsinceJacquesBertin'ssemiologyof informationgraphics.It isboththoroughandthought- provoking;a remarkablebreakthrough.' l(evinG.Barnhurst,SyracuseUniversity,USA 'Freshandstimulating.Thesociocentricapproachisbyfar themostpenetratingapproachto thesubjectcurrently available.' PaulCobley,LondonGuildhallUniversity 'A usefultextfor alI studentswhoareinvolvedin areas whichrelyonbothlanguageandvisualimagesfortheir exoressionandarticulationof ideas.' CatrionaScott,MiddlesexUniversity 'Thisisthebestdetailedandsustaineddevelopmentof the "socialsemiotic"approachto theanalysisof visuals.Clear, informativeandtheoreticallydevelopmental.' Dr S.Cottle,BathHE College 'Excellent- wideranging- accessible- tutors'"Bible".' Jan Mair,EdgeHill UniversityCollegeof Higher Education 'Extremelyattractiveandwelllaidout.Veryuseful bibliography.' Dr IVl.Brottman,EastLondonUniversity 'Veryclearlywritten- it makesgoodconnectionsbetween differentareasof visualpractice- especiallyusefulfor studentsfroma varietyof backgroundsattempting "mixed"coursework.' Amy Sargeant,PlymouthUniversity i t i o nf i
  3. 3. readingimage-s G U N T H E R T H E 0 v a n K R E S S a n d L E E U W E N T H E 6 R A M M A R O F V I S U A L D E S I 6 N S E C O N DE D I T I O N Hltiy"'l:s,g:",LONDON AND NEW YORK
  4. 4. Firstoublished1995 byRoutledge 2 ParkSquare,MiltonPark,Abingdon,0xon0X144RN SimultaneouslypublishedintheUSAandCanada byRoutledge 270MadisonAve,NewYork,NY10016 Secondeditionpublished2006 Reprinted2007(twice),2008 Routledgeisan imprint of the Taylor& FrancisGroup,an informabusiness @1996,2006GuntherKressandTheovanLeeuwen Typesetin BellGothicbyRefineCatchLtd,Bungay,Suffolk PrintedandboundinGreatBritainby TJ InternationalLtd,Padstow,Cornwall All rightsreserved.Nopartofthisbookmaybereprintedor reproducedor utilizedinanyformor byanyelectronic,mechanical,or othermeans/now knownorhereafterinvented,includingphotocopyingandrecording,or inany informationstorageor retrievalsystem,withoutpermissionin writingfrom theoublishers. British Library Cataloguingin PublicationData A cataloguerecordforthisbookisavailablefromtheBritishLibrary Library of CongressCatalogingin PublicationData l(ress.GuntherR. Readingimages: thegrammarof visualdesign/ Guntherl(ressandTheo vanLeeuwen.- 2nded. 0.cm. Includesbibliographicalreferencesandindex. 1.Communicationindesign. I. VanLeeuwen,Theo,I94T- II. Title. Nl(1510.K642006 70I-dc22 ISBN10:0-415-31914-5(hbk) ISBN10:0-415-31915-3(pbk) ISBN10:0-203-61972-2 Gbk) ISBN13:978-0415-3L9I4-0 (hbk) ISBN13:97844I5-3L915-7 (pbk) ISBN13:978-0-203-6L972-!(ebk) 2006002242
  5. 5. C O N T E N T S v i i i x x i I P r e f a c et o t h es e c o n de d i t i o n P r e fa c et o t h ef i r s te d i t i o n A c k n o w l e d g e m e n t s I n t r o d u c t i o n :t h eg r a m m a ro f v i s u a l d e s i g n T h es e m i o t i cl a n d s c a p e :l a n g u a g ea n d v i s u a lc o m m u n i c a t i o n N a r r a t i v er e p r e s e n t a t i o n s :d e s ig n i n g s o c i a la c t i o n C o n c e p t u a lr e p r e s e n t a t i o n s :d e s i g n i n g socialconstructs R e p r e s e n t a t i o na n di n t e t a c t i o n : d e s i g n i n gt h e p o s i t i o no f t h ev i e w e r M o d a l i t y :d e s i g n i n gm o d e l so f r e a l i t y T h em e a n i n go f c o m p o s i t i o n M a t e r i a l i t ya n dm e a n i n g T h et h i r dd i m e n s i o n C o l o u r f u lt h o u g h t s( a p o s t s c r i p t ) R e f e r e n c e s I n d e x 1 6 4 5 7 9 t t 4 t54 L75 2t5 239 266 27t 287 4 5 7 B 9
  6. 6. P r e fa c et o t h e s e c o n de d i t i o n Thefirsteditionof ReadingImageshashada positivereceptionamonga widegroupfrom the professionsand disciplineswhichhaveto dealwith real problemsand real issues involvingimages.Thishasgonealongwith a broaderagendaof concernwith'multi- modality',a rapidlygrowingrealizationthatrepresentationisalwaysmultiple.Wedo not thinkfor a momentthatthisbookrepresentsanythinglikea settledapproach,a definitive 'grammar'of images,andat timeswehavebeenworriedbyattemptsto treatit inthatway. Weseeit asanearlyattempt/oneamongmanyothers,andwewouldliketo seeit treated verymuchasa resourcefor beginningto makeinroadsintounderstandingthevisualas representationandcommunication- in a semioticfashion- andalsoasa resourcein the developmentoftheoriesand'grammars'of visualcommunication.In thatspiritwewantto stressthatweseeeverythingwe havewrittenheresimultaneouslyasourfullyseriousand yetentirelyprovisionalsenseof thisfield. Whenwe completedthe first editionof this bookwe wereawareof a numberof 'omissions'-thingswe felt stillneededdoing.Someof thesewe havetakenup in other ways,for instanceinourattemptto developatheoryof multimodality;otherswehavetried to addressin this secondedition.Foremostamongthesehavebeenthe quitedifferent issuesof the movingimageandof colour.Thefirst of thesehasbeenconstantlyraisedby thosewho haveusedthe book,andrightlyso.We hopethat whatwe havesaidherecan beginto integratethe fieldof movingimagesintoour socialsemioticapproachto visual communication.Theissueof colourwaslessfrequentlyraised,yetconstitutedfor usa l<ind of theoreticaltestcase,asmuchto dowiththeissueof colouritselfasto dowitha theory of multimodalsocialsemioticsmuchmorewidelyconsidered.Here,too,wefeelthatwe haveprovidedjust a first attemptfor a differentapproach.In additionwe haveaddeda numberof newexamplesfromCD-R0Msandwebsites,domainsof visualcommunication that hadhardlybegunto developwhenwewrotethefirst edition,andare nowof central importancefor manyusersof thisbook. 0nepersistentcriticismof the first editionfrom a groupof readershasbeenthat the bookwas(too)linguistic.Thefirstcommentwewouldmakeisto saythatforus'formality' inthedomainof representationisnotinanywaythesameas'beinglinguistic'.Soto some extentwethinkthat that criticismrestsonthat kindof misunderstanding.Wealsothink thatthereis a differencebetweenexplicitnessandformality.Wecertainlyhaveaimedfor theformer,andoften(butnotalways)forthelatter.Nordowethinkthateitherexplicitness or formalityarethe enemiesof innovation,creativity,imagination:oftenall theselatter restontheformer.It isthe casethat our startingpointhasbeenthe systemicfunctional grammarof Englishdevelopedby MichaelHalliday,thoughwehadandhaveattemptedto useits generalsemioticaspectsratherthan its specificlinguisticallyfocusedfeaturesas the groundingfor our grammar.As Ferdinandde Saussurehaddoneat the beginningof thelastcentury,weseelinguisticsasa partof semiotics;butwedonotseelinguisticsasthe
  7. 7. vil| Preface to the secondedition disciplinethat can furnisha ready-mademodelfor the descriptionof semioticmodes otherthan language.Thenwe hadthought,in our first attempt,that to showhowvisual communicationworksincomparisonto languagemightbehelpfulin understandingeither and both- but that,too,was misunderstoodmaybeas an attemptto imposelinguistic categoriesonthevisual.Wehavethereforetriedto refineandclarifythosesectionsof the bookthatdealwiththerelationbetweenlanguageandvisualcommunication,andto delete or reformulatematerialwhichwethinkmighthavegivenriseto thesemisunderstandings, hopefullywithno lossof clarity.A carefulreadingof thissecondeditionof ourbookwill show,we trust,that we are as concernedto bringout the differencesbetweenlanguage and visualcommunicationas we are the connections,the broadersemioticprinciples that connect,not just languageand image,but all the multiplemodesin multimodal communication. In our growingunderstandingof this domain,reffectedin the reworkingof this book, we owea debtof gratitudefor support,commentandcritiqueto manymorepeoplethan wecanmentionor eventhanweactuallyknow.Butthenamesof somefriends,colleagues, students,fellowresearchersandcriticswhowerenotalreadyacknowledgedinourpreface to the first editionhaveto be mentioned.Amongtheseare CareyJewitt,Jim Gee,Ron Scollon,Paul Mercer,Brian Street,RadanMartinec,Adam Jaworski,DavidMachin, l(las Prytz,TealTriggs,AndrewBurn,Bob Ferguson,PippaStein,DeniseNewfield,Len Unsworth,LesleyLancasterand the manyresearcherswhosework has both givenus confidenceandnewideas,andextendedour understandingof thisfield- andof course, andcrucially,we acknowledgethe supportfrom our publishersandeditorsat Routledge, LouisaSemlyenandChristabell(irkpatrick.
  8. 8. P r e fa c et o t h e f i r s t e d i t i o n Thisbool<grewoutof discussionsaboutvisualcommunicationwhichspanneda periodof sevenyears.Bothof ushadworkedontheanalysisof verbaltexts,andincreasinglyfeltthe needof a betterunderstandingof all thethingsthat 9owiththeverbal:facialexpressions, gestures,images,music,andso on.Thiswasnot onlybecausewe wantedto analysethe wholeofthetextsinwhichthesesemioticmodesplaya vitalroleratherthanjusttheverbal part,but alsoto understandlanguagebetter.Justasa knowledgeof otherlanguagescan 0pennewperspectivesonone'sownlanguage,soa knowledgeof othersemioticmodescan 0pennewperspectiveson language. In 1990we publisheda firstversionof our ideason visualcommunication,Reading Images,with DeakinUniversityPress.It waswrittenfor teachers,andweconcentratedon children'sdrawingsandschooltextbookillustrations,althoughwealsoincludedexamples from the massmedia,suchas advertisementsandmagazinelayout.Sincethenwe have expandedour researchto otherfieldsof visualcommunication:a muchwiderrangeof massmediamaterials;scientific(andother)diagrams,mapsandcharts;andthe visual arts.Wehavealsomadea beginningwiththestudyof three-dimensionalcommunication: sculpture,children'stoys,architectureandeverydaydesignedobjects.Thepresentbook thereforeoffersa muchmorecomprehensivetheoryof visualcommunicationthan the earlierbook. In Australia,andincreasinglyelsewhere,ourworkhasbeenusedin courseson com- municationandmediastudies,andasa methodologyfor researchin areassuchasmedia representation,film studies,children'sliteratureandthe useof illustrationsandlayoutin schooltextbooks.Thepresentbookhasbenefitedgreatlyfrom the suggestionsandcom- mentsof thosewhohaveusedourwork intheseways,andof our ownundergraduateand postgraduatestudents,initiallyat the Universityof TechnologyandMacquarieUniversity in Sydney,laterat the Instituteof Educationandthe LondonCollegeof Printingin London,andalsoat theTemasekPolytechnicin Singapore. We beganour work on visualcommunicationin the supportiveand stimulating environmentof the NewtownSemioticsCirclein Sydney;discussionswithourfriends,the membersof thisCircle,helpedshapeour ideasin morewaysthanwe canacknowledge. If anytwo peoplefrom that first periodwereto besingledout,it wouldbeJim Martin, who gaveusmeticulous,detailed,extensiveandchallengingcommentson severalof the chaptersof theearlierbook,andFranChristie,whohadurgedusto write it. But herewe wouldalsoliketo makea specialmentionof BobHodge,whoseideasappearinthisbookin manyways/evenif notalwaysobviouslyso. 0f thosewho usedour bookin teachingand research,andwhosecommentson the earlierbookhavehelpedus rethinkand refineour ideas,we wouldlike to mention theresearchteamof the DisadvantagedSchoolsProgrammein Sydney,in particularRick Iedema,SusanFeez,PeterWhite,RobertVeeland Sally Humphrey;StaffanSelander,
  9. 9. x . Preface to the first edition throughwhoseCentrefor TextbookResearchin Hdrnosandourworkcameto betakenup by researchersin the fieldof textbookresearchin SwedenandseveralotherEuropean countries;the membersof the 'Languageand Science'researchteam at the Institute of Education,IsabelMartins,Jon 0gbornand l(ieranMcGillicuddy;PhilipBell;Basil Bernstein;PaulGillenandTeunvanDijk. Threewritersinffuencedour ideasin differentandfundamentalways.Oneis Roland Barthes.Althoughweseeourworkasgoingbeyondhisseminalwritingonvisualsemiotics in severalways,he remainsa stronginspiration.Thereis not a subjectin semioticson whichBartheshasnotwrittenoriginallyandinspiringly.Hehasprovidedfor usa model of what semioticscanbe,in the rangeof his interests,in the depthof hiswork,and in hisengagementwiththe socialandculturalworld.Equallysignificantfor us is Michael Halliday.His viewof languageas a socialsemiotic,andthe widerimplicationsof his theories,gaveusthemeansto gobeyondthestructuralistapproachof 1960sParisSchool semiotics,and our work is everywhereinffuencedby his ideas.Thenthereis Rudolf Arnheim.Themorewe readhiswork,the morewe realizethat mostof whatwe haveto sayhasalreadybeensaidby him,oftenbetterthanwe havedoneit, albeitit usuallyin commentarieson individualworksof art ratherthanintheformof a moregeneraltheory. HeiscommonlyassociatedwithGestaltpsychology:wewouldliketo claimhimasa great socialsemiotician. Wewouldliketo thanoureditor,JuliaHall,for herencouragementandinvaluablehelp in producingthisbook.Jill Brewsterand LauraLopez-Bonillawereinvolvedin various stagesofthebook;theirencouragementandhelpmadetheworkpossibleandenjoyable.
  10. 10. A c l t n o w l e d g e m e n t s Plate3 JoshuaSmithbyWilliamDobell,1943,@DACS2004. Plate5 CossacksbyVassilyl(andinsl<y,1910-1911.@ADAGP,ParisandDACS,London 2004.Photography@Tate,London2005. Plate6 HistoricColoursby ColinPoole,reproducedby kindpermissionof PhotoWord SyndicationLtd. Plate7 Palgravecolourschemereproducedwithpermissionof PalgraveMacmillan. 1.1+1.13'My bath' from Baby'sFirst Book by B. Lewis,illustratedby H. Wooley, copyright@ LadybirdBooks1td.,1950. L2 Bird in treefrom 0n My Wallgby Dick Bruna,1988.IllustrationDick Bruna@ Mercisb41972. 1.4 Magazinecoverwith naturalisticphotograph,coverof Newsweek,April9,2004 @ 2004 Newsweek,Inc. Photographby l(arim Sahib-AFP-GettyImages.Reprintedby permission. 1.5 Magazinecoverwithconceptualphotograph,coverof Newsweek,November12,200I O 2001Newsweek,Inc.Reprintedbypermission. 1.6 Imagefrom'lnteractivePhysics'.Courtesyof MSCSoftware. 2.5 CommunicationmodelfromWatson,J. andHill,A.(1980)A Dictionaryof Communi- cationandMediaStudiegLondon,Arnold,p.I43. Reproducedby permissionof Hodder HeadlinePLC. 2.6 TwoCommunicationModelsfrom Watson,J. and Hill,A. (1980)A Dictionaryof CommunicationandMediaStudieqLondon,Arnoldp.I47. Reproducedby permissionof HodderHeadlinePLC. 2.IO Beatthe WhiteswiththeRedWedgeby El Lissitzky,I9I9-20.@ DACS2004. 2.11 l(asimirMalevich(1878-1935)"SuprematistComposition:RedSquareandBlack Square",1914,NewYork,Museumof ModernArt (MolVlA)@ 2004,Digitalimage,The Museumof ModernArt, NewYorl</Scala,Florence. 2.I7 Gulf War Diagram,SydneyMorningHerald,14 February,1991 reproducedby permissionof SydneyMorningHerald. 2.18 Speechcircuitfrom Saussure'sCoursein GeneralLinguistics,I9T4,F.deSaussure, translatedby RoyHarrisbypermissionof GeraldDuckworth& Co.
  11. 11. xii . Acknowledgements 2.20 Vitteladvertisementsreproducedbykindpermissionof Nestl6Group. 2.22 CommunicationModel from Watson,J. and Hill, A. (1980) A Dictionaryof CommunicationandMediaStudieELondon,Arnoldp.I47. Reproducedby permissionof HodderHeadlinePLC. 2.23 Arctictundrasystem/fi1.7.5,p.1,72fromSale,C.,Friedman,B.andWilson, G.)ur ChangingWorld,Book1, PearsonEducationAustralia.Reproducedby permissionof the publisher. 2.24 CommunicationModelfrom Watson,J. and Hill,A. (1980) A Dictionaryof Com- municationandMediaStudies,London,Arnoldp.54.Reproducedbypermissionof Hodder HeadlinePLC. 3.1 Guideinterfacefrom'DangerousCreatures',7994.Screenshotreprintedby permis- sionfromMicrosoftCorporation. 3.2 Sekondaadvertisementreproducedbykindpermissionof Sekonda/TimeProducts. 3.3 Sourcesof signsfrom Eco,U. (1976) A theoryof semiotics,Bloomington,Indiana UniversityPress,p.177.Reproducedbypermissionof thepublisher. 3.5 Semanticfielddiagramfrom Eco,U. (I976) A theoryof semiotics,Bloomington, IndianaUniversityPress,p.78.Reproducedbypermissionof thepublisher. 3.7 NetworkfromSharples,M.andPemberton,'Representingwriting:externalrepresen- tationsandthe writingprocess'inN.Williamsand P.Holt,edsComputersand Writing. Reproducedbykindpermissionof IntellectLtd,www.intellectbooks.com 3.13 Resortwear,AustralianWomen'sWeekly,December1987.@ AustralianWomen's Weekly/ACPSyndication.Reproducedwith permission. 3.19 Electricalcircuitdiagramfrom J. Hill,1980,IntroductoryPhysics.Reproducedby permissionof Taylor& FrancisGroup. 3.20 Theplaceof linguisticson the mapof knowledgefrom Halliday,M.A.l(.(1978) Languageas SocialSemiotic,London,Arnold.Reproducedby permissionof Hodder HeadlinePLC. 3.21 'Womenat work',Fig.III-6, p.29,fromPictographsandGraphs:Howto Makeand UseThemby RudolfModleyandDynoLowenstein.@ 1952by Harper& Brothers.Copy- rightrenewed1980by PeterM. ModleyandMarionE.Schilling.Reprintedbypermission of HarperCollinsPublishersInc. 3.28 'Funwithfungi',SydneyMorningHerald,18 June1992,reproducedby permission of SydneyMorning Herald. 4.2 ATMscreenreproducedbykindpermissionof NationalAustraliaBank. 4.4 The murderof Dr Chang,SydneyMorning Herald,5 July 1991, reproducedby permissionof SydneyMorningHerald.
  12. 12. Acknowledgements x i i i 4.5 Playstationwebsitereproducedbykindpermissionof SonyComputerEntertainment EuropeLtd. 4.7 NewlookFordMondeofromwww.ford.co.ukreproducedbyl<indpermissionof Ford. 4.8 Fiesta'Rocksolid'websitereproducedby kindpermissionof 0gilvyGroupHoldings LtdandFord. 4.16 'PrisonGuard'byDannyLyon,1969,fromConversationswith thedead.@ Danny Lyon.MagnumPhotos.Reproducedwithpermission. 4.19 Gulfwar map,SydneyMorningHerald,22January1991,reproducedbypermission of SydneyMorning Herald. 4.20 An increasein tourism,SydneyMorningHerald,22January1991,reproducedby permissionof SydneyMorningHerald. 4.22 DeIail from a fourteenth-centurySpanishnativity,from Rudolf Arnheim,Art and Visual Perception:A Psychologyof the Creative Eye. The New Version.@ I974 TheRegentsof the Universityof California.Reproducedby permissionof Universityof CaliforniaPress. 5.1 Speechcircuitfrom Saussure'sCoursein GeneralLinguistics,7974,F.de Saussure, translatedby RoyHarrisbypermissionof GeraldDuckworth& Co. 5.2 Schematizedspeechcircuitfrom Saussure'sCoursein GeneralLinguistics,1974, F.deSaussure,translatedby RoyHarrisbypermissionof GeraldDuckworth& Co. 5.6 Card-players(Van Doesburg,1916-17) photograph/picture:Tim l(oster,ICN, Rijswijk/Amsterdam.Reproducedwithpermissionof InstitutCollectieNederland. 5.7 Composition9 (YanDoesburg,I9I7). Collectionof theGemeentemuseumDenHaag. Reproducedwithpermission. 5.8 ColourProjectfortheSchroderResidence(GerritRietveld,1923-4)@ DACS2005. 5.9 PhotographoftheSchroderResidence,GerritRietveld,@ DACS2004. 5.11 @OxfordUniversityPressfrom TheYoungGeographerInvestigates:Mountainsby TerryJennings(0UP,1986),reprintedbypermissionof 0xfordUniversityPress. 5.12 Drawingby Newton.Bypermissionof theWardenandFellows,NewCollege,Oxford, andTheBodleianLibrary,Universityof 0xford.MS 361,vol.2,fol.45V. 5.13 Drawingof Stretton'sexperiment,figure8.1 (p.141)from TheEyeand Brain: Psychologyof Seeing5/eby RichardGregory,1998,reproducedby permissionof 0xford UniversityPress.Gregory,Richard,EyeandBrain.Reprintedby permissionof Princeton UniversityPress. 6.2 Gold-diggers,AustralianWomen'sWeekly,November1987.@ AustralianWomen's WeeklyiACPSyndication.Reproducedwithpermission.
  13. 13. xiv . Acknowledgements 6.3 SonylVliddleEastwebsitereproducedbykindpermissionof SonyGulfFZE. 6.9 Gerbner'scommunicationmodelfromWatson,J.andHill,A.(1980)ADictionaryof CommunicationandMediaStudies,London,Arnold.Reproducedbypermissionof Hodder HeadlinePLC. 6.10 Royalcouple.H.M. The Queen'swedding,photographby Baron,CameraPress, London.H.M.TheQueenandPrincePhilip,photographby H.R.H.PrinceAndrew,Camera Press,London.Reproducedwithpermission. 6.12 Buddhistpaintingfrom RudolfArnheim,Art and VisualPerception:A Psychology of theCreativeEye.TheNewVersion.@1974TheRegentsofthe Universityof California. Reproducedbypermissionof Universityof CaliforniaPress. 6.13 'Goingon holiday'from Prosser,R. (2000) Leisure,Recreationand Tourism, London,CollinsEducational.Reprintedbypermissionof HarperCollinsPublishersLtd@ R.Prosser,2000. 6.14 Anderschet al.'s communicationmodelfrom Watson,J. and Hill, A. (1980) A Dictionaryof Communicationand Media Studies,London,Arnold. Reproducedby permissionof HodderHeadlinePLC. 6.16 Verticaltriptychfromthewebsiteof 0xford Universityreproducedby kindpermis- sionof 0xfordUniversity. 6.23 Screenshotfrom CD-R0M'3D BodyAdventure',l(nowledgeAdventure,1993 providedcourtesyof l(nowledgeAdventure,Inc. 7.1 Roy Lichtenstein,Big Painting,1965.@ The Estateof Roy Lichtenstein/DACS 2004. 8.I Jacoband the Angel Qacob Epstein,1940) O The Estateof Jacob Epstein/Tate, London2005.Imagesuppliedbyandreproducedbypermissionof GranadaTV. 8.2 Peoplein the Wind(l(ennethArmitage,1952). Reproductioncourtesyof l(enneth ArmitageEstate.Photography@Tate,London2005. 8.5 WomanbyJoanMir6,I97O@ SuccessioMiro,DACS2004. 8.6 LesHeuresdesTraces(HouroftheTracelbyAlbertoGiacometti,1930@ADAGP, ParisandDACS,London2004.Photography@Tate,London2005. 8.7 Jacoband the AngelQacobEpstein,1940) O The Estateof JacobEpstein/Tate, London2005.Imagesuppliedbyandreproducedbypermissionof GranadaTV. 8.8 Playmobil'familyset'and'ethnicfamily'fromPlaymobilcatalogue.Reproducedby kindpermissionof PlaymobilUl<Ltd. 8.9 RecumbentFigureby HenryMoore,1938.Illustratedonp.250;hasbeenreproduced bypermissionofthe HenryMooreFoundation.Photography@Tate,London2005.
  14. 14. Acknowledgements . xu B.I0 l(ingandQueenby HenrylVloore,1952-3.lllustratedon p.253; hasbeenrepro- ducedbypermissionofthe HenryMooreFoundation.Photography@Tate,London2005. 8.11 Churchof SantaMaria DellaSpinafrom RudolfArnheim,Art and VisualPercep- tion: A Psychologyof the CreativeEye. TheNew Version.@ 7974 The Regentsof the Universityof California.Reproducedbypermissionof Universityof CaliforniaPress. 8.12 Connectedanddisconnectednarrativeprocess/pp.83-84 from S. Goodmanand D. Graddol,RedesigningEnglish- new texts,new identities,London,Routledge,1997. Reproducedbypermissionof thepublisher. 8.13 Overshouldershotincomputergame,DeltaForce.Imagecourtesyof NovaLogicInc. @ 2004.All rightsreserved. 8.14 Dynamicinterpersonalrelationsin the openingsceneol TheBig Sleep(Howard Hawks,1947),pp.9I-92 from S.Goodmanand D.Graddol,RedesigningEnglish- new texts,newidentities,London,Routledge,l997. Reproducedbypermissionofthe publisher.
  15. 15. I n t r o d u c t i o n :t h e g r a m m a ro f v i s u a l d e s i g n Thesubtitleof thisbookis'the grammarof visualdesign'.We hesitatedoverthistitle. Extensionsof the term'grammar'oftensuggest'rules'.In bookswith titleslike The Grammarof TelevisionProductiononelearns,for instance,aboutthe rulesof continuity; l<nowingtheserulesisthenwhatsetsthe'professional'apartfromthe'amateur'.Whatwe wishto expressis a littledifferent.In our view,mostaccountsof visualsemioticshave concentratedonwhatmightberegardedastheequivalentof 'words'-whatlinguistscall 'lexis'- ratherthan'grammar',andthenonthe'denotative'and'connotative',the'icono- graphical'and'iconological'significanceof theelementsin images,theindividualpeople, placesandthings(includingabstract'things')depictedthere.In thisbook,bycontrast,we will concentrateon 'grammar'andon syntax,on the way in whichtheseelementsare combinedinto meaningfulwholes.Justas grammarsof languagedescribehowwords Combinein Clauses,sentenceSandtexts,soourvisual'grammar'willdesCribethewayin whichdepictedelements- people,placesandthings- combinein visual'statements'of greateror lessercomplexityandextension. Weareby no meansthefirstto dealwiththissubject.Nevertheless,bycomparisonto the studyof visual'lexis',the studyof visual'grammar'hasbeenrelativelyneglected,or dealtwith from a differentperspective,from the pointof viewof art history,or of the formal,aestheticdescriptionof composition,or the psychologyof perception,or with a focusonmorepragmaticmatters,for instancethewaycompositioncanbeusedto attract theviewer'sattentionto onethingratherthananother,e.g.insuchappliedenvironmentsas advertisingor packaging.All theseare validapproaches,and in manyplacesandmany wayswehavemadeuseof the insightsof peoplewritingfromthesedifferentperspectives. Yetthe resulthasbeenthat,despitethe verylargeamountof work doneon images,not muchattentionhasbeenpaidto themeaningsof regularitiesinthewayimageelementsare used- in short,to their grammar- at leastnot in explicitor systematicways.It is this focusonmeaningthatweseek,aboveall,to describeandcaptureinourbook.Weintendto provideusabledescriptionsof majorcompositionalstructureswhichhavebecomeestab- lishedas conventionsin the courseof the historyof Westernvisualsemiotics,andto analysehowtheyareusedto producemeaningbycontemporaryimage-makers. Whatwehavesaidaboutvisual'grammar'istruealsoof themainstreamof linguistic grammar:grammarhasbeen,and remains,'formal'.It hasgenerallybeenstudiedin isolationfrom meaning.Howeve[the linguistsandthe schoolof linguisticthoughtfrom whichwedrawpartofourinspiration-linguistsfollowingtheworkofMichaelHalliday- havetakenissuewith this view,and seegrammaticalformsas resourcesfor encoding interpretationsof experienceand formsof social(inter)action.BenjaminLeeWhorf arguedthepointinrelationto languagesfromdifferentcultures.In whathecalled'Stand- ard AverageEuropean'languages,termslike'summer','winter','September','morning', 'noon','sunset'arecodedas nouns/asthoughtheywerethings.Hencetheselanguages
  16. 16. Introduction makeit possibleto interprettime as somethingyoucancount/use/save/etc.In Hopi,a NorthAmericanIndianlanguage,this is not possible.Timecan onlybe expressedas 'subjectiveduration-feeling'.Youcannotsay'atnoon',or'threesummers,.youhaveto sav somethinglike'whilethesummerphaseisoccurring,(Whorf,1950. Thecriticallinguistsof the EastAngliaSchool,withwhomoneof uswasconnected, haveshownthatsuchdifferentinterpretationsof experiencecanalsobeencodedusingthe resourcesof thesamelanguage,onthe basisof differentideologicalpositions.TonyTrew (1979:106-7) hasdescribedhow,whenthe Hararepolice- in whatwasin 1975still Rhodesia- firedintoa crowdof unarmedpeopleandshotthirteenof them,the Rhodesia Heraldwrote,'A politicalclashhasledto deathandiniury,,whilethe TanzanianDaily Newswrofe,'Rhodesia'swhitesuprematistpolice... openedfire and killedthirteen unarmedAfricans.'In otherwords,thepoliticalviewsof newspapersarenotonlyencoded throughdifferentvocabularies(of the well-known'terrorist'vs'freedomfighter,type), but alsothroughdifferentgrammaticalstructures;that is,throughthe choicebetween codingan eventasa noun('death','injury')or a verb('kill'),whichfor itsgrammatical completionrequiresanactivesubject('police')andanobject(,unarmedAfricans,). Grammargoesbeyondformal rulesof correctness.It is a meansof representing patternsof experience....ltenableshumanbeingsto builda mentalpictureof reality,to makesenseof theirexperienceof what goeson aroundthemandinside them. (Halliday,1985:t0t) Thesameis true for the'grammarof visualdesign'.Like linguisticstructures,visual structurespointto particularinterpretationsof experienceandformsof socialinteraction. To somedegreethesecanalsobe expressedlinguistically.Meaningsbelongto culture, ratherthanto specificsemioticmodes.Andthewaymeaningsaremappedacrossdifferent semioticmodes,thewaysomethingscan,forinstance,be'said'eithervisuallyor verbally, othersonlyvisually,againothersonlyverbally,isalsoculturallyandhistoricallyspecific.In the courseof this bookwe will constantlyelaborateandexemplifythis point.But even whenwecanexpresswhatseemto bethesamemeaningsin eitherimage-formor writing or speech,theywill be realizeddifferently.For instance,what is expressedin language throughthe choicebetweendifferentword classesandclausestructures,may,in visual communication,beexpressedthroughthechoicebetweendifferentusesof colouror differ- entcompositionalstructures.Andthiswillaffectmeaning.Expressingsomethingverbally or visuallymakesa difference. Asfor otherresonancesof theterm'grammar'(grammar,asa setof rulesonehasto obeyif one is to speakor write in'correct',sociallyacceptableways),linguistsoften protestthat theyare merelydescribingwhat peopledo,andthat othersinsiston turning descriptionsintorules.Butof courseto describeisto beinvolvedin producingknowledge whichotherswill transformfromthedescriptiveintothenormative,for instancein educa- tion.Whena semioticmodeplaysa dominantrolein publiccommunication,its usewill inevitablybe constrainedby rules,rulesenforcedthrougheducation,for instance,and
  17. 17. Introduction throughall kindsof writtenandunwrittensocialsanctions.0nlya smalleliteof experi- mentersisallowedto breaktherules- afterall,breakingrulesremainsnecessaryto keep openthepossibilityof change.Webelievethatvisualcommunicationiscomingto beless andlessthe domainof specialists,andmoreandmorecrucialin the domainsof public communication.Inevitablythiswill leadto new,and morerules,andto moreformal/ normativeteaching.Not being'visuallyIiterate'willbeginto attractsocialsanctions. 'Visualliteracy'willbeginto bea matterof survival,especiallyintheworkplace. We arewellawarethat work suchasourscanor will helppavethewayfor develop- mentsof thiskind.Thiscanbeseennegatively,asconstrainingthe relativefreedomwhich visualcommunicationhassofar enjoyed,albeitat theexpenseof a certainmarginalization bycomparisonto writing;or positively,asallowingmorepeoplegreateraccessto a wider rangeof visualskills.Nordoesit haveto standinthewayof reativity.Teachingtherulesof writinghasnotmeanttheendof creativeusesof languagein literatureandelsewhere,and teachingvisualskillswill notspelltheendof thearts.Yet,justasthegrammarcreatively employedby poetsandnovelistsis,in the end,the samegrammarwe usewhenwriting letters,memosand reports,so the'grammarof visualdesign'creativelyemployedby artistsis,intheend,thesamegrammarweneedwhenproducingattractivelayouts,images anddiagramsfor ourcoursehandouts,reports,brochures,communiqu6s,andsoon' It isworth askingherewhata linguisticgrammaris a grammarof.fhe conventional answeristo saythatit is agrammarof 'English'or'Dutch'or'French'-therulesthat defineEnglishas'English',Dutchas'Dutch',andsoon.A slightlylessconventionalanswer wouldbeto saythat agrammarisan inventoryof elementsandrulesunderlyingculture- specificformsof verbalcommunication.'Underlying'hereis a shorthandtermfor some- thingmorediffuseandcomplex,morelike'knowledgesharedmoreor lessbymembersof a group,explicitlyandimplicitly'.Thisbringsinsubtlemattersofwhatl<nowledgeisandhow it isheldandexpressed,andaboveallthesocialquestionof whata'group'is.Thatmakes definitionsof grammarverymucha socialquestion,oneof the knowledgesandpractices sharedbygroupsof people. We mightnowask,'What is our "visual grammar"agraffimarof?'First of all we wouldsaythatit describesa socialresourceof a particular9roup,itsexplicitandimplicit knowledgeaboutthisresource/anditsusesinthepracticesof thatgroup.Then,second,we wouldsaythat it isa quitegeneralgrammartbecauseweneeda termthat canencompass oil paintingaswellasmagazinelayout,thecomicstripaswellasthescientificdiagram. Drawingthesetwo pointstogether,andbearingin mindour socialdefinitionof grammar, we wouldsaythat'our'grammaris a quitegeneralgYammarof contemporaryvisual designin'Western'cultures,an accountof theexplicitandimplicitl<nowledgeandprac- ticesarounda resource,consistingof theelementsandrulesunderlyinga culture-specific formof visualcommunication.Wehavequitedeliberatelymadeourdefinitiona socialone, beginningwiththequestion'Whatisthegroup?Whatareitspractices?'andfromthere attemptingto describethegrammarat issue,ratherthanadoptinganapproachwhichsays, 'Hereisourgyammar;dothepracticesandl<nowledgesofthisgroupconformtoitornot?' In the bookwe have,by and large,confinedour examplesto visualtext-objectsfrom ,Western'culturesandassumedthatthisgeneralizationhassomevalidityasit pointsto a
  18. 18. Introduction communicationalsituationwitha longhistorythathasevolvedoverthepastfivecenturies or so/alongsidewriting(quitedespitethedifferencesbetweenEuropeanlanguages),asa 'languageof visualdesign'.Its boundariesare notthoseof nation-states,althoughthere are,andverymuchso,cultural/regionalvariations.Rather/thisvisualresourcehasspread, alwaysinteractingwiththespecificitiesof locality,whereverglobalWesterncultureisthe dominantculture. Thismeans,first of all,that jt is not a'universal'grammar.Visuallanguageis not - despiteassumptionsto thecontrary-transparentanduniversallyunderstood;it iscultur- allyspecific.We hopeourwork will continueto providesomeideasandconceptsfor the studyof visualcommunicationin non-Westernformsof visualcommunication.Togivethe mostobviousexample,Westernvisualcommunicationisdeeplyaffectedbyourconvention of writingfrom leftto right(in chapter6 we will discussthismorefully).Thewriting directionsof culturesvary:from rightto leftor from leftto right,fromtopto bottomor in circularfashionfromthecentreto theoutside.Consequentlydifferentvaluesandmeanings areattachedto suchkeydimensionsof visualspace.Thesevaluationsandmeaningsexert their inffuencebeyondwriting,and informthe meaningsaccordedto differentcom- positionalpatterns,theamountof usemadeof them,andsoon.In otherwords,weassume thatthe elements,suchas'centre,or,margin,,'top,or'bottom,,will playa rolein the visualsemioticsof any culture,but with meaningsand valuesthat are likelyto differ dependingonthatculture'shistoriesof useofvisualspace,writingincluded.The'universal, aspectof meaningliesin semioticprinciplesandprocesses,theculture-specificaspectlies in theirapplicationoverhistory,andin specificinstancesof use.Herewe merelywantto signalthat our investigationshavebeenrestricted,by andlarge,to Westernvisualcom- munication.Eventhoughothershavebegunto extendtheapplicationsof theprinciplesof thisgrammar,wemakenospecificclaimsfor theapplicationof our ideasto othercultures. WithinWesternvisualdesign,however,we believethat our theoryappliesto all formsof visualcommunication.Wehopethatthewiderangeof exampleswe usein the bookwill convincereadersof thisproposition. 0urstressontheunityof Westernvisualcommunicationdoesnotexcludethepossibility of regionalandsocialvariation.Theunityof Westerndesignisnotsomeintrinsicfeatureof visuality,butderivesfroma longhistoryof culturalconnectionandinterchange,aswellas nowfrom the globalpowerof the Westernmassmediaandcultureindustriesandtheir technologies.In manypartsof theworld,Westernvisualcommunicationexistssidebyside with localforms.Westernformsmightbeused,for instance,in certaindomainsof public communication,suchaspublicnotices,sitesof publictransport,thepress,advertising,and thevisualarts,aswellasinsomewhatmore'private,domains,inthehome,andin markets andshops,forlnstance.Oftentherelationishierarchical,withoneformoverlaidonanother (seeScollonandScollon,2003;l(ress,2oo3),andoften- asinadvertising,for instance- thetwo aremutuallytransformedandfused.WhereWesternvisualcommunicationbegins t0 exertpressureonlocalforms,therearetransitionalstagesinwhichtheformsof thetwo culturesmix in particularways.In lookingat advertisementsin English-languagemaga- zinesfrom the Philippines,for instance,we werestruckby the way in whichentirely conventionalWesterniconographicalelementswereintegratedintodesignsfollowingthe
  19. 19. Introduction rulesof a localvisualsemiotic.In advertisementsontheIVITRin HongKong,someadvert- isementsconformto the'Eastern'directionality,othersto theWestern,yetothersmixthe two.As with the Filipinoadvertisements,discoursesand iconographycan be 'Western', mixedinvariouswayswiththoseof the'East',whilecolourschemescan/at thesametime, bedistinctlynon-Western.Thesituationthereis in anycasecomplicated(asit is,differ- ently,in Japan)bythe fact that directionalityin the writingsystemhasbecomecompli- catedin severalways:bytheadoption,in certaincontexts,of 'Western'directionalityand the Romanalphabetalongsidethecontinueduseof the moretraditionaldirectionalities andformsof writing.Andaseconomic(andnowoftencultural)poweris re-weighted,the trendcango in bothor moredirections:the influenceof Asianformsof visualdesignis becomingmoreandmorepresentinthe'West'.Superimposedonall thisaretheincreas- inglyprominentdiasporiccommunities- of Greeks,Lebanese,Turks,of manygroupsof the Indiansubcontinent,of new and olderChinesecommunities(for instance,Hong l(ongChinesearoundthe PacificRim)- whichseeminglyaffectonlythe membersof this diaspora,andyetin realityarehavingdeepinfluenceswellbeyondthem. WithinEurope,increasingregionalitycounterbalancesincreasingglobalization.Solong asthe Europeannationsandregionsstill retaindifferentwaysof lifeanda differentethos, theywill usethe'grammarof visualdesign'distinctly.It is easy,for example,to find examplesof the contrastinguseof the left and right in the compositionof pagesand imagesinthe Britishmedia.It isharderto findsuchexamplesin,forinstance,theGreekor the Spanishor the Italianmedia.as studentsfrom thesecountrieshaveassuredusand demonstratedin theirworl<- aftertryingto dotheassignmentswe hadsetthemat home duringtheirholidays.in the courseof our bookwe will givesomeexamplesof this,for instanceinconnectionwithnewspaperlayoutindifferentEuropeancountries.However,we arenotableto domorethantouchonthesubject;andtheissueof different'dialects'and 'inffections'needsto beexploredmorefullyinthefuture. In anycase,the unityof languagesis a socialconstruct,a productof theoryandof socialandculturalhistories.Whenthebordersof (a) languagearenotpolicedbyacad- emies,andwhenlanguagesare not homogenizedby educationsystemsandmassmedia, peoplequitefreelycombineelementsfromthe languagestheyknowto makethemselves understood.Mixedlanguages('pidgins')developin thisway,andin timecanbecomethe languageof new generations('creoles').Visualcommunication,not subjectto such policing,has developedmorefreelythan language,but therehas neverthelessbeena dominantlanguage,'spoken'anddevelopedincentresof highculture,alongsidelesshighly valuedregionalandsocialvariants(e.9.'folkart').Thedominantvisuallanguageis now controlledbytheglobalcultural/technologicalempiresofthemassmedia,whichdissemin- atetheexamplessetbyexemplarydesignersand,Ihroughthe spreadof imagebanksand computer-imagingtechnology,exerta 'normalizing'ratherthan explicitly'normative' inffuenceonvisualcommunicationacrosstheworld.Muchasit istheprimaryaimof this bookto describethecurrentstateof the'grammarof visualdesign',wewill alsodiscuss the broadhistorical,socialand culturalconditionsthat makeand remakethe visual 'language'.
  20. 20. Introduction A SOCIALSEMIOTICTHEORYOFREPRESENTATION 0urworkonvisualrepresentationissetwithinthetheoreticalframeworkof 'socialsemiot- ics'.It isimportantthereforeto placeit inthecontextof theway'semiotics'hasdeveloped during,roughly,thepast75 years.In Europe,threeschoolsof semioticsappliedideasfrom the domainof linguisticsto non-linguisticmodesof communication.Thefirst wasthe PragueSchoolofthe1930sandearly1940s.It developedtheworkof RussianFormalists by providingit with a linguisticbasis.Notionssuchas'foregrounding,wereappliedto language(e.9.the'foregrounding',forartisticpurposes/of phonologicalor syntacticforms through'deviation'fromstandardforms,for artisticpurposes)aswellasto thestudyof art (Mukarovsky),theatre(Honzl),cinema(Jakobson)andcostume(Bogatyrev).Eachof thesesemioticsystemscouldfulfilthesamecommunicativefunctions(the'referential,and the'poetic'functions).Thesecondwasthe parisSchoolof the 1960sand1970s,which appliedideasfrom de saussureandotherIinguiststo painting(Schefer),photography (Barthes,Lindel<ens),fashion(Barthes),cinema(Metz),music(Nattiez),comicstrips (Fresnault-Deruelle),etc.TheideasdevelopedbythisSchoolarestilltaughtin countless coursesof mediastudies,art anddesign,etc.,oftenundertheheading,semiology,,despite the fact that they are at the sametime regardedas havingbeenovertakenby post- structuralism.Everywherestudentsarelearningabout'langue'and,parole,;the'signifier, andthe'signified',''arbitrary'and,motivated,signs;,icons,,,indexes,and'symbols,(these termscomefromtheworkof theAmericanphilosopherandsemioticianCharlesSanders Peirce,but areoftenincorporatedin theframeworkof 'semiology,),andsoon.Generally thishappenswithoutstudentsbeinggivena senseof,or accessto,alternativetheoriesof semiotics(orof linguistics).Wewill compareandcontrastthiskindof semioticswithour ownapproach,inthisintroductionaswellaselsewhereinthebook.Thisthird,stillffedg- ling,movementin whichinsightsfrom linguisticshavebeenappliedto othermodesof representationhastwo sources/bothdrawingonthe ideasof MichaelHalliday,onegrow- ing out of the'critical Linguistics'ofa groupof peopleworkingin the 1970sat the Universityof EastAnglia,leadingto theoutlineof a theorythatmightencompassother semioticmodes(Hodgeand l(ress),the other,in the later 1990s,as a developmentof Hallidayansystemic-functionallinguisticsbya numberof scholarsinAustralia,insemiot- icallyorientedstudiesof literature(Threadgold,Thibault),visualsemiotics(0,Toole, ourselves)andmusic(vanLeeuwen). Thekeynotionin anysemioticsisthe'sign'.0ur bookisaboutsigns- or,aswewouro ratherput it,aboutsign-making.Wewill bediscussingforms('signifiers')suchascolour, perspectiveandline,aswellasthewayinwhichtheseformsareusedto realizemeanings ('signifieds')inthemakingof signs.Butourconceptionof thesigndifferssomewhatfrom that of 'semiology',andwewishthereforeto comparethetwo viewsexplicitly.In doingso we usethe term 'semiology'to referto the way in whichthe ParisSchoolsemioticsis generallytaughtintheAnglo-Saxonworld,throughthemediationof inffuentialtextbooks suchasthe seriesof mediastudiestextbookseditedby JohnFiske(Fiskeand Hartley, r979; Dyer,r9B2;Fisl<e,r9B2;Hariley,19B2;0'sullivaneta.,1993).In doingthiswedo not seekto repudiatethosewhowentbeforeus.We seea continuitybetweentheirwork
  21. 21. Introduction andours,asshouldbeclearfrom ourmaintitle,ReadingImages,whichechoesthatof the firstvolumein Fiske'sseries,ReadingTelevision(FiskeandHartley,I979). Wewouldliketo beginwithanexampleof whatwe understandby'sign-making'.The drawingin figure0.1 was madeby a three-year-oldboy.Sittingon hisfather'slap,he talkedaboutthedrawingashewasdoingit:'Doyouwantto watchme?I'll makeacar . . . g o t t w o w h e e l s . . . a n d t w o w h e e l s a t t h e b a c k . . . a n d t w o w h e e l s h e r e . . . t h a t ' s a f u n n y wheel....'Whenhehadfinished,hesaid,'Thisis a car.'Thiswasthefirsttimehehad nameda drawing,andat firstthenamewaspuzzling.Howwasthisa car?0f coursehehad providedthe keyhimself:'Here'sa wheel.'A car,for him,wasdefinedby the criterial characteristicof 'havingwheels',andhisrepresentationfocusedonthis aspect.What he representedwas,infact,'wheelness'.Wheelsarea plausiblecriterionto chooseforthree- year-olds,andthe wheel'saction/on toy carsas on reai cars/is a readilynoticedand describablefeature.In otherwords,thisthree-year-old'sinterestincarswas,for him,most plausiblycondensedintoandexpressedasan interestinwheels.Wheels,inturn,aremost plausiblyrepresentedby circles,bothbecauseof their visualappearanceandbecauseof thecircularmotionof thehandindrawing/representingthewheel'sactionof 'goinground andround'. Togatherthisupfor a moment,weseerepresentationasa processinwhichthemal<ers of signs,whetherchildor adult,seekto makea representationof someobjector entity, whetherphysicalor semiotic,and in whichtheir interestin the object,at the pointof makingtherepresentation,isa complexone,arisingoutof thecultural,socialandpsycho- logicalhistoryof the sign-maker,andfocusedbythe specificcontextin whichthe sign- makerproducesthesign.That'interest'isthesourceoftheselectionofwhatisseenasthe criterialaspectof the object,and this criterialaspectis then regardedas adequately representativeof theobjectin a givencontext.In otherwords,it isneverthe'wholeobject' butonlyeveritscriterialaspectswhicharerepresented. Thesecriterialaspectsarerepresentedinwhatseemsto thesign-maker,at themoment of sign-mal<ing,themostaptandplausiblefashion,andthemostaptandplausiblerepre- sentationalmode(e.g.drawing,Legoblocks,painting,speech).Sign-makersthus'have'a co O g O Fig0.1 Drawingbya three-year-oldchild
  22. 22. Introduction meaning/thesignified,whichtheywishto express,andthenexpressit throughthesemiotic mode(s)that make(s)availablethesubjectivelyfelt,mostplausible,mostaptform,asthe signifier.Thismeansthatinsocialsemioticsthesignisnotthepre-existingconjunctionof a signifieranda signified,a ready-madesignto berecognized,chosenandusedasit is,inthe waythatsignsareusuallythoughtto be'availablefor use'in ,semiology,.Ratherwefocus on the processof sign-making,in whichthe signifier(the form) andthe signified(the meaning)are relativelyindependentof eachotheruntiltheyare broughttogetherbythe sign-makerin a newlymadesign.Toput it in a differentway,usingtheexamplejustabove, the processof sign-makingis the processof the constitutionof a sign/metaphorin two steps:'acaris (mostlike)wheels'and'wheelsare(mostlike)circles,. Puttingit in our terms:the sign-maker'sinterestat this momentof sign-makinghas settledon'wheelness'asthe criterialfeatureof'car'. He constructs,by a processof analogy,twometaphors/signs:first,thesignified'wheel'isaptlyrepresentedbythesignifier 'circle'tomakethemotivatedsign'wheel';second,thesignified'car'isaptlyrepresented by the signifier'manywheels'to makethe motivatedsign'car'. Theresultingsign,the drawingglossed'thisisa car',isthusa motivatedsigninthateachconjunctionof signifier and signifiedis an apt, motivatedconjunctionof the form which bestreoresentsthat whichisto bemeant.Thissignisthustheresultof a doublemetaphoricprocessinwhich analogyistheconstitutiveprinciple.Analogy,inturn,isa processof classification:x is like y (incriterialways).Whichmetaphors(and,'behind'themetaphors,whichclassifications) carrythe dayandpassintothesemioticsystemasconventional,andthenas naturalized, andthenas'natural',neutralclassifications,isgovernedbysocialrelationsof power.Like adults,childrenareengagedin theconstructionof metaphors.Unlikeadults,theyare,on the onehand,lessconstrictedby cultureand its already-existingand usuallyinvisible metaphors,but,onthe otherhand,usuallyin a positionof lesspower,sothat their meta- phorsarelesslikelyto carrytheday. It followsthatweseesignsasmotivated- notas arbitrary-conjunctionsof signifiers (forms)andsignifieds(meanings).In'semiology'motivationis usuallynotrelatedto the act of sign-makingas it is in our approach,but definedin termsof an intrinsicrelation betweenthesignifierandthesignified.It is herethat peirce's'icon',,index,and'symbol, maketheirappearance/incorporatedinto'semiology'ina waywhichin factcontradicts someof the keyideasin Peirce'ssemiotics.The'icon'isthesignin which,thesignifier- signifiedrelationshipis oneof resemblance,likeness'(Dyer,r9g2: r24) - i.e.objective likeness,ratherthananalogymotivatedby'interest',establishestherelation.The,index,is the signin which'thereis a sequentialor causalrelationbetweensignifierandsignified, (Dyer,1982:125);that is,a logicof inference,ratherthananalogymotivatedby,interest,. Thethirdtermin thetriad,'symbol',bycontrast,is relatedto signproduction,asit'rests on convention,or "contract"' (Dyer,r9g2:125), but thisveryfact makesit ,arbitrary,, 'unmotivated'/a caseof meaningbydecreeratherthanof activesign-making. In ourviewsignsareneverarbitrary,and'motivation'shouldbeformulatedin relation to thesign-makerandthecontextinwhichthesignis produced,andnot in isolationfrom theactof producinganalogiesandclassifications.Sign-makersusetheformstheyconsider aptfor theexpressionof theirmeaning,inanymediuminwhichtheycanmakesigns.when
  23. 23. Introduction childrentreat a cardboardbox as a pirateship,they do so becausethey considerthe materialform (box)an apt mediumfor the expressionof the meaningtheyhavein mind (pirateship),andbecauseof theirconceptionof thecriterialaspectsof pirateships(con- tainment,mobility,etc.).Languageis no exceptionto this processof sign-making.All linguisticformis usedin a mediated,non-arbitrarymannerintheexpressionof meaning. Forchildrenintheirearly,pre-schoolyearsthereisbothmoreandlessfreedomof expres- sion:more,becausetheyhavenotyetlearnedto confinethemakingof signsto thecultur- ally andsociallyfacilitatedmedia,andbecausetheyare unawareof establishedconven- tionsandrelativelyunconstrainedin the makingof signs;less,becausetheydo not have suchrichculturalsemioticresourcesavailableasdoadults.Sowhena three-year-oldboy, labouringto climba steephill,says,'Thisisa heavyhill',heisconstrainedbynothavingthe word'steep'asanavailablesemioticresource.Thesameisthecasewiththeresourcesof syntacticandtextualforms. 'Heavy',in'heavyhill',is,however,a motivatedsign:thechildhasfocusedonparticular aspectsof climbinga hill (it takesa lot of energy;it isexhausting)andusesanavailable formwhichheseesasaptfor theexpressionof thesemeanings.Theadultwhocorrectsby offering'steep'('Yes,it's a verysteephill') is,fromthechild'spointof view,notsomuch offeringanalternativeasa synonymfor theprecisemeaningwhichhehadgivento'heavy' in that context.Boththe childandthe parentmakeuseof 'what is available';it happens thatdifferentthingsareavailableto each.Butto concentrateonthisisto missthecentral aspectof sign-making,especiallythatof children.'Availability'is notthe issue.Children, likeadults,maketheirownresourcesof representation.Theyarenot'acquired',but made bytheindividualsign-maker. In 'semiology',countlessstudentsacrosstheworldareintroducedto theterms'langue' and'parole',with 'langue'explained,for instance,as'theabstractpotentialof a language system. . . the sharedlanguagesystemout of whichwe makeour particular,possibly unique,statements'(O'sullivanet a|.,1983:t27) or,in ourterms,asa systemof available formsalreadycoupledto availablemeanings,andwith'parole'definedas: an individualutterancethat isa particularrealizationof thepotentialof langue.. . . Byextensionwe canarguethat thetotal systemof televisionandfilm conventions andpracticesconstitutesalangue,andthewaytheyarerealizedineachprogramme or filma parole. (0'Sullivanet al.,l9B3i I27 ) Weclearlyworkwithsimilarnotions,with'availableforms'and'availableclassifications' ('langue')andindividualactsof sign-making('parole'),andweagreethatsuchnotionscan usefullybeextendedto semioticmodesotherthanlanguage.Butfor usthe ideaof'poten- tial' (whatyoucanmeanandhowyoucan'say'it, inwhatevermedium)isnotlimitedbya systemof 'availablemeanings'coupledwith'availableforms',andwewouldliketo usea slightlylessabstractformulation:a semiotic'potential'isdefinedbythesemioticresources availableto a specificindividualin a specificsocialcontext.0f course,a descriptionof semioticpotentialcanamalgamatethe resourcesof manyspeakersand manycontexts.
  24. 24. 1 0 Introduction Buttheresulting'langue'(thelangueof 'English'orof 'westernvisualdesign,)is in the endan artefactof analysis.Whatexists,andis thereforemorecrucialfor understanding representationandcommunication,aretheresourcesavailableto realpeoplein realsocial contexts.Andif weconstructa'langue',a meaningpotentialfor'Westernvisualdesign,, thenit is no moreandno lessthana tool whichcanserveto describea varietyof sign- makingpractices,withinboundariesdrawnby the analyst.It followsthat we wouldnot drawthe linebetween'langue'and'parole'assharplyas it is usuallydone.Describinga 'langue'isdescribinga specificset of semioticresourcesavailablefor communicative actionto a specificsocialgroup. Hereare someantecedentsof the car drawing.Figure0.2 is a drawingmadeby the samechild,sometenmonthsearlier.Itscircularmotionisexpressiveof thechild'sexuber- ant,enthusiasticandenergeticactionsin makingthedrawing.In figure0.3,madeabout threemonthslater,the circularmotionhasbecomemoreregular.Theexuberanceand energyarestillthere,butthedrawinghasacquiredmoreregularity,moreinterestinshape: 'circularmotion'is beginningto turn into'circle'.In otherwords,themeaningsof figure O Fig0.2 Drawingbya two-year-oldchitd
  25. 25. Introduction I I O Fig0,3 Drawingbyatwo-yeaholdchild 0.2 persistinfigure0.3,transformed,yetwith significantcontinuity:figure0.3gathersup, soto speak,themeaningsof figure0.2,andthentransformsandextendsthem. Figure0.4,finally,showsa seriesof circles,eachdrawnona separatesheet,onecircleto eachsheet.Themovementfrom figure0.2 to figure0.4 is clearenough,as is the con- ceptualandtransformativeworkdonebythechildovera periodof fourteenmonths(figure 0.4 datesfrom the sameperiodasfigure0.1).Togetherthedrawingsshowhowthe child developedtherepresentationalresourcesavailableto him,andwhycirclesseemedsuchan aptchoiceto him:theexpressive,energeticphysicalityof themotionof figure0.2 persisted asthechilddevelopedthisrepresentationalresource/sothatthecircularmotionremained part of the meaningof circle/wheel.Butsomethingwasaddedaswell:thetransformation of representationalresourceswasalsoa transformationof thechild'ssubjectivity,fromthe emotional,physicalandexpressivedispositionexpressedintheactof representing'circular motion'tothemoreconceptualandcognitivedispositionexpressedintheactof represent- inga 'car'. Children,likeallsign-makers,maketheir'own'representationalresources,anddosoas
  26. 26. 12 . Introduction O Fig0.4 Drawingbya three-year.oldchild part of a constantproductionof signs,in whichpreviouslyproducedsignsbecomethe signifier-materialto betransformedintonewsigns.Thisprocessrestson lhe interestof sign-makers.Thistransformative,productivestancetowardssign-makingis at the same timea transformationof thesign-makers'subjectivity- a notionfor whichtherewaslittle placein a 'semiology'whichdescribedthe relationbetweensignifiersand signifiedsas restingon inferenceor objectiveresemblance,or onthedecreesof thesocial'contract'. Wehaveusedchildren'sdrawingsasourexamplebecausewe believethat the produc- tionof signsbychildrenprovidesthebestmodelforthinkingaboutsign-making.It applies alsoto fully socializedand acculturatedhumans,with the exceptionof the effectsof 'convention'.As maturemembersof a culturewe haveavailablethe culturallyproduced semioticresourcesof oursocieties,andareawareof theconventionsandconstraintswhich are sociallyimposedon our makingof signs.However,as we havesuggested,in our approachadultsign-makers,too,areguidedby interest,bythat complexcondensationof culturalandsocialhistoriesandof awarenessof presentcontingencies.'Mature'sign- makersproducesignsout ol that interest,alwaysastransformationsof existingsemiotic materials,thereforealwaysinsomewaynewlymade,andalwaysasmotivatedconjunctions of meaningandform.Theeffectof conventionisto placethepressureof constantlimita- tionsof conformityon sign-making;that is,the way signifiershavebeencombinedwith signifiedsin the historyof theculture,actsasa constantlypresentconstrainton howfar onemightmovein combiningsignifierswith signifieds.Conventiondoesnot negatenew making;it attemptsto limitandconstrainthesemioticscopeof thecombinations. This,then,isourpositionvis-d-vis'European'semiology:wheredeSaussurehad(been assumedto have)saidthattherelationof signifierandsignifiedinthesignisarbitraryand conventional,wewouldsaythatthe relationisalwaysmotivatedandconventional.Where hehadseeminglyplacedsemioticweightandpowerwiththesocial,wewishto assertthe effectsof thetransformativeroleof individualagents,yetalsotheconstantpresenceof the social:in thehistoricalshapingof the resources,inthe individualagent'ssocialhistory,in
  27. 27. ( 1 ) Introduction the recognitionof presentconventions,intheeffectof theenvironmentinwhichrepresen- tationandcommunicationhappen.Yetit isthetransformativeactionof individuals,along thecontoursof socialgivens,whichconstantlyreshapestheresources,andmal<espossible theself-makingof socialsubjects. 0neof the nowtaken-for-grantedinsightsof sociallyorientedtheoriesof languageis thevariationof languagewiththevariationof socialcontext.Theaccountsofthisvariation differ,rangingfromcorrelation('languageformx relatesto socialcontexty') to determin- ation('languageformx is producedby socialactorsy or in socialcontexty').A social semioticapproachtakesthelatterview,alongthefollowinglines. Communicationrequiresthat participantsmaketheir messagesmaximallyunder- standablein a particularcontext.Theythereforechooseformsof expressionwhich theybelieveto be maximallytransparentto otherparticipants.0nthe otherhand, communicationtakesnlacein socialstructureswhichareinevitablymarkedbypower differences,andthisaffectshoweachparticipantunderstandsthenotionof'maximal understanding'.Participantsin positionsof powercanforceotherparticipantsinto greatereffortsof interpretation,andtheirnotionof'maximalunderstanding'isthere- foredifferentfrom that of participantswhodo their bestto producemessagesthat will requirea minimaleffortof interpretation,or from that of participantswho, throughlackof commandof the representationalsystem,producemessagesthat are harderto interpret(e.g.children,learnersof a foreignlanguage).Theotherpartici- pantsmaytheneithermaketheeffortrequiredto interpretthesemessagesor refuse to doso,whetherin a schoolor in a railwaystationin a foreigncountry. Representationrequiresthat sign-makerschooseformsfor the expressionof what theyhavein mind,formswhichtheyseeasmostaptandplausibleinthegivencontext. Theexamplesaboveinstantiatethis: circlestostandfor wheels,andwheelsto stand for cars; heavyto standfor significanteffort, andsignificanteffort to standfor climbinga steeps/ope.Speakersof a foreignlanguageuseexactlythe samestrategy. Theychoosethe nearest,mostplausibleform theyknowfor the expressionof what theyhavein mind.Therequirementsof communicationarenodifferentin moreusual circumstances,they are simplylessapparent.The interestof sign-makers,at the momentof makingthesign,leadsthemto chooseanaspector bundleof aspectsof the objectto berepresentedasbeingcriterial,at thatmoment,for representingwhatthey want to represent,andthen choosethe most plausible,the mostapt form for lts representation.Thisappliesalsoto the interestof thesocialinstitutionswithinwhich messagesare produced,andthereit takestheform of the (historiesof) conventions andconstraints. APPLICATIONS In theprevioussectionwehavefocusedonthetheoreticalbackgroundof ourwork,butour aimsarenotjusttheoretical.Theyarealsodescriptiveandpractical.Weseekto developa ( 2 )
  28. 28. 14 . Introduction descriptiveframeworkthatcanbeusedasa toolfor visualanalysis.Sucha toolwill have its usefor practicalaswellasanalyticalandcriticalpurposes.Togivesomeexamplesof theformer,educationalistseverywherehavebecomeawareof the increasingroleof visual communicationin learningmaterialsof variouskinds,andtheyareaskingthemselveswhat kind of maps,charts,diagrams,picturesandformsof layoutwill be mosteffectivefor learning.Toanswerthisquestiontheyneeda languagefor speakingabouttheformsand meaningsofthesevisuallearningmaterials.Withinthemedia,visualdesignislessandless the provinceof specialistswhohadgenerallyseenlittleneedfor methodicalandanalytic- ally explicitapproaches,and had reliedinsteadon creativesensibilitieshonedthrough experience.But wheremediaformsare relativelyrecentlyintroduced- as isthecase.for example,withadvertisingin EasternEuropeandpartsof Asia- thereisnosuchresistance to combiningsystematicanalysisandpractice.Andwith theadvanceof easyto usesoft- warefor desktoppublishing,the productionof diagramsandcharts,imagemanipulation, etc.,visualdesignbecomeslessof a specialistactivity,somethingmanypeoplewill do alongsideotheractivities.Thishasalreadyledto rapidgrowthinthenumberof coursesin thisarea- anddesigningsuchcoursesrequiresmoreof an analyticalgraspof principles thanlearningonthejob byexampleandosmosis.Last,andmaybeat bottomat therootof muchofthischange,is'globalization',which- maybenearlyparadoxically- demandsthat theculturalspecificitiesof semiotic,social,epistemologicalandrhetoricaleffectsof visual communicationmustbeunderstoodeverywhere,sincesemioticentitiesfromanywnerenow appearandare'consumed'everywhere. Analysingvisualcommunicationis,or shouldbe,an importantpart of the 'critical' disciplines.Althoughin thisbookwe focuson displayingthe regularitiesof visualcom- munication,ratherthan its ('interested',i.e.political/ideological)uses,we seeimagesof whateverkindasentirelywithinthe realmof the realizationsandinstantiationsof ideol- ogy/asmeans- always- for thearticulationof ideologicalpositions.Theplainfactof the matteristhat neitherpowernor itsusehasdisappeared.It hasonlybecomemoredifficult to locateandto trace.In that contextthereis an absoluteneedin democratictermsfor makingavailablethe meansof understandingthearticulationsof poweranywhere,in any form.Thestillgrowingenterpriseof 'criticaldiscourseanalysis'seeksto showhowlan- guageis usedto conveypowerandstatusin contemporarysocialinteraction,andhowthe apparentlyneutral,purelyinformative(linguistic)textswhichemergein newspaperreport- ing,governmentpublications,socialsciencereports,and so on, realize,articulateand disseminate'discourses'as ideologicalpositionsjust as muchas do textswhich more explicitlyeditorializeor propagandize.To do sowe needto beableto'read betweenthe lines',in orderto geta senseof whatdiscursive/ideologicalposition,what'interest',may havegivenriseto a particulartext,and maybeto glimpseat leastthe possibilityof an alternativeview.It is this kindof readingfor whichcriticaldiscourseanalysisseeksto providethe waysandmeans.Sofar,however,criticaldiscourseanalysishasmostlybeen confinedto language,realizedasverbaltexts,or to verbalpartsof textswhichalsouse othersemioticmodesto realizemeaning.Weseeourbookasa contributionto a broadened criticaldiscourseanalysis,andwe hopethat our exampleswill demonstrateits potential for thiskindof work.
  29. 29. Introduction . 5 0ur examplesinclude'text-objects'ofmanyI<inds,fromworksof art to entirelyordin- ary,banalartefactssuchas maps,charts,pagesof differentkinds,includingthoseof websites,etc.Wehaveincludedworl<sof art notjustbecauseof theirkeyroleinthehistory of conventionsandconstraints,hencein theformationof the'grammarof visualdesign', butalsobecausethey,too,articulateideologicalpositionsof complexandpotentkinds,and they,too,shouldbeapproachedfromthepointof viewof socialcritique. As is perhapsalreadyobviousfrom what we havesaidso far,we believethat visual design,likeallsemioticmodes,fulfilsthreemajorfunctions.TouseHalliday'sterms,every semioticfulfilsbothan'ideational'function,a functionof representing'theworldaround andinsideus'andan'interpersonal'function,a functionof enactingsocialinteractionsas socialrelations.All messageentities- texts- alsoattemptto presenta coherent'worldof thetext',whatHallidaycallsthe'textual'function- a worldin whichall theelementsof thetextcohereinternally,andwhichitselfcohereswith itsrelevantenvironment.Whether we engagein conversation,producean advertisementor playa pieceof music,we are simultaneouslycommunicating,doingsomethingto,orfor,or with,othersin thehereand nowof a socialcontext(swappingnewswitha friend;persuadingthereaderof a magazine to buysomething;entertainingan audience)and representingsomeaspectof the world 'out there',be it in concreteor abstractterms(thecontentof a film we haveseen;the qualitiesof theadvertisedproduct;a moodor melancholysentimentor exuberantenergy conveyedmusically),andwebindtheseactivitiestogetherin a coherenttextor communi- cativeevent.Thestructureof our book reffectsthis.Chapters2 and3 dealwith the patternsof representationwhichthe 'grammarof visualdesign'makesavailable,and hencewith the wayswe canencodeexperiencevisually.Chapters4 and5 dealwith the patternsof interactionwhichthe'grammarof visualdesign'mal<esavailable,andhence with thethingswe cando to, or for,eachotherwith visualcommunication,andwith the relationsbetweenthe makersandviewersof visual'texts'whichthis entails.Chapter6 dealswiththe'textual'function,withthewayinwhichrepresentationsandcommunicative actscohereintomeaningfulwholes.Chapter7 dealswiththematerialityof visualsigns- the toolswe makethemwith (ink,paint,brushstrokes,etc.)andthe materialswe make themon (paper,canvas/computerScreens/etc.);these,too,contributeto the meaningof visualtexts.Chapter8 extendsthepreviouschaptersintothedomainofthree-dimensional visualsand movingimages.Againwe assumethat there is somethinglil<ea Western 'grammarof three-dimensionalvisualdesign',a setof availableformsandmeaningsused in sculptureaswellas,for instance,inthree-dimensionalscientificmodels,or inchildren's toys- anda Western'grammarof themovingimage'. Wewill begin,however,bydiscussingsomeof thebroaderthemeswehavetouchedonin thisintroduction.
  30. 30. I T h es e m i o t i cl a n d l a n g u a g ea n dv i s u a l s c a p e : c o m m u n i c a t i o n In theearlyyearsof schooling,childrenareconstantlyencouragedto produceimages,and to illustratetheirwrittenwork.Teacherscommentontheseillustrationsas muchasthey doonthewrittenpartof thetext,thoughperhapsnotquiteinthesamevein:unlikewriting, illustrationsarenot'corrected'norsubjectedto detailedcriticism('thisneedsmorework,, 'not clear','spelling!','poorexpression',and so on).Theyare seenas self-expression, ratherthan as communication- as somethingwhichthe childrencando arreaoy,spon- taneously,ratherthanassomethingtheyhaveto betaught. Bythetimechildrenare beyondtheirfirst two yearsof secondaryschooling,illustra- tionshavelargelydisappearedfromtheirownwork.Fromhereon,in a somewhatcontra- dictorydevelopment,writingincreasesin importanceandfrequencyandimagesbecome specialized.Thisis mademoreproblematicbythe factsof the presentperiod,in which writingand imageare in an increasinglyunstablerelation.We mightcharacterizethe situationof saytwentyor thirtyyearsagointhisway:textsproducedfor theearlyyearsof schoolingwererichlyillustrated,but towardsthe lateryearsof primaryschoolimages beganto givewayto a greaterandgreaterproportionof writtentext.In asmuchasimages continued,theyhadbecomerepresentationswith a technicalfunction,maps,diagramsor photographsillustratinga particularlandformor estuaryor settlementtype in a geog- raphytextbook,for instance.Thuschildren'sownproductionof imageswaschannelledin thedirectionof specialization- awayfrom 'expression'andtowardstechnicality.In other words,imagesdidnotdisappear,buttheybecamespecializedintheirfunction. In manywaysthe situationin schoolremainsmuchthe same,with two profoundly importantprovisos.0nthe onehandall schoolsubjectsnowmakemuchmoreuseof images,particularlyso in the yearsof secondaryschooling.In manyof thesesubjects, certainlyin themoretechnical/scientificsubjectssuchas(in England)Science,Informa- tion Technologyor Geography,imageshavebecomethe major meansof representing curricularcontent.In the morehumanisticsubjects- for example,History,Englishand ReligiousStudies- imagesvary in theirfunctionbetweenillustration,decorationand information.Thistrendcontinues,andit isthecasefor worksheets,intextbooksandin CD- R0Ms.0ntheotherhand,thereisnoteachingor'instruction'inthe(new)roleof images (thoughin England,in the schoolsubjectInformationTechnology,thereis teachingin desktoppublishing).Mostimportantly,assessmentcontinuesto bebasedonwritingasthe majormode.Studentsarecalleduponto makedrawlngsin Science,GeographyandHis- tory; but,asbefore,thesedrawingstendnot to bethe subjectof the teacher,sattention, iudgingby their (written)commentson the children'swork.In otherwords,materials providedforchildrenmakeintenserepresentationaluseof images;in materialsdemanded fromchildren- invariousformsof assessmentparticularly- writingremainstheexpected anddominantmode. Outsideschool,however,imagesplayan ever-increasingrole,andnotjust in textsfor
  31. 31. Thesemiotic landscape . 1,7 children.Whetherintheprintor electronicmedia,whetherin newspapers,magazines,CD- R0Ms or websites,whetheras publicrelationsmaterials,advertisementsor as infor- mationalmaterialsof all kinds,mosttextsnowinvolvea complexinterplayof writtentext, imagesandothergraphicor soundelements,designedascoherent(oftenat thefirst level visualratherthanverbal)entitiesby meansof layout.Butthe skillof producingmulti- modaltextsof this kind,howevercentralits rolein contemporarysociety,is not taught in schools.Toputthispointharshly,intermsof thisessentialnewcommunicationability, this new'visualliteracy',institutionaleducation,underthe pressureof oftenreactionary politicaldemands,producesilliterates. 0f course,writingis itselfa form of visualcommunication.Indeed,andparadoxically, thesignof thefully literatesocialpersonis the abilityto treatwritingcompletelyasa visualmedium- for instanceby not movingone'slipsandnot vocalizingwhenoneis reading,noteven'subvocalizing'(asilent'speakingaloudinthehead',to bringoutthefull paradoxof thisactivity).Readerswhomovetheir lipswhenreading,whosubvocalize,are regardedasculturallyandintellectuallytaintedbyhavingto takerecourseto theculturally lessvaluedmodeof spol<enlanguagewhenreadingvisualscript.This'old'visualliteracy, writing,hasfor centuriesnowbeenoneof the mostessentialachievementsandvaluesof Westernculture,andoneof the mostessentialgoalsof education,so muchsothat one majorandheavilyvalue-ladendistinctionmadebyWesterncultureshasbeenthatbetween literate(advanced)andnon-literate(oralandprimitive)cultures.Nowonderthatthemove towardsa newliteracy,basedon imagesandvisualdesign,cancometo beseenasa threat, a signof thedeclineof culture,andhencea particularlypotentsymbolandrallyingpoint for conservativeandevenreactionarysocialgroupings. Thefadingout of certainkindsof textsby andfor children,then,is not a straight- forwarddisvaluationof visualcommunication,but a valuationwhichgivesparticular prominenceto onekindof visualcommunication,writing,andto onekindof visualliteracy, the'old'visualliteracy.Othervisualcommunicationis eithertreatedasthedomainof a verysmalleliteof specialists,or disvaluedasa possibleformof expressionfor articulate, reasonedcommunication/seenasa'childish'stageonegrowsoutof.Thisisnota valuation of languageassuchovervisualcommunication,becauseevennowthestructures,meanings and varietiesof spokenlanguageare largelymisunderstood,and certainlynot highly valuedrn their varietyin the educationsystem(withsomeexceptions,suchas in formal 'debating')or in publicforumsof power. Tosumup:theoppositionto theemergenceof thevisuafasa full meansof representa- tionisnotbasedonanoppositionto thevisualassuch,butonanoppositionin situations whereit formsan alternativeto writingandcanthereforebeseenasa potentialthreatto thepresentdominanceof verballiteracyamongelitegroups. In thisbookwetal<ea freshlool<at thequestionof thevisual.Wewantto treatformsof communicationemployingimagesasseriouslyaslinguisticformshavebeen.Wehavecome to this positionbecauseof the nowoverwhelmingevidenceof the importanceof visual communication,andthe nowproblematicabsenceof the meansfor talkingandthinking aboutwhatisactuallycommunicatedbyimagesandbyvisualdesign.In doingso,wehave to moveawayfromthepositionwhichRolandBarthestook in his1964essay'Rhetoricof
  32. 32. I8 Thesemiotic landscape theimage'(1977i32-51).ln thisessay(andelsewhere,asintheintroductionto Elements of Semiology;Barthes,1967d, he arguedthat the meaningof images(and of other semioticcodes,likedress,food,etc.) is alwaysrelatedto and,in a sense,dependenton, verbaltext.Bythemselves,imagesare,hethought,too 'polysemous',too opento a variety of possiblemeanings.Toarriveat a definitemeaning,languagemustcometo the rescue. Visualmeaningistooindefinite;it isa'ffoatingchainof signifieds'.Hence,Barthessaid,'in everysocietyvarioustechniquesaredevelopedintendedto fx theffoatingchainof signi- fiedsinsucha wayasto countertheterrorof uncertainsigns;the linguisticmessageisone of thesetechniques'i977'.39). Hedistinguishedbetweenan image-textrelationinwhich theverbaltextextendsthemeaningof theimage,or viceversa,asisthecase,for example, withthespeechballoonsin comicstrips,andan image-textrelationin whichtheverbal textelaboratestheimage,orviceversa.lntheformercase,whichhecalledrelay,newand differentmeaningsareaddedto completethe message.In the lattercase,thesamemean- ingsare restatedin a different(e.g.moredefiniteand precise)way,as is the case,for example,whena captionidentifiesand/orinterpretswhatisshownin a photograph.0f the two,elaborationisdominant.Relay,saidBarthes,is'morerare'.Hedistinguishedtwotypes of elaboration,oneinwhichtheverbaltextcomesfirst,sothattheimageformsan illustra' tionof it,andoneinwhichtheimagecomesfirst,sothatthetextformsa moredefiniteand preciserestatementor'fixing'ofit (a relationhecallsanchorage). Beforeapproximately1600(thetransitionis,of c0urse,verygradual),Barthesargued, 'illustration'wasdominant.Imageselaboratedtexts,morespecificallythefoundingtexts of theculture- mythology,the Bible,the'holywrit'of theculture- texts,therefore,with whichviewerscouldbeassumedto befamiliar.Thisrelation,inwhichverbaltextsformeda sourceof authorityin society,andin whichimagesdisseminatedthe dominanttextsin a particularmodeto particulargroupswithinsociety,graduallychangedto onein which nature,ratherthandiscourse,becamethesourceof authority.In theeraof science,images, evermorenaturalistic,beganto functionas'thebookof nature',as'windowsontheworld', as'observation',andverbaltextservedto identifyandinterpret,to'loadtheimage,burden- ingit witha culture,a moral,an imagination'. Thispositiondoesexplainelementsof communication.Anyoneof theimage-textrela- tionsBarthesdescribesmayat timesbedominant,althoughwefeelthattodaythereis a moveawayfrom 'anchorage'.Compare,for example,the'classic'documentaryfilm in whichthe vieweris first confrontedwith'imagesof nature',thenwith the authoritative voiceof a narratorwho identifiesand interpretsthe images,with the modern'current affairs'item,in whichthe vieweris first confrontedwith the anchorperson'sverbaldis- courseand,eithersimultaneouslyor followingon fromtheverbalintroduction,withthe 'imagesof nature'thatillustrate,exemplifyandauthenticatethediscourse.But Barthes' accountmissesan importantpoint:the visualcomponentof a text is an independently organizedandstructuredmessage,connectedwiththeverbaltext,butin nowaydependent onit - andsimilarlytheotherwayaround. Oneimportantdifferencebetweenthe accountwe developin this bookandthat of earliersemioticiansis our useof work in linguistictheoriesanddescriptions.Thisis a difficultargumentto make,but worth makingclearly.Wethinkthatthis bookwouldnot
  33. 33. Thesemiotic landscape 1 9 havebeenpossiblewithouttheachievementsof linguistics,yetwedo not,inthewaysome criticsof ourapproachhavesuggested,seeourapproachasa linguisticone.Sowhathave we usedfrom linguistics,andhowhavewe usedit? And,equally,what havewe not used from linguistics?Tostartwiththe latterquestion,we havenot importedthetheoriesand methodologiesof linguisticsdirectlyintothe domainof the visual,as hasbeendoneby othersworkinginthisfield.Forinstance,wedonotmakea separationof syntax,semantics andpragmaticsin the domainof thevisual;we do not lookfor (theanaloguesof) sen- tences,clauses,nouns,verbs,andso on,in images.We takethe viewthat languageand visualcommunicationcan bothbe usedto realizethe'same'fundamentalsystemsof meaningthat constituteour cultures,but that eachdoesso by meansof its ownspecific forms,doessodifferently,andindependently. To givean example,the distinctionbetween'subjective'and'objective'meaningshas playedan importantrole in Westerncultureeversincethe physicalsciencesbeganto developin the sixteenthcentury.Thisdistinctioncanbe realized(that is,givenconcrete, materialexpression,hencemadeperceivableandcommunicable)withlinguisticaswellas visualmeans.Theterms'subjective'and'objective'canthereforebeappliedto both:they belongto the meaningpotentialof a cultureanditssociety.Butthewaythedistinctionis realizedin languageisquitedifferentfromthewayit isrealizedin images.Forexample,in languagean ideacanberealizedsubjectivelybyusinga'mentalprocessverb'likebelieve in the first person(e.9. We believethat thereis a grammar of image); or objectively throughtheabsenceof sucha form (e.9.Thereis a grammarof image).Visualrepresen- tation,too,canrealizebothsubjectivity,throughthepresenceof a perspectivalangle,and objectivity,throughitsabsence,a pointwhichwill bediscussedmorefully in chapter4. Mentalprocessclausesandnominalizationareuniqueto language.Perspectiveis unique to images.Butthe kindsof meanrngexpressedarefrom the samebroaddomainin each case;andtheforms,differentastheyare/weredevelopedinthesameperiod,in responseto the sameculturalchanges.Bothlanguageandvisualcommunicationexpressmeanings belongingto andstructuredbyculturesin the onesociety;the semioticprocesses,though notthesemioticmeans/arebroadlysimilar;andthisresultsin a considerabledegreeof congruencebetweenthetwo. At the sametime,however,eachmediumhasits ownpossibilitiesand limitationsof meaning.Noteverythingthatcanberealizedin languagecanalsoberealizedbymeansof images,or viceversa.Aswellasa broadculturalcongruence,thereissignificantdifference betweenthetwo (andothersemioticmodes,of course).In a languagesuchas Englishone needsto usea verbin orderto makea full utterance(believe,rs);andlanguagehasto use namesto referto whateveris to berepresented(a grammarof images,believe,wd.But languagedoesnothaveor needanglesof visionto achieveperspective,nordoesit haveor needspatialdispositionsof elementsto achievethemeaningsof syntacticrelations:images haveandneedboth.Themeaningpotentialsof thetwo modesareneitherfullyconflated norentirelyopposed.Wedifferfromthosewhoseethemeaningof languageasinherentin the formsandthe meaningof imagesas derivedfrom the context,or the meaningsof languageas'conscious'andthemeaningsof imagesas'unconscior.ls'. To returnto thefirst of our two questions- What havewe usedfrom linguistics,and
  34. 34. 20 . Thesemiotic landscape howhaveweusedit?- perhapsthemostsignificantborrowingisouroverallapproach,an 'attitude'whichassumesthat,asa resourcefor representation,images,likelanguage,will displayregularities,whichcanbemadethesubjectof relativelyformaldescription.Wecall thisa'grammar'todrawattentionto culturallyproducedregularity.Morespecifically,we haveborrowed'semioticorientations',featureswhichwetakento begeneralto all human meaning-making,irrespectiveof mode.Forinstance,wethinkihat thedistinctionbetween 'objectivity'and'subjectivity'isa generalcultural/semioticissuewhichcanbe realized linguisticallyas well as visually,thoughdifferentlyso,as we havesaid.0r,as another instance,we havetaken MichaelHalliday'ssocialsemioticapproachto languageas a model,asa sourcefor thinkingaboutgeneralsocialandsemioticprocesses/ratherthanas a minefor categoriesto applyin the descriptionof images.His modelwith its three functionsis a startingpointfor our accountof images,not becausethe modelworkswell for language(whichit does,to anextent),butbecauseit workswellasa sourceforthinking aboutall modesof representation. Maybemostto thepointisthis:ourapproachto communicationstartsfroma social base.In ourviewthemeaningsexpressedbyspeakers,writers,printmakers,photographers, designers,paintersandsculptorsarefirst andforemostsocialmeanings/eventhoughwe acknowledgetheeffectandimportanceof individualdifferences.Giventhat societiesare nothomogeneous,butcomposedof groupswithvarying,andoftencontradictory,interests, themessagesproducedbyindividualswill reffectthedifferences,incongruitiesandclashes whichcharacterizesociallife.It is likely,and in our experienceoftenthe case,that the differentmodesthroughwhichtextsareconstructedshowthesesocialdifferences,sothat in a multimodaltextusingimagesandwritingthewritingmaycarryonesetof meanings andthe imagescarryanother.In an advertisement,for instance,it maybethattheverbal textisstudiously'non-sexist',whilethevisualtextencodesovertlysexiststereotypes.Given the still prevalentsenseaboutthe meaningof images,it is possibleto pretendthat the meaningcarriedin the imageisthereonly'intheeyeof the beholder',somethingthat it wouldnotbepossibleto assertaboutverballyrealizedmeanings. Ourexamplesin this bookarequitedeliberatelydrawnfrom verymanydomains,and from differenthistoricalperiods.We hopethat our ideaswill helpanyoneinterestedin communicationto seein imagesnot onlytheaestheticandexpressive,but alsothe struc- turedsocial,politicalandcommunicativedimensions.We will drawexamplesfrom the kindsof textswhicharealreadyfullybasedonthenewvisualliteracyandplaya dominant rolein anypublicsphere,magazinearticles,advertisements,textbooks,websitesandsoon. Thisisnotbecausewewantto promotethesetextsasa kindof modelwhichshouldreplace otherkindsof texts,butbecausetheirroleinthelivesof childrenandadultsissoimportant thatwesimplycannotaffordto leavetheabilityto ihinkandtalk aboutthem(and,indeed, to producethem)to a handfulof specialists.Wehavea particularinterestin the placeof thevisualinthelivesof children,andwehopeto showthatchildrenveryearlyon,andwith verylittlehelp(despitealltheencouragement),developa surprisingabilityto useelements of the visual'grammar'- an abilitywhich,we feel,shouldbe understoodbetterand developedfurlher,ratherthan beingcut off prematurelyas is,too often,the caseat present;andanabilitythatshouldalsobeavailableto adults.
  35. 35. Thesemiotic landscape 2 7 AN UNCONVENTIONALHISTORYOFWRITING Thedominanceof theverbal,writtenmediumoverothervisualmediaisfirmlycodedand buttressedin conventionalhistoriesof writing.Thesegosomethinglikethis.Languagein itsspokenformisa naturalphenomenon,commonto all humangroups.Writing,however, istheachievementof onlysome(historically,byfartheminorityof)cultures.At a particu- lar stagein the historyof certaincultures,theredevelopedthe needto makerecordsof transactionsof variouskinds,associatedusuallywithtrade,religionor (governing)power. Theserecordswereinitiallyhighlyiconic;that is,the relationbetweenthe objectto be recordedandtheformsandmeansof recordingwascloseandtransparent.For instance, thenumberof notchesina stickwouldrepresentthenumberof objectsstoredor tradedor owed.Therepresentationof the objectwouldusuallyalsobe transparent:a wavy line eventuallybecamethe Chineseideogramfor'water';the hieroglyphicimageof the ox's headwhichinitially'stoodfor''ox'eventuallybecametheletteraleph( N),alpha(a),a.This exampleillustrateswhatinthesehistoriesisregardedastherarestof all achievements,the inventionof alphabeticwriting. Alphabeticwritingdeveloped,it seemsclear,outof iconic,image-basedscripts.In these originalscriptforms,an objectwasinitiallyrepresentedbyan imageof that object.Over time,intheuseofthescriptbydifferentgroups,speakingdifferentlanguages,theimageof theobjectcameto standforthenameof theobjectandthenfor itsinitialletter.Aleph,'ox' in Egyptianhieroglyphics,aftercenturiesof travelandconstanttransformationthrough the culturesand languagesof the easternMediterranean,becamethe letteralpha,and eventuallythe lettera in the Romanalphabet.Clearlythiswasa processwhereeachstep involvedconsiderableabstraction,somuchsothat,seemingly,alphabeticwritinghasbeen inventedonlyoncein thehistoryof humancultures.All presentalphabeticscripts,from Indiato the MiddleEastto Europe,aredevelopmentsof that initialstepfrom Egyptian (or possiblySumerian)iconichieroglyphicrepresentationto the Phoenicianalphabet, and from therewestwardto the Greek-speakingworld,and eastwardto the Indian subcontinent,or,in the regionof its origin,developingintothe Arabicversionof the alphabet. Thisis indeedan impressiveculturalhistory,impressiveenoughto havestoodasthe acceptedhistoricalaccountof the achievementof (alphabetic)writing,unquestionedfor centuries.Withinthisaccount,all cultureswithformsof visualrepresentationthatarenot directlyconnectedto languagearetreatedascultureswithoutwriting.However,it isworth investigatingthis history,andin particularthe crucialstepfrom visualrepresentationto the linkwith language,a littlemoreclosely.Priorto thisstep(in realitya development spanningmillennia)thereweretwoseparateandindependentmodesof representation.0ne waslanguage-as-speech;the other,the visualimage,or visualmarks.Eachserveda par- ticularsetof purposessuchasthe constructionof historiesandmyths,the recordingof genealogiesandtransactions,andtherecordingandmeasurementof objects.In thecaseof somecultures,however,the oneform of representation'took over'the other,asa means of recording;that is,visualrepresentationbecamespecialized- onecouldsay,reduced- to functionasa meansof thevisualrepresentationof speech,perhapsin highlyorganizedand
  36. 36. 22 Thesemiotic landscaPe bureaucratizedsocieties.At thispointthevisualwassubsumed,takenover,bytheverbalas itsmeansof recording.Consequentlyitsformerpublicuses,possibilitiesandpotentialsfor independentrepresentationdisappeared,declinedandwitheredaway' In the caseof othercUltures,however,thisdevelopmentdid not occur.Herethe visual continued,alongwith theverbalmeansof representation.Instancesof thisabound:from theoneextremeof theIncaquipustrings(sensorilythetactilemodeof representation)to AustrallanAboriginaldrawings,sand-paintingsandcarvings.Theseencode,in a manner not at all directlydependentontor a 'translation'of,verballanguage,meaningsof the culturewhicharedeemedto bebestrepresentedin visualform.Theyareconnectedwith language,or languagewiththem,sothatwall-paintingsor sand-paintings,for instance,are accompaniedbyverbalrecountsof geographicalfeatures,journeystancestormyths,andso on. However,in thesecasesthereis no questionof the priorityof the oneoverthe other mode,andthe visualhascertainlynot becomesubsumedto the verbalas its form of representation. In thisconnectionit is interestingto considerthehistoryof twowordswhichina sense are synonymouswith Westernnotionsof literacy,the wordsgrammarandsyntax'Gram- mar derivesfrom the Greekgrammatike('the art of readingandwriting','qrammar', 'alphabet'),'relatedwordSweregramma('Sign','letter','alphabet'),grammatikosCliter- ate,,,(primary)teacher','grammarian').Thisetymologyrecordsthestateof thingsin the Hellenisticperiod(fromapproximately3008c);in earliertimesthe meaning'sign',as in 'paintedor drawn[etc.] mark'wastheprimarymeaning.In Homer,for example,theverb grapheinstill means'Scratch','scratchin',as in engraving/andfr6m thereit comesto meanboth'writing'and'drawing','painting'.Syntaxis,in pre-Hellenistictimes,meant 'Contract','Wage','OrganizatiOn','System','battlefOrmatiOn',with SyntagmatfOrinStanCe, 'contingentof troops','constitution(ofa state)','bookor treatise'.0nlyinthe Hellenistic perioddoessyntaxiscometo mean(amongits othermeanings)'grammaticalconstruc- tion'.The verbsyntasso,again/meansboth'arrangebattleformations'and 'concentrate (one'sthoughts)','organize','write','compose'. Whilewedonotwishto placetoomuchemphasisonetymology,neverthelessthehistory of thesetwowordswhicharesocrucialto our notionsof literacypointsto formsof social organizationandorder,on the onehand,andto visual'markings'on the other'Together theyindicatethe initiallyquiteindependentorganizationof the modeof imagesandthe modeof verballanguage.At thesametime,the subsequenthistoryof theword grammar bringsout clearlythe subordinationof the visualmediumto the mediumof verballan- guage.Cultureswhichstill retainthefull useof bothmediaof representationare,fromthe oointof viewof'literate cultures',regardedas illiterate,impoverished,underdeveloped, whenin fact theyhavea richerarrayof meansof representationthanthat overtlyand consciouslyavailableto literatecultures.Nevertheless,aswe pointedout earlier,literate culturesdo makeuseof meansof visualcommunicationotherthanwriting,beit thatthey areseenasuncodedreplicasof realityor asa meansof individualexpressionbychildrenor artists.In otherwords,theyare nottreatedaseitherthe expressionsof,or accessibleto meansof readingbasedon,articulated,rationalandsocialmeanings' 0ur unconventionalhistoryof writingis onethat treatsthe comingtogetherof visual
  37. 37. Thesemiotic landscape 2 3 andverbalrepresentationasonlyonepossibility/andone/furthermore,thatbringswith it notjustthosebenefitsof writingwhicharewelI enoughunderstood,butalsothenegative aspectsincurredin the lossof an independentform of representation,the diminutionof modesof expressionandrepresentation.Fromthat pointof viewculturessuchasAustral- ianAboriginalculturesareseenas havingbothmodesof representation:thevisual(or perhapsa wholesetof visualformsof representation)andtheverbal.Thepointof this historyis not onlythe politicaloneof underminingthe notionof illiterateculture'(or 'merelyoralculture'),butalsotheattemptto seeto whatextenttheconventionalhistory blindsusto thefactsandusesof visualcommunicationinso-calledliteratecultures. In thisbookwe developthe hypothesisthat in a literateculturethe visualmeansof communicationare rationalexpressionsof culturalmeanings,amenableto rational accountsandanalysis.Theproblemwhichwe faceis that literatecultureshavesystem- aticallysuppressedmeansof analysisof thevisualformsof representation,sothatthereis not,at the moment,an establishedtheoreticalframeworkwithinwhichvisualformsof reoresentationcanbediscussed. THE'OLD'ANDTHE'NEW'VISUALLITERACYIN BOOKSFORTHEVERY Y O U N G Sofar wehavedistinguishedtwo kindsof visualliteracy:oneinwhichvisualcommunica- tionhasbeenmadesubservientto languageandinwhichimageshavecometo beregarded asunstructuredreplicasof reality(the'oldvisualliteracy',inourterms);andanotherin which(spoken)languageexistssideby sidewith,and independentof,formsof visual representationwhichare openlystructured,ratherthan viewedas moreor lessfaithful duplicatesof reality(the'new',in ourterms).Wehavelookedat theseashistoricaland culturalalternatives.But theyalsoexistsideby side,at leastin contemporaryWestern culture,andwesuggestthatweareinthemiddleof a shiftin valuationandusesfromthe onemodeto theother,fromthe'old'tothe'new'visualliteracy,in manyimportantsocial contexts.Theexampleswe will now discusssuggestthat the veryfirst bookschildren encountermayalreadyintroducethemto particularkindsof visualliteracy. Figure1.1 showsa typicaltwo-pagespreadfrom Baby'sFirstBook,a bookwhich,on its insidecover,declaresthat'thetextandillustrations,thoughoversimpleto grown-ups, will satisfytheirti.e.thetoddlers'lcravingsfor therepetitionof whattheyalreadyknow, andwill helpthemassociatethewordswiththeobjects'.Whenwewrotethefirstversionof thischapter,in 1989,it wasstill widelydistributed,andtodayit is alreadymakinga comebackasanobjectof nostalgia. Figure1.2 showsa typicalpagefrom Dicl<Bruna's0n My Walk.Thisbookis oneof a setof four,theothersbeingIn My Home,In My ToyCupboardand0n theFarm.lt consists of eightpagesand,with theexceptionof thefront andbackcovers,the pagescontainno wordswhatsoever. Comparedto the pictureof the bird in the tree,the pictureof the bath is realistic, detailedand complex.If we wereto analyseit into its components,if we wereto try
  38. 38. Thesemiotic landscape {-l - , ' - - - , , . - , , i . - _y * i ' 1 d -,J i;*iir*o** i'i:u **:,::1 f: i,; i: i: , -"i i i'lii !ii:l f-Ttili 3 r !,-,,t E "r O Figl.l My bath(from Bahls Fitst Boo&Ladybird) andidentifyall thedifferentelementsof thispicture,we mightencounterproblems.Are the ripplesin the waterto becountedascomponents?Are the shadows,castbythetub andtowel?And if we wereto try and identifythe relationsbetweenthesecomponents, what wouldwe haveto say,for example,aboutthe relationbetweenthe duckandthe soap?We ask thesequestionsbecausetheyare the kindsof questionswith whichone might start if one wantedto showthat imagesare structuredmessages,amenable to constituentanalysis.Isn't the structureherethat of the culturalobject'bathroom', ratherthanoneimposedbytheconventionsof a visualcode?Isn'tthispictureunproblem- atically,transparentlyreadable(recognizable),providedoneknowswhatbathroomslook like? Thisisthe lineParisSchoolsemioticianssuchas RolandBarthesandChristianMetz tookinthe1960s.Commentingonphotography,Barthessaid: In orderto movefromtherealityto itsphotographit isin nowaynecessaryto divide upthisrealityintounitsandto constitutetheseunitsassigns,substantiallydifferent fromtheobjecttheycommunicate....Certainly,theimageis nottherealitybutat leastit is its perfectanalogonandit isexactlythisanalogicalperfectionwhich,to commonsense,definesthe photograph.Thuscanbe seenthe specialstatusof the photographicimage:.it is a messagewithouta code. (Barthes.1977:l-7) !-:*I i
  39. 39. Thesemiotic landscape . 25 Q fig f.Z Birdin tree(Bruna,t988) Andheextendsthisargumentto otherpictorialmodes,albeitwitha qualification: Are thereothermessageswithouta code?At first sight,yes:preciselythe whole rangeof analogicalreproductionsof reality- drawings,paintings,cinema,theatre. However,eachof thosemessagesdevelopsin an immediateand obvious supplementarymessage... whichis what is commonlycalledthe style reoroduction. (Barthes,1977:17) Thepictureof the bird in thetree,onthe otherhand,is muchlessnaturalistic,muchless detailedandmuchsimplerthanthepictureofthebathroom.It isstylizedandconventional, andquiteclearlya'coded'image.No depth,no shadows,no subtlenuancesof colour: everythingisplainandboldandsimple.Andthestructureoftheimage,withitsonecentral andfourmarginalimages,doesnotimitateanythingintherealworld.It isa conventional visualarrangement,basedona visualcode.Asa resultthecomponentsof thewholestand out asseparate,distinctunits,andthe picturewouldseemquiteamenableto constituent analysis.Thisis not just a matterof style:the structureof this picturecouldalsobe realizedin moredetailedstyles.Bruna'sbookdatesfrom 1953,well beforethe era of computer'imaging',butthepictureofthebirdinthetreecouldhavebeencomposedwitha computer,aligningready-madesimpleiconsina compositionalconfiguration- it isinfact quitesimilarto thecomputer-drawndinnerinvitationinfiqure1.3. way a of the
  40. 40. Thesemiotic landscape @ fig f.: Computer-drawndinnerinyitation Second,the pictureof thebathroomis part of a two-pagelayout,andaccompaniedby words.Languagecomesfirst,authoritativelyimposingmeaningon the image,turningit intoa typicalinstanceof a bathroombymeansof thegenericlabel'Bath'.Asa resultthe picturecouldbe replacedby otherimagesof bathroomswithoutmuchlossof meaning (oneverbaltext,manyimages,manypossibleillustrations).Herelanguageis general, bestowingsimilarityandorderon the diverse,heterogeneousworldof images.Thusthe bookpresents,ontheonehand,an'uncoded',naturalisticrepresentation('theworldasit is'- empirical,factual,specific)and,on the otherhand,a specific,authoritativelypre- scribedwayof readingthis'uncoded'naturalisticpicture.Wewillshowlaterthat,contrary to what Barthesandothersarguedinthe 1960s,picturesof thiskindarealsostructured, whetherthey are photographs,drawings,paintingsor otherkindsof pictures.For the moment,however,theimportantpointisthattheyarenotusuallyinterpretedassuch,that awarenessof thestructurednessof imagesof thiskindis,in oursociety,suppressedandnot oartof tcommonsense'. In DickBruna's0n My Walk,bycontrast,therearenowordsto authoritativelyimpose meaningon the image,andthe imageis no longeran illustration:the imagecarriesthe meaning,thewordscomesecond.Parentswhoreadthisbookwiththeirchildrencouldall tell a differentstory,couldevenusedifferentlanguages(oneimage,manyverbaltexts). irrulNer{1'
  41. 41. Thesemiotic landscape Theworldof '0ne image,manydifferentverbaltexts'('commentariesi)imposesa new modeof controlovermeaning,and turnsthe image,formerlya recordof natureor a playgroundfor childrenandartists,intoa morepowerful,but alsomorerigorouslycon- trolledandcodifiedpubliclanguage,whileit giveslanguage,formerlycloselypolicedin manysocialinstitutions,a moreprivateandlesscontrolled,butalsolesspowerful,status. The'readings'whichparentsproducewhentheyread0n My Walkwiththeirchildrenmay all bedifferent,yetthesedifferentreadingswill necessarilyhavecommonelements,deriv- ing from their commonbasis- the elementsincludedin the image,andthe way these elementsarecompositionally broughttogether. Whateverstoryparentswilltellaboutthepagewiththebirdinthetree,it will necessar- ily haveto bea storythat createsa relationbetween,for instance,birdsandaeroplanes (natureandtechnology)andbirdsandcats(preyandpredator).It will alsohaveto bea storyinwhichthebird,safelyin itstree,isthecentralcharacter,literallyandfiguratively.In howmanywayscancatsandbirdsberelated?Notthat many,at leastnot if oneassumes that bookslike0n My Walkserveto introducechildrento theworldaroundthem,rather thanto the possibleworldsof fantasiesandutopias.Catscan'hunt,,'torture,,.kill,and'eat' birds.Birdscan'escape'catsor fail to do so.Thereare not that manystoriesto choosefrom.0ntheotherhand,parentsandtheirchildrencanchoosetheorderin which theywantto dealwiththevariouselements:the pageis'non-linear,.It doesnot imposea sequentialstructure.Andtheycanchoosewhetherto tellthestoryof thebirdandthecat as a politicaistory,a story of powerfulpredatorscomingfrom anothercontinentand nativebirdskilledandthreatenedwithextinction(asmightbedone,for instance,in Aus- tralia),or asa storythat legitimizesthesurvivalof thefittest.Thestoryof thebirdandthe aeroplane,similarly,maybetoldfroman environmentalistpointof view,or asa storyof evolutionarytriumphsand humantechnologicalprogress.Evenwheresuchdiscourses are not explicitlyinvoked,theywill still communicatethemsejvesto childrenthrouqhthe parents'attitudestowardsthecharactersandtheactions. Not onlytheelementson the individualpages,but alsothe pagesthemselvesmustbe broughtin relationto eachother.Thebookas a wholemustbe readableas a conerent sequence.Thisispromptedbythetitle(0n My Walb aswellasbythepictureonthefront cover,whichshowsall the elementstogether.We haveinvestigatedthis a littlefurtherin connectionwith anotherbool<in the Brunaseries,0nthe Farm.Thisbookcontainsthe followingcentralpictures:house,farmer,cat,dog,appletree,rooster,lamb,cow.Listingthe waysinwhichthesepicturescanplausiblybelinkedto eachother,wefoundthatsome(e.g. theappletreeandthehouse)canonlybelinkedin spatial,locativeterms(e.g.theapple treeisnextto thehouse).Others(e.g.theanimalsandthehouse)canberelatedbyverbsof'dwelling'(e.9.thecowliesundertheappletree)or by theverbsof 'motion,(e.g.thecat climbsuptheappletree).Twoof theanimals(thecatandthedog)canrelateto theother animalsandto eachotherby meansof antagonisticor co-operativeactions(e.g.thedog barksat the cow;the dog leadsthe sheep).only the farmercan relateto all the other elementsin an agentiveway.He canbuythem,ownthem,buildthem,growthem,keep them,raisethem,harvestthem,shearthem,slaughterthem,andso on.In otherwords, whateverwaythe parentsreadthesepictures,theywill, in the end,haveto dealwith the

×