The document discusses the concept of multimodality and its impact on reading, writing, and thinking in the context of technological convergence and digital culture. It emphasizes that communication has always been multimodal, incorporating various modes such as text, image, and sound, and highlights the necessity for multiliteracies in education to accommodate diverse forms of communication. Additionally, it examines the political implications of a multimodal approach that challenges traditional literacy models.

1. THE MULTIMODAL TURNTHE MULTIMODAL TURN
How Technology is Changing the Way We
Read, Write & Think
ISP “Pbro. Antonio Sáenz”
Primeras Jornadas de Lenguas
Prof. Lic. Mariana Ferrarelli
June 2013

2. Overview
•Multimodality: Mode /
Affordances /
Technological Convergence
•Digital & Oral Culture
•Sample text
•Multiliteracies
•Political Implications

3. MultimodalityMultimodality
Social Semiotics
Signs Context
Meaning made up
of different modes

4. MODESocially shaped and
culturally given
semiotic resources
for making meaning

5. Words
Image
Colour
MultimodalityMultimodality

6. Words name
Images show
Colours highlight
(Kress, 2003)
Combination of a
variety of modes
or languages
Perceptual
experience
MultimodalityMultimodality
AFFORDANCES
Potentials for making meaning (Kress, 2010)

7. Multimodality Multimediality
Deals with how information is presented
Text Book/Page
Image Photography
Moving Image Cinema/ TV
Sound Radio/CD
MultimodalityMultimodality

8. TechnologicalTechnological ConvergenceConvergence
All modes can now be conveyed through
only one medium:
the screen

9. TechnologicalTechnological ConvergenceConvergence
Watch TV / movie
Read the newspaper
Listen to the radio/music
Take pictures
Record videos
Send e-mails
Facebook & Twitter
Search the web
Diary - alarm clock – calculator
– voice recorder
Take notes

10. Oral /Oral / PrintPrint / Digital Culture/ Digital Culture
Multimodality is not new.
Human communication is (and has always
been) multimodal.

12. Oral Culture
ORALITY Words
Gestures & facial expressions
Eye contact
Proxemics
Kinesics
Intonation
Volume
Oral /Oral / PrintPrint / Digital Culture/ Digital Culture

13. Oral Culture
WORDS one way of creating meaning
(among others)
Oral Culture
WORDS one way of creating meaning
(among others)
Oral /Oral / PrintPrint / Digital Culture/ Digital Culture
Gestures & facial
expressions
Eye contact
Proxemics
Kinesics
Intonation
Volume

14. Print Culture
one way of creating meaning WORDS
Oral /Oral / PrintPrint / Digital Culture/ Digital Culture
Words
Sentences
Paragraphs
Pages
Books

15. Oral /Oral / PrintPrint / Digital Culture/ Digital Culture
Digital Culture
WORDS one way of creating meaning
(among others)
Digital Culture
WORDS one way of creating meaning
(among others)
Colour
Image
Animation
Sound
Music
Moving image
Layout

16. Oral /Oral / PrintPrint / Digital Culture/ Digital Culture

17. Reading & Writing
No longer associated with mechanical
literacy (pencil & paper)
Digital CultureDigital Culture
Keyboard + Mouse + Screen
(Cassany, 2002)

18. Thinking
New cognitive skills
The ability to search for & deal with huge
loads of information efficiently.
Select
Discern
Connect
Create
Digital CultureDigital Culture

19. Flight Paths
A networked novel by Kate Pullinger, Chris Joseph
and participants.
Chapter 5: Paths Crossing.
http://www.flightpaths.net/stories/pathscrossing.html
Image
Sound
Text

20. IMAGE:
What does red suggest?
What is the effect of the blurred
image of the car?
What does the button in Arabic say
about the story/ chatacters/ setting?
How do all these elements
contribute to the creation of a
mysterious atmosphere?

21. SOUND:
What does the eerie music imply?
Where does the story take place?
What does the sound of birds say
about this?
How is ‘silence’ represented in the
story?

22. TEXT:
Who is ‘he’? Does this ambiguity
build up expectation?
Why does the narrator feel surprised
when ‘he’ speaks ‘perfect English’?
Is there any repetition of words? Any
parallel structures? (‘waited for
people’, waited for sirens’)
What is the effect of alliteration/
consonance/ similes?
Are there any sensory images?

23. Multimodal Texts
Andy Campbell:
Inside: A Journal of Dreams (2001)
Nightingale’s Playground (2010)
http://www.dreamingmethods.com/portfolio.html
Chris Joseph:
Bully for You (2008)
Tube Lines (2013)
http://www.chrisjoseph.org/
http://narrativa-postparentetica.blogspot.com.ar/

24. MultiliteraciesMultiliteracies
Social Semiotics Multimodality
Education Multiliteracies
Abilities required to read, write, respond to
and produce multimodal texts.

25. A multimodal approach to education
Draws on a wider variety of texts
Moves the focus from a language to other
forms of communication
Allows for diversity & differentiation
Rg & Wg increasingly screen-based
Author as fluid and transitory role
AFFORDANCES: potentialities &
limitations

26. Political Implications
Multimodal approach
We challenge the dominant model of
literacy written language
We allow for differentiation & diversity
POLITICAL CHOICE

27. SourcesSources andand CreditsCredits
Gee, J.P. (2003), What Video Games Have to Teach Us about Learning and Literacy,
Palgrave Macmillan, New York.
Gee, J.P. (2005) Why Video Games are Good for Your Soul: Pleasure and Learning,
Common Ground, Melbourne.
Cope, B. and Kalantzis, M. (2004), ‘Text-made text’, E-Learning 1, 198–282.
Jewitt, C. (2009) The Routledge Handbook of Multimodal Analysis, Routledge, London.
Kress, G. (1990) Linguistic Process and Sociocultural Change, Oxford University Press,
Oxford, England.
Kress, G.(2003) Literacy in the New Media Age, Routledge, London.
Kress, G. (2010) Multimodality: A Social Semiotic Approach to Contemporary
Communication, Routledge, London.
Kress, G. and van Leeuwen, T.(1996) Reading Images: The Grammar of Visual Design,
Routledge, London.
Lauer, C. (2009) Contending with Terms: “Multimodal” and “Multimedia” in the
Academic and Public Spheres, Computers and Composition 26 (2009) 225–239
Mavers, D. (2011) Children’s Drawing and Writing: The Remarkable in the
Unremarkable, New York, Routledge.
New London Group.:(1996) ‘A pedagogy of multiliteracies: Designing social
futures’, Harvard Educational Review 66, 60–92.

28. Oliver, M. (2005)The problem with affordance’The E-Learning Journal 2 (4): 402-
413
Public Schools NSW (2012) Using digital and multimodal texts K-6, Department of
Education and Communities. Available at: www.tale.edu.au/. Retrieved: March
2013.
Siege, M. (2006) Rereading the Signs: Multimodal Transformations in the Field of
Literacy Education; Language Arts, Vol. 84, No. 1, September 2006.
Snyder, I. (2004) Alfabetismos digitales. Comunicación, innovación y educación en la
era electrónica, Colección Aulae, número 2. Ediciones Aljibe, Enseñanza
Abierta de Andalucía y Consorcio Fernando de los Ríos.
Stein, P. (2008) Multimodal Pedagogies in Diverse Classrooms: Representation, Rights
and Resources, Routledge, London.
Unsworth , L. (2001) Teaching multiliteracies across the curriculum, USA, Open
University Press.
http://www.dreamingmethods.com/portfolio.html; allposters.com;
howfreeisfreedom.wordpress.com;
http://www.chrisjoseph.org/; www.freefoto.com
pta.chaparral.groupfusion.net; www.pcadvisor.co.uk
brucethink.wordpress.com; www.123rf.com ; gamespot.com
commons.wikimedia.org ; neonumbrella.net ;
discoscicatriz.blogspot.com ; diez.hn ; https://encrypted-tbn3.gstatic.com/images;
hubpages.com; javiercordero.com; pidopaso.blogspot.com;

29. My special thanx to Ma.Cristina Llorente &
Patricia Onganía…!
mariana.ferrarelli@gmail.com