- The 'What' of multiliteracies - Analysing multimodality in literacy

1. CULTURAL LITERACY SKILLS
IN CONTEXT AND
TECHNOLOGY
DR. YEE BEE CHOO
INSTITUTE OF TEACHER EDUCATION TUN HUSSEIN ONN CAMPUS
(IPGKTHO)
TOPIC 2c: Multimodality and Literacy

2. a.
• Multiculturalism
b.
• Applying Cultural Literacy Skills in Context and
Technology
c.
• Multimodality and Literacy
Course Contents

3. Course Contents
c. Multimodality and Literacy
• Multimodality (written, oral, visual, audio,
tactile, gestural, spatial)
The ‘What’ of
Multiliteracies
• 4 resources model (code breaking, meaning
making, text user, text analyst)
• Recognition, reproduction, reflection
Analysing Multimodality
in Literacy

4. Multiliteracies
Multimodal
Digital Literacy
Media Literacy
Visual Literacy
Information
Literacy
Critical
Literacy

5. What meaning does this picture have?

6. How about now?

7. What meaning does this picture have?

8. How about now?

9. Multiliteracies during the Pandemic

10. The ‘What’ of Multiliteracies
 Multimodality refers to the interplay
between different representational
modes, for instance, between images
and written/spoken word.
 Multimodal representations mediate
the sociocultural ways in which these
modes are combined in the
communication process.
(Kress & Van Leeuwen 2001, p. 20)

11. The ‘What’ of Multiliteracies
What is a mode?
 Mode is a socially shaped and culturally given
resource for making meaning (Kress, 2010).
 In fact, it is now no longer possible to
understand language and its uses without
understanding the effect of all modes of
communication that are co-present any text
(Kress, 2010)

12. The ‘What’ of Multiliteracies
 Mode – signs, sound, graphic
material, print
 Media – the manner of dissemination
 A traffic sign as the medium of
communication and the red border
and the image inside it as the mode.
(Kress, G., 1997)

13. Multimodality
Multimodalit
y
Written
Oral
Visual
Audio
Tactile
Gestural
Spatial

14. Multimodality
Written
• Written
words
• Use of
vocabulary
and
grammar
• Handwriting
Oral
• Spoken
words
• Live or
recorded
speech
Visual
• Still Images
• Moving
images
• 3D objects
• Colours
Audio
• Music
• Soundtrack
• Narration
• Use of
volume
• Pitch and
rhythm
https://newlearningonline.com/learning-by-
design/multimodality

15. Multimodality
Tactile
• Touch
• Smell
• Taste
Gestural
• Gestures
• Facial
expressions
• Body
language
Spatial
• Spacing
• Layout and
organisation
of objects
and space
https://newlearningonline.com/learning-by-
design/multimodality

16. Tutorial 2 (v)
Group Work
1. Watch the video of your drama performance which you
submitted for the coursework TSL1064.
2. Discuss the modes used in the video.
3. List the modes in a storyboard.
4. Suggest how you can improve the video presentation.

17. Tutorial 2 (v)
Sample of Storyboard
Scene
No
Descriptions Textual Audio Image Video
1
2
Storyboard for Multimedia Presentation
Project Title :
Designers :
Suggestions:
1.
2.

18. Tutorial 2 (vi)
Group Work (Coursework Task 2)
1. In a group of three, collectively identify an issue (geographical, political, social
environmental or others) that you feel strongly about.
2. Identify and select ONE text each that addresses the issue above. The text should
represent different perspectives of ideas in relation to the issue stated. The text can
be a/an novel, article, song, poetry, script, PPT etc.
3. Cross examine the selected texts and choose ONE text (that you have agreed upon
collectively) and identify the factors that influence you as a reader.
4. Keep in mind the BIG IDEA: Teachers as Agents of Social Transformation.
5. Plan a storyboard for a 20-minute multimedia presentation.

19. Analysing Multimodality in Literacy
Text Decoder Text User
Text Participant Text Analyst
4 Resources Model
(Freebody & Luke, 1990)

20. Analysing Multimodality in Literacy
1. Text Decoder (Code Breaking)
• Breaking the code of language is about
recognising and using the fundamental
features and architecture of written texts
including: alphabetic knowledge, sounds in
words, spelling, conventions and patterns of
sentence structure and text.
Text decoders:
• know the relationship between the spoken
and written language
• interpret graphic symbols and their contexts
of use.
https://www.education.vic.gov.au/school/teachers/teachin
gresources/discipline/english/literacy/readingviewing/Pag
es/fourres.aspx

21. Analysing Multimodality in Literacy
2. Text Participant (Participate in Meaning)
• Participating in the meaning of text
involves understanding and composing
meaningful written, visual and spoken texts
from within the meaning systems of
particular cultures, institutions, families,
communities, and nation-states.
Text participants:
• know the meaning patterns operating in
written texts
• make literal and inferential meanings of
texts.
https://www.education.vic.gov.au/school/teachers/teac
hingresources/discipline/english/literacy/readingviewi
ng/Pages/fourres.aspx
Your English is quite
fluent.
No, no, my English
is quite poor.

22. Analysing Multimodality in Literacy
3. Text User (Use texts functionally)
 Using texts functionally is about negotiating the social
relations around texts.
 It is about knowing about and acting on the different
cultural and social functions that various texts perform
both inside and outside school and knowing that these
functions shape the way texts are structured, their tone,
their degree of formality and their sequence of
components.
Text users:
 know and use social and cultural functions of reading
and writing practices
 understand text types are used for particular purposes in
and out of school.
https://www.education.vic.gov.au/school/teachers/te
achingresources/discipline/english/literacy/readingvi
ewing/Pages/fourres.aspx

23. Analysing Multimodality in Literacy
4. Text Analyst (Critically Analyse and Transform Texts)
 Critically analysing and transforming texts involves understanding and acting on the knowledge that
texts are created for specific purposes, for example, to entertain, explain or influence.
 Therefore, it is important students understand that texts are not neutral, that they represent particular
views and silence other points of view, and that their designs and discourses can be critiqued and
redesigned, in novel and hybrid ways.
Text analysts:
 understand that texts are crafted according to the values, views and interests of the writer or creator.
 know that texts position readers differently, and both constrain and influence them.
https://www.education.vic.gov.au/school/teachers/teachin
gresources/discipline/english/literacy/readingviewing/Pag
es/fourres.aspx

24. Analysing Multimodality in Literacy
https://www.youtube.com/watch?v=X4y
Ryivequg
Watch the video and discuss it using the
model:

25. Analysing Multimodality in Literacy
Recognition Reflection Reproduction
Critical Literacy
(Unsworth, 2001)

26. Analysing Multimodality in Literacy
1. Recognition literacy involves learning to recognise and produce the verbal, visual and
electronic codes that are used to construct and communicate meanings. It can also
refer to the literacy practices that are very familiar to members of a culture as they
are ubiquitous and integral to common experiences of everyday life.
2. Reproduction literacy involves understanding and producing the conventional visual
and verbal text forms that construct and communicate the established systematic
knowledge of cultural institutions.
3. Reflection literacy necessitates an understanding that all social practices, and hence
all literacies, are socially constructed.
https://www.researchgate.net/publication/247701560_Teaching_Multiliteracies_Across
_the_Curriculum

27. Tutorial 2 (vii)
Individual
1. Watch one of your friend’s video of drama performance which was
submitted for the coursework TSL1064
2. Analyse the video using the models:
 4 Resources Model (Freebody & Luke, 1990)
 Recognition, Reproduction, Reflection (Unsworth, 2001)
3. Give suggestions on how to make the video presentation better.

28. Tutorial 2 (viii)
Group Work (Coursework Task 2)
1. Based on the storyboard in Tutorial (vi), design a 20-minute
multimedia presentation based on multiple perspectives.
2. Submit your texts (a/an novel, article, song, poetry, script, PPT
etc.) a week before presentation date (15 August 2021).
3. Keep in mind the BIG IDEA: Teachers as Agents of Social
Transformation.

29. References
 Freebody, P., & Luke, A. (1990). Literacies programs: Debates and demands in
cultural context. Prospect: Australian Journal of TESOL,5(7), 7–16.
 Kress, G. (2010).Multimodality: A social semiotic approach to contemporary
communication. London: Routledge.
 Kress, G., & van Leeuwen, T. (2001).Multimodal discourse: The modes and
media of contemporary communication. London: Edward Arnold.
 Unsworth, L. (2001). Teaching multiliteracies across the curriculum: Changing
contexts of text and image in classroom practice. Open University Press.