Multimodal instruction uses verbal and non-verbal modes like images, video and audio to represent content. It draws on multiple literacies and modalities like visual and auditory. This approach supports diverse learners by allowing them to demonstrate knowledge through their strengths. It also increases engagement and knowledge construction by integrating representations. While implementation challenges include assessment and access to resources, providing choice and scaffolding in multimodal projects can increase student motivation and creativity.

1. Multimodal Instruction
to Support Diverse Learners and Foster
Multiliteracies
Rhiannon Vining

2. We’ve probably heard of multimodal
instruction,
but what does that mean?

3. Let’s start from the beginning…
Creating and Communicating Meaning
• Mode
• “Code used to represent information” (Moreno & Mayer, 2007, p.310)
• “A fully semiotically articulated means of representation and
communication.” (Stein & Newfield, 2006, p.2)
• Verbal (printed or spoken words)
• Non-Verbal (illustrations, photos, video)
• Modalities
• “Sense receptors used to receive information” (Moreno & Mayer, 2007,
p.310)
• Auditory (through the ears)
• Visual (through the eyes)

4. Multimodality
• Multimodal Learning Environments “use two different modes to represent the
content knowledge: verbal and non-verbal” (Moreno & Mayer, 2007, p.310)
• “The most effective designs for learning adapt to include a variety of media,
combinations of modalities, levels of interactivity, learner characteristics, and
pedagogy based on a complex set of circumstances.” (Metiri, 2008, p.14)

5. image source: http://primaryblog.wordpress.com/2012/12/17/new-literacies-and-their-affordances/

6. Semiotics
• Semiotic resources are “resources that people draw on and configure in
specific moments and places to represent events and relations.” (Loerts,
2010, p.27)
• “Semiotic resources of modes are shaped by how people use them to make
meaning—the social functions that they are put to. Halliday classified these
social functions into three meta-functions, three different kinds of meaning.
Every sign simultaneously tells us something about ‘the world’ (ideational
meaning), positions us in relation to someone or something (interpersonal
meaning) and produces a structured text (textual meaning). Halliday and
others explored how these three kinds of meaning are ‘held by’ the grammar
and elements of language” (Jewitt, 2005, p.311).

7. Social Semiotics
• Multimodal social semiotics “concerns itself with how human beings use
different modes of communication, such as speech, writing, image, gesture
and sound, as resources to represent or make meaning in the social world....
Teaching and learning are multimodal: they happen mainly through the modes
of speech, writing, action, gesture, image and space, all of which work in
different ways with different effects, to create multi-layered, communicational
ensembles.” (Stein & Newfield, 2006, p.2)
• Social semiotic theory of communication “emphasizes the social aspect of
communication, where people are active designers of meaning.” (Loerts,
2010, p.27)

8. Doesn’t technology have something to do
with it?

9. Technology as a Tool
• Achievement is more strongly correlated with pedagogy than use of media. It
is not whether or not we use technology, but how we use it. (Metiri, 2008)
• Makes available dynamic representations
• Audio, video, animations, dynamically changing graphs/tables, interactive
visuals (van der Meij & de Jong, 2006)
• Facilitates interactivity
• Responsive to learner’s actions during learning (Moreno & Mayer, 2007)
• As students create digital media they begin to think more critically about it.
(Cooper, Lockyer, & Brown, 2013)

10. Technology as a Literacy
• “The question before us is how to redesign school literacy so that youth can develop
the broad repertoire of literacy knowledge and practices they will need to
successfully participate as citizens of local and global communities characterized by
constant change and increasing diversity.” (Siegel, 2012, p.672)
• “Today’s learners require an expanded knowledge base and set of skills to support
their participation in our evolving digital world. This digital world requires learners to
construct knowledge in nonlinear environemnts using a range of tools. These
changing learning needs have implications for how we define literacy and the literacy
practices necessary to ensure education is relevant, reflecting a diversity of
contexts.” (Cooper, Lockyer, & Brown, 2013)
• Multimodal learning clubs as “a classroom strategy that supports students’
acquisition of new knowledge by pairing digital tools with requisite literacy
strategies.” (Casey, 2012, p.39)

11. OK. So why is multimodality so great?

12. Knowledge Construction
• Information Acquisition: Information is added to learner’s memory.
• Knowledge Construction: “Learner is a sense-maker who works to select,
organize, and integrate new information with existing knowledge.” (Moreno &
Mayer, 2007, p.312)
• Actively integrating multiple representations improves learning outcomes (van
der Meij & de Jong, 2006)

13. Engagement
• When students are presented with multimodal options they are provided the
opportunity to become “agents of their own meaning making. When
communication is agentive, students take greater responsibility for their
learning which in turn makes them more motivated to engage in further
literacy practices that support new ways of learning that are important to
them, their school, and their community.” (Loerts, 2010, p.27)
• Particularly motivating for “students who struggle due to cultural, economic,
or gender differences.” (Loerts, 2010, p.30)

14. Diverse Learners
• “One size does not fit all learners.” Optimal instructional design depends on the
content, the context, and the learner. (Metiri, 2008, p.8)
• “Many literacy educators have tapped multimodality because it often recasts
students who are labeled ‘at risk’ students - whether English-language learners,
low-achieving or reluctant readers, deaf, or learning disabled - as students ‘of
promise’.... Creating opportunities for these students to draw on all the resources
in their semiotic toolkits allowed them to demonstrate knowledge and critical
understandings far beyond that which they displayed when language was the
sole mode of meaning making and communication.” (Siegel, 2012, p.674)
• Diverse Learning Styles & Multiple Intelligences
• Socio-Cultural Context

15. To continue, we need to tackle another
concept.

16. Multiliteracies
• “The concept of ‘multiliteracies’... has been taken up by numerous scholars to
challenge the idea of a singular, universal literacy restricted to monolingual,
monocultural and rule-governed standard forms of language.” (Stein &
Newfield, 2006, p.1)
• “Traditional definitions of literacy have focused on reading and writing of
printed materials to communicate.... By drawing on a multiliteracies
perspective that naturally embraces multimodal pedagogy, teachers can
become motivated to embrace the tools needed to equip students with
communicational options.” (Loerts, 2010, p.24)

17. Literacies in Critical Theory
• Communicational ensembles “are never neutral: they are meaning-bearing
signs which are produced in particular contexts of power, culture and history.”
(Stein & Newfield, 2006, p.2)
• “Issues of power, policy and choice of medium are clearly part of the policypractice nexus in education and impact profoundly on how students learn and
construct their identities. They also affect curriculum, pedagogy and
assessment practices” (Stein & Newfield, 2006, p.3)

18. Social Justice, Equity, & Democracy
• “A pedagogy of multiliteracies is concerned with building learning conditions
leading to full and equitable social participation.... Its basic principle of plurality
opens the space for indigenous, local epistemologies, languages and literatures
to co-exist alongside standard English and canonical English literature and
culture.” (Stein & Newfield, 2006, p.5)
• “Education is not about limiting or controlling students’ communicational options,
but instead, should be representative of all ‘design’ processes, linking them to
various cultural and linguistic diversities.” (Loerts, 2010, p.25)
• “The forms of representation through which students are permitted to make their
meanings are a critical component in constructing classrooms as hybrid,
democratic spaces which value diversity and difference. Positively valuing social
difference as a resource is an essential component in building democratic
culture.” (Stein quoted in Loerts, 2010, p.28)

19. Empowerment
• Helps students develop confidence in own ability to make meaning (Loerts,
2010)
• “Even students with limited English proficiency, who are often labelled as
‘outsiders’ benefit from language arts activities that include multiple modes of
representation because it more easily allows students’ personal voices to
prevail. Through webcasts, gestures, play, music, speech, drawing and
writing, students [are] empowered to communicate in ways that [minimize] the
limitations that language alone would have afforded.” (Loerts, 2010, p.29)
• Multimodality increases the semiotic resources with which students can
represent their learning. This has the potential to afford them confidence and
previously deemed unavailable career options. (Loerts, 2010, p.29)

20. Cool. So are there any drawbacks?

21. Challenges - Implementation
• Inequitable access to resources. (Siegel, 2012, p.674)
• Novices may have problems translating between various representations. (van
der Meij & de Jong, 2006)
• Teachers need to be fluent in literacies to provide pretraining, give explanatory
feedback, and guide reflection and cognitive processing. (Moreno & Mayer,
2007)
• “A major tension and difficulty is developing a viable metalanguage for
teaching and assessing multimodal texts.” (Stein & Newfield, 2006, p.17)
• “From a practical point of view, teachers have been comfortable working
within the traditional view of literacy which privileges language, in part
because accountability standards are traditionally based, and because their
perceived lack of knowledge in forms of multimodal communication has been
seen as an obstacle. (Loerts, 2010, p.25)

22. Challenges - Assessment
• We hold the power. Students are empowered only so far as the socio-cultural
context validates their contributions and literacies.
• “Pedagogies which work with multiliteracies and multimodality face the urgent
task of reconceptualizing assessment criteria and practices. Without reliable
assessment tools, multimodal instructional practices ... carry little weight and
authority where it matters.” (Stein & Newfield, 2006, p.16)
• Standardized Testing
• Post-Secondary Expectations

23. So how does this look in the classroom?

24. Implementing
• What are the learning objectives?
• e.g. Psychology
• PLOs (e.g. “demonstrate…” - not specific as to how)
• Post-Secondary preparedness or life skills?
• Academic writing an objective in itself? (Or simply a mode of instruction?)
• What are possible strategies for achieving the learning objectives?
• How can students show their learning? Appropriate assessments?
• What resources are available? Differentiated Instruction?
• Room for student choice?

25. Fashion Design Unit
• https://www.dropbox.com/s/ip06nrzfdb30dcq/Unit%20-%20Fashion%20design%20intro.docx
• Relevant PLOs (Textiles, grade 12)
• A6 explain and use appropriate textile terminology
• B1 demonstrate an understanding of preconstruction procedures
• C1 create textile items incorporating the elements and principles of design
• D1 investigate historical, political, and cultural influences on fashion
• Overall Objective: Make a mini fashion design portfolio. Students will present inspiration portfolios during
informal peer-feedback sessions. Students will present final portfolio with complete looks to the class.
Presentation time will be as needed.
• Students several choices of how to prepare and present their projects.
• Projects can be done individually or collaboratively. 3 looks per student.
• Terminology, construction procedures, elements and principles of design, and historical, political, and
cultural influences can be incorporated into visual or oral presentation. (Written definitions and
reflections, presenting terminology with appropriate images, or speaking to images.)
• Project can be done in any way that meets the requirements. (Journal, scrapbook, poster, power point
presentation, digital mood board, etc.)
• Several classes will be devoted to developing a variety of skills. (Drawing, designing using croquis,
watercolour pencils, watercolour painting, using online mood boards, searching images online, etc.)
• In class curriculum will be supported by access to on-line resources via a teaching blog.

26. Measuring
• Assessment for learning at beginning of unit
• How do you like to learn?
• Prompts: overhead notes, lectures, reading, discussion, activities, research projects
• When do you feel most involved in you own learning?
• Prompts: during discussion, when reading, after exercising, right before bedtime, when
working by yourself, when working with others
• Which way of expressing yourself do you feel best represents your knowledge and how hard you
have worked?
• Prompts: tests, essays, presentations, interviews / oral exams, frequent but short reflections
(in a journal or blog), creative art-based projects
• Do you think of yourself as creative? (Why or why not?)
• Are you interested in fashion design? What do you like or not like about it?
• What are you hoping to learn or explore in this project?
• Project assessment after unit
• Students will reflect on the same key points.
• e.g. Did you feel creative woking on this project? What were your favourite parts?

27. What can we expect to find?

28. Expectations
• Students will tend to prefer modes & literacies they have employed in the
past.
• Students will tend to prefer similar modes & literacies to each other
• Students will explore alternative modes & literacies more where teacher
guidance (scaffolding, instruction, feedback) is provided.
• Providing choice and scaffolding for mediums of expression will increase
motivation and creative engagement.
• Actively integrating various modes of communication will deepen learning.

29. References
•Benson, S. (2008). A restart of what language arts is: Bringing multimodal assignments into secondary language arts. Journal of
Advanced Academics, 19(4), 634-674.
•Casey, H. (2012). Multimodal learning clubs. Middle School Journal, 44(2), 39-48.
•Cooper, N., Lockyer, L., & Brown, I. (2013). Developing multiliteracies in a technology-mediated environment. Educational Media
International, 50(2), 93-107.
•Jewitt, C. (2005). Classrooms and the design of pedagogic discourse: a multimodal approach. Culture & Psychology, 11(3), 309320.
•Loerts, T. (2010). Multimodal literacy and its impact on student empowerment. English Quarterly Canada (Online),41(3/4), 24.
•Matusiak, K. K. (2013). Image and multimedia resources in an academic environment: A qualitative study of students' experiences
and literacy practices. Journal of the American Society for Information Science and Technology, 64(8), 1577-1589.
•Metiri Group. (2008). Multimodal learning through media: What the research says. Commissioned by Cisco Systems Inc. Retrieved
from: http://www.cisco.com/web/strategy/docs/education/Multimodal-Learning-Through-Media.pdf
•Moreno, R., & Mayer, R. (2007). Interactive multimodal learning environments. Educational Psychology Review, 19(3), 309-326.
•Siegel, M. (2012). New Times for Multimodality? Confronting the Accountability Culture. Journal Of Adolescent & Adult
Literacy, 55(8), 671-681.
•Stein, P., & Newfield, D. (2006). Multiliteracies and multimodality in English in education in Africa: Mapping the terrain. English
Studies in Africa, 49(1), 1-21.
•van der Meij, J., & de Jong, T. (2006). Supporting students' learning with multiple representations in a dynamic simulation-based
learning environment. Learning and Instruction, 16(3), 199-212. doi:10.1016/j.learninstruc.2006.03.007