Multimodal instruction uses verbal and non-verbal modes like images, video and words to represent content. It has benefits like increased engagement, supporting diverse learners, and allowing students to demonstrate knowledge in different ways. While challenges include inequitable access to resources and developing assessments, providing choice and scaffolding in multimodal projects can deepen learning. A fashion design unit was proposed that incorporates terminology, design principles and cultural influences through a student-chosen portfolio presentation format.

1. Multimodal Instruction
to Support Diverse Learners and Foster Multiliteracies
by Rhiannon Vining

2. We’ve heard of multimodal instruction,
but what does that mean?

3. Let’s start from the beginning…
Creating and Communicating Meaning
•A mode is a…
•“Code used to represent information” (Moreno & Mayer, 2007, p.310)
•“Fully semiotically articulated means of representation and
communication.” (Stein & Newfield, 2006, p.2)
•Verbal (printed or spoken words)
•Non-Verbal (illustrations, photos, video)
•Modalities are…
•“Sense receptors used to receive information” (Moreno & Mayer, 2007, p.310)
•Auditory (through the ears)
•Visual (through the eyes)

4. Multimodality
• Multimodal Learning Environments “use two different
modes to represent the content knowledge: verbal
and non-verbal” (Moreno & Mayer, 2007, p.310)
• “The most effective designs for learning adapt to
include a variety of media, combinations of
modalities, levels of interactivity, learner
characteristics, and pedagogy based on a
complex set of circumstances.” (Metiri, 2008, p.14)

5. image source: http://primaryblog.wordpress.com/2012/12/17/new-literacies-and-their-affordances/

6. Semiotics
• Semiotic resources are “resources that people draw
on and configure in specific moments and places to
represent events and relations.” (Loerts, 2010, p.27)
• “Semiotic resources of modes are shaped by how
people use them to make meaning - the social
functions that they are put to.” (Jewitt, 2005, p.311)
• These social functions are classified into three meta-
functions (three different kinds of meaning):
• Ideational meaning
• Interpersonal meaning
• Textual meaning
(Jewitt, 2005, p.311)

7. Social Semiotics
• Multimodal social semiotics “concerns itself with how
human beings use different modes of communication,
such as speech, writing, image, gesture and sound, as
resources to represent or make meaning in the social
world.... Teaching and learning are multimodal: they
happen mainly through the modes of speech, writing,
action, gesture, image and space, all of which work in
different ways with different effects, to create multi-
layered, communicational ensembles.” (Stein & Newfield, 2006, p.2)
• Social semiotic theory of communication “emphasizes
the social aspect of communication, where people are
active designers of meaning.” (Loerts, 2010, p.27)

8. Doesn’t technology have something to do with it?

9. Technology as a Tool
• Achievement is more strongly correlated with
pedagogy than use of media. It is not whether or
not we use technology, but how we use it. (Metiri, 2008)
• Makes available dynamic representations
• Audio, video, animations, dynamically changing
graphs/tables, interactive visuals (van der Meij & de Jong, 2006)
• Facilitates interactivity
• Responsive to learner’s actions during learning (Moreno
& Mayer, 2007)
• As students create digital media they begin to think
more critically about it. (Cooper, Lockyer, & Brown, 2013)

10. Technology as a Literacy
• “The question before us is how to redesign school
literacy so that youth can develop the broad
repertoire of literacy knowledge and practices they
will need to successfully participate as citizens of
local and global communities characterized by
constant change and increasing diversity.” (Siegel, 2012, p.672)
• Multimodal learning clubs as “a classroom strategy
that supports students’ acquisition of new
knowledge by pairing digital tools with requisite
literacy strategies.” (Casey, 2012, p.39)

11. “Today’s learners require an expanded knowledge
base and set of skills to support their participation in
our evolving digital world. This digital world requires
learners to construct knowledge in nonlinear
environments using a range of tools. These
changing learning needs have implications for how
we define literacy and the literacy practices
necessary to ensure education is relevant,
reflecting a diversity of contexts.”
(Cooper, Lockyer, & Brown, 2013)

12. OK. So why is multimodality so great?

13. Knowledge Construction
• Information Acquisition: Information is added to
learner’s memory.
• Knowledge Construction: “Learner is a sense-maker
who works to select, organize, and integrate new
information with existing knowledge.” (Moreno & Mayer, 2007, p.312)
• Actively integrating multiple representations
improves learning outcomes (van der Meij & de Jong, 2006)

14. Engagement
• When students are presented with multimodal
options they are provided the opportunity to
become “agents of their own meaning making.
When communication is agentive, students take
greater responsibility for their learning which in turn
makes them more motivated to engage in further
literacy practices that support new ways of learning
that are important to them, their school, and their
community.” (Loerts, 2010, p.27)
• Particularly motivating for “students who struggle
due to cultural, economic, or gender differences.”
(Loerts, 2010, p.30)

15. Diverse Learners
• “One size does not fit all learners.” Optimal
instructional design depends on the content, the
context, and the learner. (Metiri, 2008, p.8)
• Diverse Learning Styles & Multiple Intelligences
• Socio-Cultural Context

16. “Many literacy educators have tapped multimodality
because it often recasts students who are labeled ‘at
risk’ students - whether English-language learners,
low-achieving or reluctant readers, deaf, or learning
disabled - as students ‘of promise’.... Creating
opportunities for these students to draw on all the
resources in their semiotic toolkits allowed them to
demonstrate knowledge and critical understandings
far beyond that which they displayed when language
was the sole mode of meaning making and
communication.”
(Siegel, 2012, p.674)

17. To continue, we need to tackle another concept.

18. Multiliteracies
• “The concept of ‘multiliteracies’... has been taken
up by numerous scholars to challenge the idea of a
singular, universal literacy restricted to monolingual,
monocultural and rule-governed standard forms of
language.” (Stein & Newfield, 2006, p.1)
• “Traditional definitions of literacy have focused on
reading and writing of printed materials to
communicate.... By drawing on a multiliteracies
perspective that naturally embraces multimodal
pedagogy, teachers can become motivated to
embrace the tools needed to equip students with
communicational options.” (Loerts, 2010, p.24)

19. Literacies in Critical Theory
• Communicational ensembles “are never neutral:
they are meaning-bearing signs which are
produced in particular contexts of power, culture
and history.” (Stein & Newfield, 2006, p.2)
• “Issues of power, policy and choice of medium are
clearly part of the policy-practice nexus in
education and impact profoundly on how students
learn and construct their identities. They also affect
curriculum, pedagogy and assessment practices.”
(Stein & Newfield, 2006, p.3)

20. Social Justice, Equity, & Democracy
• “A pedagogy of multiliteracies is concerned with
building learning conditions leading to full and
equitable social participation.... Its basic principle of
plurality opens the space for indigenous, local
epistemologies, languages and literatures to co-exist
alongside standard English and canonical English
literature and culture.” (Stein & Newfield, 2006, p.5)
• “Education is not about limiting or controlling
students’ communicational options, but instead,
should be representative of all ‘design’ processes,
linking them to various cultural and linguistic
diversities.” (Loerts, 2010, p.25)

21. “The forms of representation through which students
are permitted to make their meanings are a critical
component in constructing classrooms as hybrid,
democratic spaces which value diversity and
difference. Positively valuing social difference as a
resource is an essential component in building
democratic culture.”
(Stein quoted in Loerts, 2010, p.28)

22. Empowerment
• Helps students develop confidence in own ability to
make meaning (Loerts, 2010)
• “Even students with limited English proficiency, who
are often labeled as ‘outsiders’ benefit from
language arts activities that include multiple modes
of representation because it more easily allows
students’ personal voices to prevail. Through
webcasts, gestures, play, music, speech, drawing
and writing, students [are] empowered to
communicate in ways that [minimize] the limitations
that language alone would have afforded.” (Loerts, 2010, p.29)

23. Multimodality increases the semiotic resources with
which students can represent their learning. This has
the potential to afford them confidence and
previously deemed unavailable career options.
(Loerts, 2010, p.29)

24. Cool. So are there any drawbacks?

25. Challenges - Implementation
• Inequitable access to resources. (Siegel, 2012, p.674)
• Novices may have problems translating between
various representations. (van der Meij & de Jong, 2006)
• Teachers need to be fluent in literacies to provide
pretraining, give explanatory feedback, and guide
reflection and cognitive processing. (Moreno & Mayer, 2007)
• “A major tension and difficulty is developing a
viable metalanguage for teaching and assessing
multimodal texts.” (Stein & Newfield, 2006, p.17)

26. “From a practical point of view, teachers have been
comfortable working within the traditional view of
literacy which privileges language, in part because
accountability standards are traditionally based, and
because their perceived lack of knowledge in forms of
multimodal communication has been seen as an
obstacle.”
(Loerts, 2010, p.25)

27. Challenges - Assessment
• We hold the power. Students are empowered only
so far as the socio-cultural context validates their
contributions and literacies.
• “Pedagogies which work with multiliteracies and
multimodality face the urgent task of
reconceptualizing assessment criteria and practices.
Without reliable assessment tools, multimodal
instructional practices ... carry little weight and
authority where it matters.” (Stein & Newfield, 2006, p.16)
• Standardized Testing
• Post-Secondary Expectations

28. So how does this look in the classroom?

29. Implementing
• What are the learning objectives?
• e.g. Psychology
• PLOs (e.g. “demonstrate…” - not specific as to
how)
• Post-Secondary preparedness or life skills?
• Academic writing an objective in itself? (Or simply
a mode of instruction?)
• What are possible strategies for achieving the learning
objectives?
• How can students show their learning? Appropriate
assessments?
• What resources are available? Differentiated Instruction?
• Room for student choice?

30. Fashion Design Unit
• Relevant PLOs (Textiles, grade 12)
• A6 explain and use appropriate textile terminology
• B1 demonstrate an understanding of preconstruction
procedures
• C1 create textile items incorporating the elements
and principles of design
• D1 investigate historical, political, and cultural
influences on fashion
• Performance Task: Make a mini fashion design portfolio.
Students will present inspiration portfolios during informal
peer-feedback sessions. Students will present final
portfolio with complete looks to the class. Presentation
time will be as needed.
• In-class curriculum will be supported by access to on-line
resources via a teaching blog.

31. • Students have several choices of how to prepare and
present their projects.
• Projects can be done individually or collaboratively. 3
looks per student.
• Terminology, construction procedures, elements and
principles of design, and historical, political, and
cultural influences can be incorporated into visual or
oral presentation. (Written definitions and reflections,
presenting terminology with appropriate images, or
speaking to images.)
• Project can be done in any way that meets the
requirements. (Journal, scrapbook, poster, power
point presentation, digital mood board, etc.)
• Several classes will be devoted to developing a
variety of skills. (Drawing, designing using croquis,
watercolour pencils, watercolour painting, using online
mood boards, searching images online, etc.)

32. Measuring
• Assessment for learning at beginning of unit
• How do you learn best?
• When do you feel most involved in you own learning?
• Which way of expressing yourself do you feel best
represents your knowledge and how hard you have
worked?
• Do you think of yourself as creative? (Why or why not?)
• Are you interested in fashion design? What do you like or
not like about it?
• What are you hoping to learn or explore in this project?
• Project assessment after unit
• Students will reflect on the same key points.
• e.g. Did you feel creative woking on this project? What
were your favourite parts?

33. What can we expect to find?

34. Expectations
• Students will tend to prefer modes & literacies they
have employed in the past.
• Students will tend to prefer similar modes & literacies
to each other
• Students will explore alternative modes & literacies
more where teacher guidance (scaffolding,
instruction, feedback) is provided.
• Providing choice and scaffolding for mediums of
expression will increase motivation and creative
engagement.
• Actively integrating various modes of
communication will deepen learning.

35. References
• Benson, S. (2008). A restart of what language arts is: Bringing multimodal assignments into
secondary language arts. Journal of Advanced Academics, 19(4), 634-674.
• Casey, H. (2012). Multimodal learning clubs. Middle School Journal, 44(2), 39-48.
• Cooper, N., Lockyer, L., & Brown, I. (2013). Developing multiliteracies in a technology-
mediated environment. Educational Media International, 50(2), 93-107.
• Jewitt, C. (2005). Classrooms and the design of pedagogic discourse: a multimodal
approach. Culture & Psychology, 11(3), 309-320.
• Loerts, T. (2010). Multimodal literacy and its impact on student empowerment. English
Quarterly Canada (Online),41(3/4), 24.
• Matusiak, K. K. (2013). Image and multimedia resources in an academic environment: A
qualitative study of students' experiences and literacy practices. Journal of the American
Society for Information Science and Technology, 64(8), 1577-1589.
• Metiri Group. (2008). Multimodal learning through media: What the research says.
Commissioned by Cisco Systems Inc. Retrieved from:
http://www.cisco.com/web/strategy/docs/education/Multimodal-Learning-Through-
Media.pdf
• Moreno, R., & Mayer, R. (2007). Interactive multimodal learning environments. Educational
Psychology Review, 19(3), 309-326.
• Siegel, M. (2012). New Times for Multimodality? Confronting the Accountability
Culture. Journal Of Adolescent & Adult Literacy, 55(8), 671-681.
• Stein, P., & Newfield, D. (2006). Multiliteracies and multimodality in English in education in
Africa: Mapping the terrain. English Studies in Africa, 49(1), 1-21.
• van der Meij, J., & de Jong, T. (2006). Supporting students' learning with multiple
representations in a dynamic simulation-based learning environment. Learning and
Instruction, 16(3), 199-212. doi:10.1016/j.learninstruc.2006.03.007