Multimodal Instruction and special populations

1. Using Multimodal Instruction to
Support ELAAcquisition in
Testing Subgroups
Maureen Kramanak
Teachers College; Columbia
University Spring 2013

2. What is Multimodality?
 Multimodality is an inter-disciplinary
approach that understands communication
and representation to be more than about
language. It has been developed over the
past decade to systematically address much-
debated questions about changes in society,
for instance in relation to new media and
technologies.

3. Approaches to Multimodality
 Multimodal approaches have provided
concepts, methods and a framework for the
collection and analysis of visual, aural,
embodied, and spatial aspects of interaction
and environments, and the relationships
between them.

4. Theoretical Assumption #1
 It focuses on analyzing and describing the
full repertoire of meaning-making resources
that people use (visual, spoken, gestures,
written, three-dimensional, and others,
depending on the domain of representation)
in different contexts, and on developing
means that show how these are organized to
make meaning.

5. Theoretical Assumption #2
 Multimodality assumes that resources are
socially shaped over time to become
meaning making resources that articulate
the (social, individual/affective) meanings
demanded by the requirements of different
communities.

6. Multimodality Assumption #3
 People orchestrate meaning through their
selection and configuration of modes,
foregrounding the significance of the
interaction between modes.

7. Core Concepts
 Mode
 Semiotic Resource
 Modal Affordance
 Inter-semiotic relations

8. Mode
 An outcome of the cultural shaping of a
material through its use in the daily social
interaction of people.

9. Semiotic Resources
 Come to display regularities through the
ways in which people use them and can be
thought of as the connection between
representational resources and what people
do with them.

10. Modal Affordance
 The material and the cultural aspects of
modes: what it is possible to express and
represent easily with a mode. It is a concept
connected to both the material as well as the
cultural and social historical use of a mode.

11. What is the Focus of Multimodal
Research?
 Multimodality can be used to build inventories of
the semiotic resources, organizing principles, and
cultural references that modes make available to
people in particular places and times: the actions,
materials and artifacts people communicate with.
This has included contributions to mapping the
semiotic resources of visual communication and
color, gesture and movement, gaze, voice and
music, to name a few

12. Another Focus…
 Multimodal studies have also been
conducted that set out to understand how
semiotic resources are used to articulate
discourses across a variety of contexts and
media for instance school, workplaces,
online environments, textbooks and
advertisements.

13. And a Third…
 Multimodal research makes a significant
contribution to research methods for the
collection and analysis of digital data and
environments within social research. It
provides novel methods for the collection
and analysis of types of visual data, video
data and innovative methods of multimodal
transcription and digital data management

14. Multimodality and Education
 “Text” is no longer the single, driving force in
literacy instruction
 Print, visual images, and design are also critical
components
 Emergent research on literacy highlights the
imaginative, interpretive, non-linear, interactive,
dynamic, visual, and mobile features of
communication (Coiro, Knobel, Lankshear,& Leu,
2008).

15. “Real-World” Connections
 As Siegel (2006) noted, “language arts
education can no longer ignore the way that
our social, cultural, and economic worlds
now require facility with texts and practices
involving the full range of representational
modes” (p. 65).

16. Instructional Dynamics
 A transition from print-based education to
multimodal education indicates a profound shift in
the notion of reading as a whole.
 Although possessing traditional print literacy
skills continues to be sufficient for many
communication tasks, the demands of digital
media and visual texts within a multimodal culture
require complex new ways of coding and
decoding image–text relations.

17. Ch-Ch-Changes
 One significant difference in the social
context of communication today is that print
itself can take on many forms through
visual design and synergy with images
(Dresang, 1999; Hammerberg[Hassett],
2001; Hassett, 2006a; Sipe, 1998,
2001),until it is literally pushed off the page
(Kress, 1998).

18. Buh-bye Robins and Bluebirds!
 Therefore, if we are to talk about reading
instruction within this new context, we need
“to realize that written language is being
displaced from its hitherto unchallenged
central position” (Kress, 1999, p. 68).

19. Pedagogogical Heresy?
 Print represents only one mode of communication,
and it is not al-ways the most important focus.
When reading these books in the classroom,
students and teachers alike need to focus on all of
the various textual elements(e.g., print, images,
graphics) as well as other modes of
communication that can occur within the social
context of the classroom, such as the interactive
and playful conversations that can happen around
the text as meaning is being made.

20. In theory!
 Theoretically, then,we see multimodal
literacy instruction as combining the modes
available in a text with the modes avail-able
through sociocognitive reading processes.

21. Inter-semiotic Relations
 How modes are configured in particular
contexts. These four concepts provide the
starting point for multimodal analysis.

22. Multimodal Aspects of Text
 Talk
 Gesture
 Dramatics
 Drawing
 Linguistic Signs

23. Multimodality and Minority
Groups
 Communication skills, if understood from a
multimodal and multilingual perspective, are
highly developed
 Able to switch between languages and modalities
(symbolic competence)
 Through codeswitching and modeswitching, shift
frames of reference to relevant local as well as
global semiotic resources.

24. Benefits of Multimodality
 Communication is more natural than in single
modality systems
 Interpretation accuracy can be increased with the
help of redundant and/or complementary
modalities.
 Users have different modality preferences and
different needs, and multimodal systems offer
them the freedom of choice: they can choose the
interface and modality that best suits them in a
particular situation.
 Increases usability of the system as well as general
accessibility to information technology.

25. How does Glogster fit into
Multimodal Instruction
 Multimodal literacy refers to the use of
numerous modes to communicate ones’
message. Text, audio, graphics, and video
are examples of digital media that can be
combined to present multimodal messages.
There are clear differences in presentation
between a traditional printed textbook and a
website on the same topic.

26. More Research
 A printed textbook consists of static images
and words, whereas most websites include
interactive, multimedia textual features
(Karchmer, 2001).

27. Still More Research…
 Research recommends explicit teaching of
multimodal literacy so that students
understand how various modes can be used
to develop dynamic multidimensional texts
that effectively communicate messages to
different audiences (Callow, 2008; Hassett
& Curwood, 2009; Leu et al., 2004

28. The Writing Connection
 Writers can develop multimodal texts using
Glogster by embedding a range of modes to
convey a unified message. These virtual
posters can be shared with a global
audience, providing important opportunities
for students to critically examine their work.

29. Meet the “Gloggers”
 Adrian and JaQuan

30. Fall Test Scores (Adrian)
 NWEA MAP Fall 2012 - Aug 27, 2012
(Grade 05)
 Reading 210

 Below Proficient RIT
 Reading 3.3 Std Error
 Reading 781 Lexile Score
 Reading 681 Lexile Min
 Reading 831 Lexile Max
 Reading 58 Percentile

31. Winter Scores (Adrian)
 NWEA MAP Winter 2013 - Feb 08, 2013
(Grade 05)
 Reading 232

 At/Above Proficient RIT
 Reading 3.4 Std Error
 Reading 1177 Lexile Score
 Reading 1077 Lexile Min
 Reading 1227 Lexile Max
 Reading 94 Percentile

32. Test Scores Fall (JaQuan)
 NWEA MAP Fall 2012 - Aug 27, 2012
(Grade 05)
 Reading 196

 Below Proficient RIT
 Reading 3.3 Std Error
 Reading 529 Lexile Score
 Reading 429 Lexile Min
 Reading 579 Lexile Max
 Reading 22 Percentile

33. Winter Scores (JaQuan)
 NWEA MAP Winter 2013 - Feb 08, 2013
(Grade 05)
 Reading 196

 Below Proficient RIT
 Reading 3.3 Std Error
 Reading 529 Lexile Score
 Reading 429 Lexile Min
 Reading 579 Lexile Min

34. Spring Scores (JaQuan)
 Reading 202

 Below Proficient RIT
 Reading 3.3 Std Error
 Reading 637 Lexile Score
 Reading 537 Lexile Min
 Reading 687 Lexile Max
 Reading 23 Percentile

35. Examples of Glogs
 http://edu.glogster.com
 Login: mkramanak
 Password: Glitter

36. Conclusions and
Recommendations
 Multimodal instruction assisted students in
improving ELA skills
 Students were enthusiastic about glogging
 Glogs will “follow” students to middle school
 Glogs can serve as an “electronic portfolio”
 Research needs to continue about the effectiveness
of glogs as an instructional tool

37. References and Resources
Bezemer, J. and Mavers, D. (2011) Multimodal Trans
cription as Academic
Practice, International Journal of Social Research Methodology Vol. 14, No. 3,
May 2011, 191-206
Callow, J. (2008). Show me: Principles for assessing studentsÕ
visual literacy.
The Read
ing Teacher, 61(8), pp. 616Ğ
626
Hassett, D.D., & Cur
wood, J. (2009). Theories and practices of multimodal
educat
ion: The instructional dynamics of picture books and primary classrooms.
The Read
ing Teacher, 63(4), 270Ğ
282
Jewitt, C. (ed.) (2009) TheRoutledge Handbook of Multimodal Analysis,
London:R
outledge
Karchmer, R.A. (2001). The journey ahead: Thirteen teachers report how the
Internet influences literacy andliteracy instruction in their KĞ
12 classrooms.
Reading Research Quarterly, 36(4), 442Ğ
466
Kress, G. (
2009) Multimodality: a Social Semiotic Approach to Contemporary
Communication, London: Routledge