Carey Jewitt, from the University of London’s Institute of Education Knowledge Lab, wrote…What it means to be literate in the digital era of the 21st century is different thanwhat was needed previously. If school literacy is to be relevant tothe demands of the multimodal environment of the larger world it must move awayfrom the reduction of literacy to ‘‘a static series of technical skills’’ or risk ‘‘fostering apopulation of functional illiterates’’. In short, school literacy needsto be expanded to reflect the semiotic systems that young people use. Schools that promote alinguistic view of literacy and a linear view of readingfailto connect the kindsof literacy required in the school with the ‘‘out-of-school worlds’’ of most people.Jewitt, 2005, p.330
The skill sets listed here are the most commonly agreed to skills identified by numerousorganizations involved in aligning the needs of business, industry, and society in the 21st century with educational institutions and curriculum. In this course we have all learned and discovered the tremendous potential that emerging technologies have to help build these skills while staying true to the theories of educational pioneers. In this presentation we set out to focus on a specific aspect of emerging technologies, the use of multimodal text and multimodal composition in literacy education. Our consensus is that while there are significant positives to be gained by the use of such technology, there are significant challenges and risks that warrant attention and careful consideration of design and application.Let’s first look at the positive potential of multimodal texts and then consider the challenges that are presented.
As we go further in our research of 21st century skills – We must ask the questions… What are Multimodal Texts? Anstey, M & Bull, G present assistance to teachers in the exploration of multimodal texts. Within this realm of research they provide a definition of literacy for explanation and clarification of purpose: In the 21st century, the definition of literacy has expanded to refer to a flexible, sustainable command of a set of capabilities in the use and production of traditional texts and new communications technologies, using spoken language, print and multimedia. (p 5)The aim of increased proficiency among educators is to provide students the ability to ‘understand, interpret, reflect on and create spoken, written and multimodal texts throughout a variety of learning environments’.
A text may be defined as multimodal when it combines two or more semiotic systems. There are five semiotic systems in total:Linguistic: comprising aspects such as vocabulary, generic structure and the grammar of oral and written languageVisual: comprising aspects such as colour, vectors and viewpoint in still and moving imagesAudio: comprising aspects such as volume, pitch and rhythm of music and sound effectsGestural: comprising aspects such as movement, speed and stillness in facial expression and body languageSpatial: comprising aspects such as proximity, direction, position of layout and organization of objects in space.
1. Knowledge and understanding about reading and writing multimodal texts that are delivered in different ways (paper, live and digital electronic).2. Knowledge of the five semiotic systems of which a multimodal text can be composed.3. Metalanguage that facilitates the analysis, discussion and understanding of how multimodal texts work.4. Explicit pedagogies that make the processes of reading and writing multimodal texts transparent.5. Ensure that the school has a balanced, school-wide approach to the teaching of literacy and multimodal texts.
Since at least the Industrial Revolution we have conceived of the brain as a fixed structure; we have conceived of the brain as a machine. With the development of the digital computer, the machine metaphor was extended so that by the late 20th century, we began to talk about our brain and our behavior as hardwired. But now, in the 21st century, neuroscientists suggest that our brain is plastic. The brain adapts through the way we live and through the tools we use. The more we use a tool, the more our brain adapts to its form and function.
We now live much of our lives connected to the Internet, which, to be sure is a powerful tool, albeit is also has become a habit of mind, a form and way of life. And, as Nicholas Carr argues: we “tend to use the Net regularly, even obsessively…It delivers precisely the kind of sensory and cognitive stimuli—repetitive, intensive, interactive, addictive—that have been shown to result in strong and rapid alterations in brain circuits and functions (116).
Earlier in the semester we read about Jean Piaget’s constructivism. He posited four stages of adaptation and organization: assimilation, accommodation, equilibration, and schemata. Human beings are confronted by external stimuli that we assimilate into our existing mental schemata, essentially hard-wired pigeon holes through which we structure knowledge. Prior to new stimuli being assimilated and accommodated, human beings find themselves in a state of disequilibrium. We accommodate new stimuli by constructing new schemata or by reconfiguring existing ones. Piaget believed that all human beings are hardwired to in their desire, or drive, to achieve equilibrium; this hard-wired drive he called equilibration.
Although we no longer understand our brains as hardwired, Carr would likely argue that we human beings are more often in a state of disequilibrium now than we have been since at least the advent of the book. Here’s one piece of evidence in support of this claim: educational psychologist, John Sweller’s studies of the relationship between working memory and long-term memory indicate that the way our minds process information when we surf the web has a strong effect on the depth of our understanding: The depth of our intelligence hinges on our ability to transfer information from the working memory to long-term memory and weave it into conceptual schemas…The information flowing into our working memory at any given moment is called our “cognitive load.” When the load exceeds our mind’s ability to store and process the information…we’re unable to retain the information or to draw connections with the information already stored in our long-term memory. We can’t translate the new information into schemas. Our ability to learn suffers, and our understanding remains shallow.
In other words, cognitive overload leads to decreased comprehension and retention. Overall, our intensive use of the net increases our capacity to access and to some degree, to process, a breadth of information, but this increase diminishes our capacity for in-depth understanding and critical thinking.
In writing about the uses of new technology in schools, Lankshear and Knobel (2001) have suggested, “it does not follow from the fact that so-called new technologies are being used in literacy education that new literacies are being engaged with. Still less does it imply that learners are developing, critiquing, analyzing, or even becoming technically proficient with new literacies” (189). If it’s the case that time spent surfing the web detracts from digital natives’ abilty to sustain attention and to develop significant cognitive skills, then there are severe ramifications for what we still think of as true literacy. As Ken Pugh, a cognitive neuroscientist at Yale argues: “‘Reading a book, and taking the time to ruminate and make inferences and engage the imaginational processing, is more cognitively enriching, without doubt, than the short little bits that you might get if you’re into the 30-second digital mode.”
As an “outsider” myself, but one who works with “insiders” every day, I concur with Pugh’s observations. Many students lack the skills and habits of mind to attend closely to one text and to sustain a logical argument for any significant period of time. Such skills and habits are necessary to understand so much of what has been written prior to the digital age. Still, others dismiss these concerns: “[Young people] “aren’t so troubled as some of us older folks are by reading that doesn’t go in a line,” said Rand J. Spiro, a professor of educational psychology at Michigan State University, who is studying reading practices on the Internet. “That’s a good thing because the world doesn’t go in a line, and the world isn’t organized into separate compartments or chapters”(as cited in Rich, 2008). Spiro’s argument, is not one that compels me: it is, after all, It is, after all, through the processes of reading and analyzing and interpreting what William James (1981/1890) called the “blooming, buzzing confusion” (p. 462) that we order it, that we understand it, and there is no reason, whatsoever, to think that the language we use to examine the world should have the same structure of that which is being examined. Just because the world is not linear doesn’t mean our reading practices ought not to be.
Literacy remains a “process whereby people acquire—in dialogue facilitated by reading and writing—intellectual access to their world and their place within that world, a conception of their unique human status and vocation, and the commitment to pursue that vocation” (Lankshear & Knobel, p. 18). That this process changes when the medium through which people acquire it changes is evident, but also evident is that literacy is always cognizant of and subsumes its own historical precedents.
1. Training for Educators to become more comfortable in the usage and understanding of a variety of multimodal tools.2. Increasing the Educators ability to used more complex media resources as their knowledge and confidence increases.3. The awareness that the multimodal medias used possess text using audio, gestural, visual, linguistic and spatial semiotic skills. 4. Necessary support for Educators throughout their professional learning – through discussions and knowledge of what works and what clearly does not. 5. Exchange of ideas between educators – regarding challenges encountered and support for each area of need – when necessary.
Transcript of "Multimodal Learning "
Eileen Abrahams, Irene Cruz, & Matt Hibbard
Learning with Emerging Technologies
SUNY Empire State College
Dr. Eileen O’Connor
December 9, 2013
Global and cultural
The way our minds process information when we surf
the web has a strong effect on the depth of our
“The depth of our
intelligence hinges on
our ability to transfer
information from the
working memory to
long-term memory and
weave it into conceptual
Carr, 2011, p.122-3
Intensive use of the
Increases some cognitive
Rapid focal attention
Capacity for in-depth
Anstey, M & Bull, G., (June 4, 2010). Retrieved from,
Bhattacharya, K. & Han, S. (2001). Piaget and cognitive development. In M. Orey (Ed.), Emerging perspectives on
learning, teaching, and technology. Retrieved September 13, 2013, from http://projects.coe.uga.edu/epltt/
Carr, Nicholas (2011). The Shallows. New York: Norton, 2011. 114-143. Print
Hanover Research. (2011). A crosswalk of 21st century skills. Retrieved from http://www.hanoverresearch.com/wpcontent/uploads/2011/12/A-Crosswalk-of-21st-Century-Skills-Membership.pdf
James, William. Principles of psychology. Cambridge, MA: Harvard University Press, 1890.Reprinted 1981.
Jewitt, C. (2005). Multimodality,‘‘Reading’’, and ‘‘Writing’’for the 21st Century.Discourse: studies in the cultural politics
of education, 26(3), retrieved from http://culturemediaed.pressible.org/files/2011/09/jewitt_multimodality.pdf
Lankshear, C. Ideas of functional literacy: critique and redefinition of an educational goal.” In Literacies: social,
cultural, and historical perspectives. Eds. Colin Lankshear and Michele Knobel. New York: Peter Lang, 2011. 3-20.
Lankshear, C and Knobel, M. The “new literacy studies” and the study of “new” literacies.”Literacies: social, cultural,
and historical perspectives. Eds. Colin Lankshear and MicheleKnobel. New York: Peter Lang, 2011. 183-209. Print.
Reprinted from New Zealand Journal of Educational Studies, 1985.
Rich, M. (2005, July 27). "Literacy Debate: Online, R U Really Reading?" New York Times.
Veletsianos, George. “A Definition of Emerging Technology in Distance Education.” In Emerging technologies in
distance education. (pp. 3-22). Retrieved September 13, 2013, from http://aupress.ca/index/php/books/120177
Atirador (Own work) [CC-BY-SA-3.0 (http://creativecommons.org/licenses/by-sa/3.0) or GFDL
(http://www.gnu.org/copyleft/fdl.html)], via Wikimedia Commons
Demonte, J. (2013, July 15). High-quality professional development for teachers. Retrieved from
Hos-McGrane, M. (2010, April 20). Games-based learning. Retrieved from http://www.maggiehosmcgrane.com/2010/04/gamesbased-learning.html
Keathley, M. (2012, Spet 4). How online students can turn distractions into academic success. Retrieved from
Leonard Low from Australia (Concept for augmented reality mobile phone) [CC-BY-2.0
(http://creativecommons.org/licenses/by/2.0)], via Wikimedia Commons
Loreth (Own work) [CC-BY-SA-3.0 (http://creativecommons.org/licenses/by-sa/3.0)], via Wikimedia Commons
McLeod, S. (2012). Jean piaget. Retrieved from http://www.simplypsychology.org/piaget.html
NNU-11-22100506 (Own work) [CC-BY-SA-3.0 (http://creativecommons.org/licenses/by-sa/3.0)], via Wikimedia Commons
Rhose island school of Design. (2010, Oct 20). Learning the language of gesture. Retrieved from
Riley, S. (2013). What the 21st century classroom looks like. Retrieved from http://educationcloset.com/2012/08/07/what-the21st-century-classroom-looks-like/
Schrok, K. (2012, January). Equipping teachers to infuse technology. Retrieved from
Smart Link Corporation. (2013). Text to speech. Retrieved from http://text-to-speech.imtranslator.net/
Tekks (English Wikipedia) [CC-BY-SA-3.0 (http://creativecommons.org/licenses/by-sa/3.0)], via Wikimedia Commons
U.S. Navy photo by Mass Communication Specialist 3rd Class Chelsea Kennedy [Public domain], via Wikimedia Commons
Visual Leap. (2011). About visual thinking. Retrieved from http://www.thevisualleap.com/about.html
Vom Leiter der Landesbibliothek Oberösterreich zur Verwendung mit unten angeführter Lizenz
(http://www.landesbibliothek.at/) [CC-BY-SA-2.0-at (http://creativecommons.org/licenses/by-sa/2.0/at/deed.en)], via
XIIIfromTOKYO (Own work) [CC-BY-SA-3.0 (http://creativecommons.org/licenses/by-sa/3.0)], via Wikimedia Commons
A particular slide catching your eye?
Clipping is a handy way to collect important slides you want to go back to later.