I will review a sample of our results and brieflyexplain each of the primary meaning-making and decision-making strategies using examples from our data. First, the meaning-making strategies that we identified…
In the lived experience theme, one set of learners concluded that the image on the right was a library—because it corresponded with their lived experience of what a library contains—big tables and more books than the classroom.
Extended narrative consists of a learner making up a story using cues within an image or across multiple images. In this example from the warmup activity, the learner imagined the conversation that the speech bubble might symbolize, creating a narrative that was not intended by the creator of the image.
Image-text switching is used to generate possible meanings in an image, with the learners moving between the image and text to find potential meaning. In this example, the learners recognized the image of a library, and then looked in the text for the Arabic word for library.
Learners also negotiated meaning between themselves, reaching consensus on which meaning of a particular image that they would agree on. In this example, one learner noticed the zit on the girl’s face, while the other learner did not, and a shared meaning was negotiated in this conversation.
Internal context involved the identification of salient features across a set of images. In this case, the learners noticed that multiple images included seated girls, and they had to move past this initial interpretation of a set of images to identify which element of the sentence was salient to make the appropriate choice.
This code was applied relatively infrequently, and was always triggered by a question from the researcher rather than emerging naturally in the conversation. The theme identifies instances in which the learners address the potential intentions of the designer of the materials. In this example, the learners are discussing how the images work as a set, and the continuity that learners may see as a “movie” when noticing repeating elements in multiple images.
Next, we will look at the decision-making strategies, which focused on the taskpresented and the choices required to complete the task.
Learners narrowed the field, by eliminating items from consideration separate from meaning. In this case, the learners crossed out answers they had already used, to identify which answers remained.
Learners also took advantage of their knowledge of grammar and syntax. In this example, they identified a word as masculine in gender, which allowed them to eliminate some images that pictured females.
This theme is related to the previous image-text switching theme in meaning-making strategies, but focuses on checking interpretation rather than agreeing upon meaning. In this case, the learners switched back and forth between text and image to check or raise confidence in a decision. In this example, the learners confirm that this image of an individual listening to a iPod matches the text of “listening to something.”
In partner-checking, the learner confirms a decision that they have made together. In this example, the learners codify their answer and make their final selection.
Extrapolation from minimal cues indicates cases where the learner uses an element of an image or word to make a decision, without explicitly making meaning. In this example, the learner identifies a particular word as reminding him of blood, so he chose that answer using his instinct, rather than explicitly recognizing the meaning of the answer he selected.
Add text prompt in place of letters
Learners' Strategies for Interpreting Instructional Images
learners’ strategies for interpreting instructional images elizabeth bolingresearch abdullah atuwaijri, jiyoon jung, group colin gray, micah modell, craig howard,members funda ergulec, muruvvet demiral
background andpurpose of the study- The largely deterministic view of message design (Fleming, 1987; Fleming & Levie, 1993) and cognitive load theory related to multimedia (Mayer, Hegarty, S. Mayer, & Campbell, 2005) focuses on properties of images and their effects on learners’ performance- The semiotic view (von Engelhardt, 2002; Kress, 2004; Sless, 1986; Van Leeuwen, 2001) includes the role of learners’ interpretation of visuals, recognizing that learners are active in their interpretations (Schriver, 1996); that their aesthetic frame for learning is influential in addition to their cognitive frame (Parrish, Wilson, & Dunlap, 2010)- Empirical evidence shows that individuals do not interpret even simple images as their designers intend, and that they bring information into their interpretations that is not present in the images (Boling, Eccarius, Smith, Frick, 2004)
research questionwhat are the strategies learners use to makesense of images during learning activities?
participants & context- Eight dyads, four each from the Intensive English Program and the Near Eastern Languages and Cultures Program - First language Arabic learning English - First language English learning Arabic- Students completed the activity, agreeing on their choice of images to match sentences in vocabulary practice -- prompts were used by the researchers to encourage verbalization; stimulated recall was used to probe for further data program male female Arabic speakers learning 6 2 English English speakers learning 6 2 Arabic
materials- authentic language learning practice activity including someone giving blood? images redesigned using message design guidelines a girl who is sitting on a floor? a girl who is sitting in a garden? a pupil in a classroom? Which picture shows a look of concern? someone listening to something? a library? someone who is missing someone?
materials- authentic language learning practice activity including - images redesigned using message design guidelines - - -
data collection & analysis- Subjects were recorded on video completing the instructional activity in their native language video- Arabic video recordings were transcribed and translated into English by an outside party for further analysis by the research group arabic translation- English video recordings were transcribed and any Arabic words used were translated- Analysis was accomplished through stages: transcription - Repeated viewing of video with transcription - Coding and re-coding observed themes using Dedoose - Discussion and agreement during subgroup and whole-group review of data and themes
findings- Participants in this study, working in the context of a learning activity, used two readily observable and distinct (but often co-occurring) types of strategies: - meaning-making strategies focused on the images and their intended significance within the activity - decision-making strategies focused on the task presented and the choices required to complete the task- Frequent use was made of these strategies during the short task; the design of images was not deterministic of their interpretation without the addition of these strategies
meaning-making strategieslived experienceLearner uses combinedelements of the image toactivate culturally, personally-situated schema.–––––––––––––––––––––––––––––PR, PL: Right, when we see theimage then we could see thatit must be a library as theclassroom does not contain asmany books as the library inaddition to the big tables.‖(IEP2 4663-5150)
meaning-making strategiesextended narrativeLearner makes up a story relatedto, but not literally depicted in,one or several available images.–––––––––––––––––––––––––––––PL: Um, well, uh we have—I knowthis word means ―news‖ in Arabic,so I figure like since they’retalking—there’s like aconversation about something, Iassume like he’s asking whoeverthis is like ―did you see the news‖or something. Granted, I don’tknow ―to see‖ is in Arabic, but…[NELC3 2293-2579]
meaning-making strategiesimage-text switching to generate possibilitiesLearner changes the focus ofattention from image to text andback to generate possibledirections for interpretingmeaning.–––––––––––––––––––––––––––––M: Being in the library, pen, So weneed to look for something withLa Albeah [points at the thirdimage along the top of the sheet.Then he scans the sentences onhis half of the paper][NELC2 1449-1635]
meaning-making strategiesnegotiating meaningLearner discusses individualexpectations to reach consensuson the meaning they will accept.–––––––––––––––––––––––––––––PL: She’s looking at something inthe mirror. And when I look closeat that dot, I noticed that erh sowondered if they were talkingabout a zit.PR: I mean I didn’t even noticethat until then. I was trying to, I justnoticed she was looking at herself.Trying to find something thatPL: Looking at her eyeball.[NELC2 10341-10722]
meaning-making strategiesinternal contextLearner identifies salient featuresof an image based on its beingpart of a set.–––––––––––––––––––––––––––––PR: I think something that like forme was kind of confusing was like Iwas looking at the [inaudible] andlike they were sitting, but there’slike quite a few where they weresitting, but then I was like a child,like which one of these is a child.That was kind of the problem. Youcouldn’t like these twodifferentiate[NELC3 19563-19884]
meaning-making strategiesenvisioning the designerLearner, directly or indirectly, addresses the possibleintentions of a designer/writer who has produced the text.–––––––––––––––––––––––––––––PR: Yeah, to see the two that were kind of similarPL: Yeah, and people always look for continuityPR: Yeah—PL: So if you use like similar, like the same objects or likesimilar objects throughout your images, it’s like—keeppeople like [inaudible] I guess. Or like say if this was amovie or something, they would—[NELC3 19246-19562]
decision-making strategiesnarrowing the fieldLearner eliminates items separately from content of images ortext (as in eliminating items already decided upon).–––––––––––––––––––––––––––––PL: So, alright, let’s just—let’s just cross out the ones we have.OK. Cause I feel like that helps for us to know what ouroptions are.PR: So we have F and C, OK[NELC3 9169-9332]
decision-making strategieslanguage mechanicsLearner figures out an answer by relying on knowledgeof grammar and syntax.–––––––––––––––––––––––––––––LP: so this is tasahar, well this one is masculine, so it has tobe, so it should be C, right? Because the other two areboth girls.RP: umm , but this one is masculine tooLP: I know, dang[NELC1 4390-4580]
decision-making strategiesimage-text switching to check interpretationLearner changes the focus ofattention from image to text andback to check or raise confidencein a candidate decision.–––––––––––––––––––––––––––––PL: the answer is there is nothingbut it, then we connect it tommmm…..someone … someonelistening to something, so maybe ifhe is listening to something then itcould be this one.[IEP4 5705-5889]
decision-making strategiespartner-checkingLearner asks about/confirms candidate decision.–––––––––––––––––––––––––––––PL: to tell you the truth, I am confused between F and BPR: it’s F I think, this how it looks.PL: Ok then put it as F[IEP3 2947-3139]
decision-making strategiesextrapolation from minimal cuesLearner selects an image element or a word that allows him to actwithout explicitly making meaning.–––––––––––––––––––––––––––––F: So like say in C, what made you pick the blood instead ofsomething else that was going on in the image?PL: Well, for some reason, this last word reminded me of bloodF: OKPL: I don’t know why—I—I had no idea why, it just did. So I just wentwith it [laughs] And uh, I mean they always say go with your firstinstinct, right?[NELC3 12491-12995]
implications- Learners are active participants in using images during learning activities rather than passive recipients of messages- Learners are not automatically led to meanings by the properties of images alone- Strategies for interpreting images are invoked together with those for completing the learning task; learners are not focused or reliant solely on interpretation strategies- Designers of instructional images cannot rely on deterministic principles alone to construct effective images- Designers of instructional images must attend to the structure of the learning task, not simply to the content
limitations & future research- Limitations - small number of participants (n=8 dyads) - single context of learning - difficulty of articulating visual strategies - simple, still images- Future research - increased number of participants - variable ages of participants - extended contexts of learning - rigorous training for stimulated recall - increased complication in visual/textual task
referencesBoling, E., Eccarius, M., Smith, K., & Frick, T. (2004). Instructional illustrations: Intended meanings and learner interpretations. Journal ofVisual Literacy, 24(2), 185-204.Fleming, M. L. (1987). Designing pictorial/verbal instruction: Some speculative extensions from research to practice. In The psychologyof illustration. (pp. 136-57). New York: Springer Verlag.Fleming, M., & Levie, W. H. (1993). Instructional message design: Principles from the behavioral and cognitive sciences (2nd ed.).Englewood Cliffs, New Jersey: Educational Technology Publications.Kress, G. (2004). Reading images: Multimodality, representation and new media. Information Design Journal, 12(2), 110-119.Mayer, R. E., Hegarty, M., Mayer, S., & Campbell, J. (2005). When static media promote active learning: Annotated illustrations versusnarrated animations in multimedia instruction. Journal of Experimental Psychology: Applied, 11(4), 256-65.Parrish, P., Wilson, B. G., & Dunlap, J. C. (in press). Learning experience as transaction: A framework for instructional design.Educational Technology.Schriver, K. A. (1996). Dynamics in document design: Creating text for readers. New York: Wiley Computer Publishing.Sless, D. (1986). In search of semiotics. Totowa, New Jersey: Barnes & Noble Books.van Leeuwen, T. (2001). Semiotics and iconography. In T. van Leeuwen & C. Jewitt (Eds.), Handbook of visual analysis. (pp. 92-118).London: Sage Publications.von Engelhardt, J. (2002). The language of graphics: A framework for the analysis of syntax and meaning in maps, charts anddiagrams. Amsterdam: Institute for Logic, Language and Computation, Universiteit van Amsterdam.
slide assignments- elizabeth boling … slides 1-3; slides 12-13- micah modell … slide 4- jiyoon jung … slide 5- abdullah atuwaijri … 6- craig howard … slide 7- colin gray … slides 8-11
boy scout slides- Images at larger size- Activity content – English- Activity content – Arabic- Warm up activity- Script for stimulated recall- Summary slide from Boling, et al.
a girl who is sitting someone giving someone who is a library? on a floor? blood? missing someone? a pupil in a girl who is sitting someone listening a look ofa classroom? in a garden? to something? concern?
activity content (english)- someone giving blood? look of concern?- a girl who is sitting on a - someone listening to floor? something?- a girl who is sitting in a - a library? garden? - someone who is missing- a pupil in a classroom? someone?- Which picture shows a
warm-up activity A BWhich picture shows:Someone telling a story? __________Someone reporting the news? __________
recruitment scriptThe researchers for this study have studied graphics used for learning before. We haveseen that it may be difficult for learners to decide what a picture is supposed to mean. Inthis study we want to learn more about your thoughts as you learn using materials thatinclude images.We want some students to work together in pairs and complete a vocabulary activitythat have been approved by the Intensive English Program (IEP) and the Near EasternLanguages and Cultures (NELC) department at Indiana University. For each part of theactivity, both of you will need to agree before you mark the answer.We would like to understand what you say and see what you point to, so we need yourpermission to record you on video as you work. None of your personal information will beshared with anyone outside the study. The video recording will not be seen by anyoneoutside the study. If you volunteer for the study, you can stop at any time.We think it will take about 30 minutes to complete this activity. During the activity you willbe practicing vocabulary you need to know for class. Will some of you volunteer to dothis during your break time?
image interpretation―The International Standards Organization (ISO) standard for correctinterpretations of public information symbols is 85% correct interpretations(Olgay, 2001). While instructional illustrations, usually seen in context and withaccompanying text, may not need to reach this standard in order to beuseful, it is important to note that no images were interpreted correctly at 85%or above across all sample groups (although no running - image 10 andtalking - image 8) come close, each with only one sample group missing the85% mark).‖ (Boling, et al., p. 200, bolding added)―Even when the threshold for interpretation is adjusted informally to accountfor the additional support that may be present in an instructional situations,designers may be disappointed to know that some sample groups averagedas little as 47% interpretations consistent with their intentions across the 16items, and that the highest average for a sample group was only 70%.‖(Boling, et al., p. 201, bolding added)
meaning-making strategies THEME DESCRIPTION EXAMPLE Learner uses combined elements of PR, PL: Right, when we see the image then we the image to activate culturally, could see that it must be a library as theLived personally-situated schema. classroom does not contain as many books as theExperience library in addition to the big tables.” (IEP2 4663- 5150) Learner makes up a story related to, “PL: Um, well, uh we have—I know this word but not literally depicted in, one or means “news” in Arabic, so I figure like since several available images. they’re talking—there’s like a conversation aboutExtended something, I assume like he’s asking whoever thisNarrative is like “did you see the news” or something. Granted, I don’t know “to see” is in Arabic, but...” (NELC3 2293-2579)Image-text Learner changes the focus of “M: Being in the library, pen, So we need to look attention from image to text and back for something with La Albeah [points at the thirdSwitching to to generate possible directions for image along the top of the sheet. Then he scansGenerate interpreting meaning. the sentences on his half of the paper]” (NELC2Possibilities 1449-1635)
meaning-making strategies THEME DESCRIPTION EXAMPLE Learner discusses individual “M: She’s looking at something in the mirror. And expectations to reach consensus on when I look close at that dot, I noticed that erh so the meaning they will accept. wondered if they were talking about a zit.Negotiating F: I mean I didn’t even notice that until then. I wasMeaning trying to, I just noticed she was looking at herself. Trying to find something that M: Looking at her eyeball.” (NELC2 10341-10722) Learner identifies salient features of “PR: I think something that like for me was kind of an image based on its being part of a confusing was like I was looking at the [inaudible] set. and like they were sitting, but there’s like quite aInternal few where they were sitting, but then I was like aContext child, like which one of these is a child. That was kind of the problem. You couldn’t like these two differentiate” (NELC3 19563-19884) Learner, directly or indirectly, “PR: Yeah, to see the two that were kind of similar addresses the possible intentions of a PL: Yeah, and people always look for continuity designer/writer who has produced the PR: Yeah—Envisioning text. PL: So if you use like similar, like the same objectsthe Designer or like similar objects throughout your images, it’s like—keep people like [inaudible] I guess. Or like say if this was a movie or something, they would— ” (NELC3 19246-19562)
decision-making strategies THEME DESCRIPTION EXAMPLE Learner eliminates items separately “PL: So, alright, let’s just—let’s just cross out the from content of images or text (as in ones we have. OK. Cause I feel like that helps forNarrowing eliminating items already decided us to know what our options are.the Field upon). PR: So we have F and C, OK” (NELC3 9169- 9332) Learner figures out an answer by “LP: so this is tasahar, well this one is masculine, relying on knowledge of grammar and so it has to be, so it should be C, right? BecauseLanguage syntax. the other two are both girls.Mechanics RP: umm , but this one is masculine too LP: I know, dang” (NELC1 4390-4580)Image-text Learner changes the focus of “PL: the answer is there is nothing but it, then weSwitching to attention from image to text and back connect it to mmmm…..someone … someoneCheck to check or raise confidence in a listening to something, so maybe if he is listeningInterpretatio candidate decision. to something then it could be this one.” (IEP4n 5705-5889)
decision-making strategies THEME DESCRIPTION EXAMPLE Learner asks about/confirms “PL: to tell you the truth, I am confused between FPartner- candidate decision. and Bchecking PR: it’s F I think, this how it looks. PL: Ok then put it as F” (IEP3 2947-3139) Learner selects an image element or “CG: So like say in C, what made you pick the a word that allows him to act without blood instead of something else that was going on explicitly making meaning. in the image?Extrapolatio PL: Well, for some reason, this last word remindedn from me of bloodMinimal CG: OKCues PL: I don’t know why—I—I had no idea why, it just did. So I just went with it [laughs] And uh, I mean they always say go with your first instinct, right?” (NELC3 12491-12995)