Learners' Strategies for Interpreting Instructional Images
1. learners’ strategies
research abdullah atuwaijri, jiyoon jung,
group colin gray, micah modell, craig howard,
members funda ergulec, muruvvet demiral
2. background and
purpose of the study
- The largely deterministic view of message design (Fleming, 1987; Fleming &
Levie, 1993) and cognitive load theory related to multimedia (Mayer,
Hegarty, S. Mayer, & Campbell, 2005) focuses on properties of images and their
effects on learners’ performance
- The semiotic view (von Engelhardt, 2002; Kress, 2004; Sless, 1986; Van Leeuwen, 2001)
includes the role of learners’ interpretation of visuals, recognizing
that learners are active in their interpretations (Schriver, 1996); that their
aesthetic frame for learning is influential in addition to their cognitive
frame (Parrish, Wilson, & Dunlap, 2010)
- Empirical evidence shows that individuals do not interpret even
simple images as their designers intend, and that they bring
information into their interpretations that is not present in the images
(Boling, Eccarius, Smith, Frick, 2004)
4. participants & context
- Eight dyads, four each from the Intensive English Program
and the Near Eastern Languages and Cultures Program
- First language Arabic learning English
- First language English learning Arabic
- Students completed the activity, agreeing on their choice
of images to match sentences in vocabulary practice --
prompts were used by the researchers to encourage
verbalization; stimulated recall was used to probe for
program male female
Arabic speakers learning
English speakers learning
- authentic language learning practice activity including someone giving
images redesigned using message design guidelines
a girl who is sitting
on a floor?
a girl who is sitting
in a garden?
a pupil in a
shows a look of
someone who is
7. data collection & analysis
- Subjects were recorded on video completing the
instructional activity in their native language video
- Arabic video recordings were transcribed and
translated into English by an outside party for further
analysis by the research group arabic
- English video recordings were transcribed and any
Arabic words used were translated
- Analysis was accomplished through stages: transcription
- Repeated viewing of video with transcription
- Coding and re-coding observed themes using Dedoose
- Discussion and agreement during subgroup and
whole-group review of data and themes
- Participants in this study, working in the context of
a learning activity, used two readily observable
and distinct (but often co-occurring) types of
- meaning-making strategies focused on the images
and their intended significance within the activity
- decision-making strategies focused on the task
presented and the choices required to complete
- Frequent use was made of these strategies
during the short task; the design of images was
not deterministic of their interpretation without
the addition of these strategies
10. meaning-making strategies
Learner uses combined
elements of the image to
activate culturally, personally-
PR, PL: Right, when we see the
image then we could see that
it must be a library as the
classroom does not contain as
many books as the library in
addition to the big tables.‖
11. meaning-making strategies
Learner makes up a story related
to, but not literally depicted in,
one or several available images.
PL: Um, well, uh we have—I know
this word means ―news‖ in Arabic,
so I figure like since they’re
talking—there’s like a
conversation about something, I
assume like he’s asking whoever
this is like ―did you see the news‖
or something. Granted, I don’t
know ―to see‖ is in Arabic, but…
12. meaning-making strategies
image-text switching to generate possibilities
Learner changes the focus of
attention from image to text and
back to generate possible
directions for interpreting
M: Being in the library, pen, So we
need to look for something with
La Albeah [points at the third
image along the top of the sheet.
Then he scans the sentences on
his half of the paper]
13. meaning-making strategies
Learner discusses individual
expectations to reach consensus
on the meaning they will accept.
PL: She’s looking at something in
the mirror. And when I look close
at that dot, I noticed that erh so
wondered if they were talking
about a zit.
PR: I mean I didn’t even notice
that until then. I was trying to, I just
noticed she was looking at herself.
Trying to find something that
PL: Looking at her eyeball.
14. meaning-making strategies
Learner identifies salient features
of an image based on its being
part of a set.
PR: I think something that like for
me was kind of confusing was like I
was looking at the [inaudible] and
like they were sitting, but there’s
like quite a few where they were
sitting, but then I was like a child,
like which one of these is a child.
That was kind of the problem. You
couldn’t like these two
15. meaning-making strategies
envisioning the designer
Learner, directly or indirectly, addresses the possible
intentions of a designer/writer who has produced the text.
PR: Yeah, to see the two that were kind of similar
PL: Yeah, and people always look for continuity
PL: So if you use like similar, like the same objects or like
similar objects throughout your images, it’s like—keep
people like [inaudible] I guess. Or like say if this was a
movie or something, they would—
17. decision-making strategies
narrowing the field
Learner eliminates items separately from content of images or
text (as in eliminating items already decided upon).
PL: So, alright, let’s just—let’s just cross out the ones we have.
OK. Cause I feel like that helps for us to know what our
PR: So we have F and C, OK
18. decision-making strategies
Learner figures out an answer by relying on knowledge
of grammar and syntax.
LP: so this is tasahar, well this one is masculine, so it has to
be, so it should be C, right? Because the other two are
RP: umm , but this one is masculine too
LP: I know, dang
19. decision-making strategies
image-text switching to check interpretation
Learner changes the focus of
attention from image to text and
back to check or raise confidence
in a candidate decision.
PL: the answer is there is nothing
but it, then we connect it to
mmmm…..someone … someone
listening to something, so maybe if
he is listening to something then it
could be this one.
21. decision-making strategies
extrapolation from minimal cues
Learner selects an image element or a word that allows him to act
without explicitly making meaning.
F: So like say in C, what made you pick the blood instead of
something else that was going on in the image?
PL: Well, for some reason, this last word reminded me of blood
PL: I don’t know why—I—I had no idea why, it just did. So I just went
with it [laughs] And uh, I mean they always say go with your first
- Learners are active participants in using images during
learning activities rather than passive recipients of
- Learners are not automatically led to meanings by the
properties of images alone
- Strategies for interpreting images are invoked together
with those for completing the learning task; learners are
not focused or reliant solely on interpretation strategies
- Designers of instructional images cannot rely on
deterministic principles alone to construct effective
- Designers of instructional images must attend to the
structure of the learning task, not simply to the content
23. limitations & future research
- small number of participants (n=8 dyads)
- single context of learning
- difficulty of articulating visual strategies
- simple, still images
- Future research
- increased number of participants
- variable ages of participants
- extended contexts of learning
- rigorous training for stimulated recall
- increased complication in visual/textual task
Boling, E., Eccarius, M., Smith, K., & Frick, T. (2004). Instructional illustrations: Intended meanings and learner interpretations. Journal of
Visual Literacy, 24(2), 185-204.
Fleming, M. L. (1987). Designing pictorial/verbal instruction: Some speculative extensions from research to practice. In The psychology
of illustration. (pp. 136-57). New York: Springer Verlag.
Fleming, M., & Levie, W. H. (1993). Instructional message design: Principles from the behavioral and cognitive sciences (2nd ed.).
Englewood Cliffs, New Jersey: Educational Technology Publications.
Kress, G. (2004). Reading images: Multimodality, representation and new media. Information Design Journal, 12(2), 110-119.
Mayer, R. E., Hegarty, M., Mayer, S., & Campbell, J. (2005). When static media promote active learning: Annotated illustrations versus
narrated animations in multimedia instruction. Journal of Experimental Psychology: Applied, 11(4), 256-65.
Parrish, P., Wilson, B. G., & Dunlap, J. C. (in press). Learning experience as transaction: A framework for instructional design.
Schriver, K. A. (1996). Dynamics in document design: Creating text for readers. New York: Wiley Computer Publishing.
Sless, D. (1986). In search of semiotics. Totowa, New Jersey: Barnes & Noble Books.
van Leeuwen, T. (2001). Semiotics and iconography. In T. van Leeuwen & C. Jewitt (Eds.), Handbook of visual analysis. (pp. 92-118).
London: Sage Publications.
von Engelhardt, J. (2002). The language of graphics: A framework for the analysis of syntax and meaning in maps, charts and
diagrams. Amsterdam: Institute for Logic, Language and Computation, Universiteit van Amsterdam.
25. slide assignments
- elizabeth boling … slides 1-3; slides 12-13
- micah modell … slide 4
- jiyoon jung … slide 5
- abdullah atuwaijri … 6
- craig howard … slide 7
- colin gray … slides 8-11
26. boy scout slides
- Images at larger size
- Activity content – English
- Activity content – Arabic
- Warm up activity
- Script for stimulated recall
- Summary slide from Boling, et al.
27. a girl who is sitting someone giving someone who is
on a floor? blood? missing someone?
a pupil in a girl who is sitting someone listening a look of
a classroom? in a garden? to something? concern?
28. activity content (english)
- someone giving blood? look of concern?
- a girl who is sitting on a - someone listening to
- a girl who is sitting in a - a library?
- someone who is missing
- a pupil in a classroom? someone?
- Which picture shows a
30. warm-up activity
Which picture shows:
Someone telling a story? __________
Someone reporting the news? __________
31. recruitment script
The researchers for this study have studied graphics used for learning before. We have
seen that it may be difficult for learners to decide what a picture is supposed to mean. In
this study we want to learn more about your thoughts as you learn using materials that
We want some students to work together in pairs and complete a vocabulary activity
that have been approved by the Intensive English Program (IEP) and the Near Eastern
Languages and Cultures (NELC) department at Indiana University. For each part of the
activity, both of you will need to agree before you mark the answer.
We would like to understand what you say and see what you point to, so we need your
permission to record you on video as you work. None of your personal information will be
shared with anyone outside the study. The video recording will not be seen by anyone
outside the study. If you volunteer for the study, you can stop at any time.
We think it will take about 30 minutes to complete this activity. During the activity you will
be practicing vocabulary you need to know for class. Will some of you volunteer to do
this during your break time?
32. image interpretation
―The International Standards Organization (ISO) standard for correct
interpretations of public information symbols is 85% correct interpretations
(Olgay, 2001). While instructional illustrations, usually seen in context and with
accompanying text, may not need to reach this standard in order to be
useful, it is important to note that no images were interpreted correctly at 85%
or above across all sample groups (although no running - image 10 and
talking - image 8) come close, each with only one sample group missing the
85% mark).‖ (Boling, et al., p. 200, bolding added)
―Even when the threshold for interpretation is adjusted informally to account
for the additional support that may be present in an instructional situations,
designers may be disappointed to know that some sample groups averaged
as little as 47% interpretations consistent with their intentions across the 16
items, and that the highest average for a sample group was only 70%.‖
(Boling, et al., p. 201, bolding added)
33. meaning-making strategies
THEME DESCRIPTION EXAMPLE
Learner uses combined elements of PR, PL: Right, when we see the image then we
the image to activate culturally, could see that it must be a library as the
personally-situated schema. classroom does not contain as many books as the
Experience library in addition to the big tables.” (IEP2 4663-
Learner makes up a story related to, “PL: Um, well, uh we have—I know this word
but not literally depicted in, one or means “news” in Arabic, so I figure like since
several available images. they’re talking—there’s like a conversation about
Extended something, I assume like he’s asking whoever this
Narrative is like “did you see the news” or something.
Granted, I don’t know “to see” is in Arabic, but...”
Image-text Learner changes the focus of “M: Being in the library, pen, So we need to look
attention from image to text and back for something with La Albeah [points at the third
Switching to to generate possible directions for image along the top of the sheet. Then he scans
Generate interpreting meaning. the sentences on his half of the paper]” (NELC2
34. meaning-making strategies
THEME DESCRIPTION EXAMPLE
Learner discusses individual “M: She’s looking at something in the mirror. And
expectations to reach consensus on when I look close at that dot, I noticed that erh so
the meaning they will accept. wondered if they were talking about a zit.
F: I mean I didn’t even notice that until then. I was
Meaning trying to, I just noticed she was looking at herself.
Trying to find something that
M: Looking at her eyeball.” (NELC2 10341-10722)
Learner identifies salient features of “PR: I think something that like for me was kind of
an image based on its being part of a confusing was like I was looking at the [inaudible]
set. and like they were sitting, but there’s like quite a
few where they were sitting, but then I was like a
Context child, like which one of these is a child. That was
kind of the problem. You couldn’t like these two
differentiate” (NELC3 19563-19884)
Learner, directly or indirectly, “PR: Yeah, to see the two that were kind of similar
addresses the possible intentions of a PL: Yeah, and people always look for continuity
designer/writer who has produced the PR: Yeah—
Envisioning text. PL: So if you use like similar, like the same objects
the Designer or like similar objects throughout your images, it’s
like—keep people like [inaudible] I guess. Or like
say if this was a movie or something, they would—
” (NELC3 19246-19562)
35. decision-making strategies
THEME DESCRIPTION EXAMPLE
Learner eliminates items separately “PL: So, alright, let’s just—let’s just cross out the
from content of images or text (as in ones we have. OK. Cause I feel like that helps for
eliminating items already decided us to know what our options are.
the Field upon). PR: So we have F and C, OK” (NELC3 9169-
Learner figures out an answer by “LP: so this is tasahar, well this one is masculine,
relying on knowledge of grammar and so it has to be, so it should be C, right? Because
Language syntax. the other two are both girls.
Mechanics RP: umm , but this one is masculine too
LP: I know, dang” (NELC1 4390-4580)
Image-text Learner changes the focus of “PL: the answer is there is nothing but it, then we
Switching to attention from image to text and back connect it to mmmm…..someone … someone
Check to check or raise confidence in a listening to something, so maybe if he is listening
Interpretatio candidate decision. to something then it could be this one.” (IEP4
36. decision-making strategies
THEME DESCRIPTION EXAMPLE
Learner asks about/confirms “PL: to tell you the truth, I am confused between F
Partner- candidate decision. and B
checking PR: it’s F I think, this how it looks.
PL: Ok then put it as F” (IEP3 2947-3139)
Learner selects an image element or “CG: So like say in C, what made you pick the
a word that allows him to act without blood instead of something else that was going on
explicitly making meaning. in the image?
Extrapolatio PL: Well, for some reason, this last word reminded
n from me of blood
Minimal CG: OK
Cues PL: I don’t know why—I—I had no idea why, it just
did. So I just went with it [laughs] And uh, I mean
they always say go with your first instinct, right?”
Change to title of the proposal
I will review a sample of our results and brieflyexplain each of the primary meaning-making and decision-making strategies using examples from our data. First, the meaning-making strategies that we identified…
In the lived experience theme, one set of learners concluded that the image on the right was a library—because it corresponded with their lived experience of what a library contains—big tables and more books than the classroom.
Extended narrative consists of a learner making up a story using cues within an image or across multiple images. In this example from the warmup activity, the learner imagined the conversation that the speech bubble might symbolize, creating a narrative that was not intended by the creator of the image.
Image-text switching is used to generate possible meanings in an image, with the learners moving between the image and text to find potential meaning. In this example, the learners recognized the image of a library, and then looked in the text for the Arabic word for library.
Learners also negotiated meaning between themselves, reaching consensus on which meaning of a particular image that they would agree on. In this example, one learner noticed the zit on the girl’s face, while the other learner did not, and a shared meaning was negotiated in this conversation.
Internal context involved the identification of salient features across a set of images. In this case, the learners noticed that multiple images included seated girls, and they had to move past this initial interpretation of a set of images to identify which element of the sentence was salient to make the appropriate choice.
This code was applied relatively infrequently, and was always triggered by a question from the researcher rather than emerging naturally in the conversation. The theme identifies instances in which the learners address the potential intentions of the designer of the materials. In this example, the learners are discussing how the images work as a set, and the continuity that learners may see as a “movie” when noticing repeating elements in multiple images.
Next, we will look at the decision-making strategies, which focused on the taskpresented and the choices required to complete the task.
Learners narrowed the field, by eliminating items from consideration separate from meaning. In this case, the learners crossed out answers they had already used, to identify which answers remained.
Learners also took advantage of their knowledge of grammar and syntax. In this example, they identified a word as masculine in gender, which allowed them to eliminate some images that pictured females.
This theme is related to the previous image-text switching theme in meaning-making strategies, but focuses on checking interpretation rather than agreeing upon meaning. In this case, the learners switched back and forth between text and image to check or raise confidence in a decision. In this example, the learners confirm that this image of an individual listening to a iPod matches the text of “listening to something.”
In partner-checking, the learner confirms a decision that they have made together. In this example, the learners codify their answer and make their final selection.
Extrapolation from minimal cues indicates cases where the learner uses an element of an image or word to make a decision, without explicitly making meaning. In this example, the learner identifies a particular word as reminding him of blood, so he chose that answer using his instinct, rather than explicitly recognizing the meaning of the answer he selected.