1. learners’ strategies
for interpreting
instructional images
elizabeth boling
research abdullah atuwaijri, jiyoon jung,
group colin gray, micah modell, craig howard,
members funda ergulec, muruvvet demiral

2. background and
purpose of the study
- The largely deterministic view of message design (Fleming, 1987; Fleming &
Levie, 1993) and cognitive load theory related to multimedia (Mayer,
Hegarty, S. Mayer, & Campbell, 2005) focuses on properties of images and their
effects on learners’ performance
- The semiotic view (von Engelhardt, 2002; Kress, 2004; Sless, 1986; Van Leeuwen, 2001)
includes the role of learners’ interpretation of visuals, recognizing
that learners are active in their interpretations (Schriver, 1996); that their
aesthetic frame for learning is influential in addition to their cognitive
frame (Parrish, Wilson, & Dunlap, 2010)
- Empirical evidence shows that individuals do not interpret even
simple images as their designers intend, and that they bring
information into their interpretations that is not present in the images
(Boling, Eccarius, Smith, Frick, 2004)

3. research question
what are the strategies
learners use to make
sense of images during
learning activities?

4. participants & context
- Eight dyads, four each from the Intensive English Program
and the Near Eastern Languages and Cultures Program
- First language Arabic learning English
- First language English learning Arabic
- Students completed the activity, agreeing on their choice
of images to match sentences in vocabulary practice --
prompts were used by the researchers to encourage
verbalization; stimulated recall was used to probe for
further data
program male female
Arabic speakers learning
6 2
English
English speakers learning
6 2
Arabic

5. materials
- authentic language learning practice activity including someone giving
blood?
images redesigned using message design guidelines
a girl who is sitting
on a floor?
a girl who is sitting
in a garden?
a pupil in a
classroom?
Which picture
shows a look of
concern?
someone listening
to something?
a library?
someone who is
missing someone?

6. materials
- authentic language learning practice activity including -
images redesigned using message design guidelines
-
-
-

7. data collection & analysis
- Subjects were recorded on video completing the
instructional activity in their native language video
- Arabic video recordings were transcribed and
translated into English by an outside party for further
analysis by the research group arabic
translation
- English video recordings were transcribed and any
Arabic words used were translated
- Analysis was accomplished through stages: transcription
- Repeated viewing of video with transcription
- Coding and re-coding observed themes using Dedoose
- Discussion and agreement during subgroup and
whole-group review of data and themes

8. findings
- Participants in this study, working in the context of
a learning activity, used two readily observable
and distinct (but often co-occurring) types of
strategies:
- meaning-making strategies focused on the images
and their intended significance within the activity
- decision-making strategies focused on the task
presented and the choices required to complete
the task
- Frequent use was made of these strategies
during the short task; the design of images was
not deterministic of their interpretation without
the addition of these strategies

9. meaning-making
strategies

10. meaning-making strategies
lived experience
Learner uses combined
elements of the image to
activate culturally, personally-
situated schema.
–––––––––––––––––––––––––––––
PR, PL: Right, when we see the
image then we could see that
it must be a library as the
classroom does not contain as
many books as the library in
addition to the big tables.‖
(IEP2 4663-5150)

11. meaning-making strategies
extended narrative
Learner makes up a story related
to, but not literally depicted in,
one or several available images.
–––––––––––––––––––––––––––––
PL: Um, well, uh we have—I know
this word means ―news‖ in Arabic,
so I figure like since they’re
talking—there’s like a
conversation about something, I
assume like he’s asking whoever
this is like ―did you see the news‖
or something. Granted, I don’t
know ―to see‖ is in Arabic, but…
[NELC3 2293-2579]

12. meaning-making strategies
image-text switching to generate possibilities
Learner changes the focus of
attention from image to text and
back to generate possible
directions for interpreting
meaning.
–––––––––––––––––––––––––––––
M: Being in the library, pen, So we
need to look for something with
La Albeah [points at the third
image along the top of the sheet.
Then he scans the sentences on
his half of the paper]
[NELC2 1449-1635]

13. meaning-making strategies
negotiating meaning
Learner discusses individual
expectations to reach consensus
on the meaning they will accept.
–––––––––––––––––––––––––––––
PL: She’s looking at something in
the mirror. And when I look close
at that dot, I noticed that erh so
wondered if they were talking
about a zit.
PR: I mean I didn’t even notice
that until then. I was trying to, I just
noticed she was looking at herself.
Trying to find something that
PL: Looking at her eyeball.
[NELC2 10341-10722]

14. meaning-making strategies
internal context
Learner identifies salient features
of an image based on its being
part of a set.
–––––––––––––––––––––––––––––
PR: I think something that like for
me was kind of confusing was like I
was looking at the [inaudible] and
like they were sitting, but there’s
like quite a few where they were
sitting, but then I was like a child,
like which one of these is a child.
That was kind of the problem. You
couldn’t like these two
differentiate
[NELC3 19563-19884]

15. meaning-making strategies
envisioning the designer
Learner, directly or indirectly, addresses the possible
intentions of a designer/writer who has produced the text.
–––––––––––––––––––––––––––––
PR: Yeah, to see the two that were kind of similar
PL: Yeah, and people always look for continuity
PR: Yeah—
PL: So if you use like similar, like the same objects or like
similar objects throughout your images, it’s like—keep
people like [inaudible] I guess. Or like say if this was a
movie or something, they would—
[NELC3 19246-19562]

16. decision-making
strategies

17. decision-making strategies
narrowing the field
Learner eliminates items separately from content of images or
text (as in eliminating items already decided upon).
–––––––––––––––––––––––––––––
PL: So, alright, let’s just—let’s just cross out the ones we have.
OK. Cause I feel like that helps for us to know what our
options are.
PR: So we have F and C, OK
[NELC3 9169-9332]

18. decision-making strategies
language mechanics
Learner figures out an answer by relying on knowledge
of grammar and syntax.
–––––––––––––––––––––––––––––
LP: so this is tasahar, well this one is masculine, so it has to
be, so it should be C, right? Because the other two are
both girls.
RP: umm , but this one is masculine too
LP: I know, dang
[NELC1 4390-4580]

19. decision-making strategies
image-text switching to check interpretation
Learner changes the focus of
attention from image to text and
back to check or raise confidence
in a candidate decision.
–––––––––––––––––––––––––––––
PL: the answer is there is nothing
but it, then we connect it to
mmmm…..someone … someone
listening to something, so maybe if
he is listening to something then it
could be this one.
[IEP4 5705-5889]

20. decision-making strategies
partner-checking
Learner asks about/confirms candidate decision.
–––––––––––––––––––––––––––––
PL: to tell you the truth, I am confused between F and B
PR: it’s F I think, this how it looks.
PL: Ok then put it as F
[IEP3 2947-3139]

21. decision-making strategies
extrapolation from minimal cues
Learner selects an image element or a word that allows him to act
without explicitly making meaning.
–––––––––––––––––––––––––––––
F: So like say in C, what made you pick the blood instead of
something else that was going on in the image?
PL: Well, for some reason, this last word reminded me of blood
F: OK
PL: I don’t know why—I—I had no idea why, it just did. So I just went
with it [laughs] And uh, I mean they always say go with your first
instinct, right?
[NELC3 12491-12995]

22. implications
- Learners are active participants in using images during
learning activities rather than passive recipients of
messages
- Learners are not automatically led to meanings by the
properties of images alone
- Strategies for interpreting images are invoked together
with those for completing the learning task; learners are
not focused or reliant solely on interpretation strategies
- Designers of instructional images cannot rely on
deterministic principles alone to construct effective
images
- Designers of instructional images must attend to the
structure of the learning task, not simply to the content

23. limitations & future research
- Limitations
- small number of participants (n=8 dyads)
- single context of learning
- difficulty of articulating visual strategies
- simple, still images
- Future research
- increased number of participants
- variable ages of participants
- extended contexts of learning
- rigorous training for stimulated recall
- increased complication in visual/textual task

24. references
Boling, E., Eccarius, M., Smith, K., & Frick, T. (2004). Instructional illustrations: Intended meanings and learner interpretations. Journal of
Visual Literacy, 24(2), 185-204.
Fleming, M. L. (1987). Designing pictorial/verbal instruction: Some speculative extensions from research to practice. In The psychology
of illustration. (pp. 136-57). New York: Springer Verlag.
Fleming, M., & Levie, W. H. (1993). Instructional message design: Principles from the behavioral and cognitive sciences (2nd ed.).
Englewood Cliffs, New Jersey: Educational Technology Publications.
Kress, G. (2004). Reading images: Multimodality, representation and new media. Information Design Journal, 12(2), 110-119.
Mayer, R. E., Hegarty, M., Mayer, S., & Campbell, J. (2005). When static media promote active learning: Annotated illustrations versus
narrated animations in multimedia instruction. Journal of Experimental Psychology: Applied, 11(4), 256-65.
Parrish, P., Wilson, B. G., & Dunlap, J. C. (in press). Learning experience as transaction: A framework for instructional design.
Educational Technology.
Schriver, K. A. (1996). Dynamics in document design: Creating text for readers. New York: Wiley Computer Publishing.
Sless, D. (1986). In search of semiotics. Totowa, New Jersey: Barnes & Noble Books.
van Leeuwen, T. (2001). Semiotics and iconography. In T. van Leeuwen & C. Jewitt (Eds.), Handbook of visual analysis. (pp. 92-118).
London: Sage Publications.
von Engelhardt, J. (2002). The language of graphics: A framework for the analysis of syntax and meaning in maps, charts and
diagrams. Amsterdam: Institute for Logic, Language and Computation, Universiteit van Amsterdam.

25. slide assignments
- elizabeth boling … slides 1-3; slides 12-13
- micah modell … slide 4
- jiyoon jung … slide 5
- abdullah atuwaijri … 6
- craig howard … slide 7
- colin gray … slides 8-11

26. boy scout slides
- Images at larger size
- Activity content – English
- Activity content – Arabic
- Warm up activity
- Script for stimulated recall
- Summary slide from Boling, et al.

27. a girl who is sitting someone giving someone who is
a library?
on a floor? blood? missing someone?
a pupil in a girl who is sitting someone listening a look of
a classroom? in a garden? to something? concern?

28. activity content (english)
- someone giving blood? look of concern?
- a girl who is sitting on a - someone listening to
floor? something?
- a girl who is sitting in a - a library?
garden?
- someone who is missing
- a pupil in a classroom? someone?
- Which picture shows a

29. activity content (arabic)
- - -
- -
-
-
- -
-
- -

30. warm-up activity
A B
Which picture shows:
Someone telling a story? __________
Someone reporting the news? __________

31. recruitment script
The researchers for this study have studied graphics used for learning before. We have
seen that it may be difficult for learners to decide what a picture is supposed to mean. In
this study we want to learn more about your thoughts as you learn using materials that
include images.
We want some students to work together in pairs and complete a vocabulary activity
that have been approved by the Intensive English Program (IEP) and the Near Eastern
Languages and Cultures (NELC) department at Indiana University. For each part of the
activity, both of you will need to agree before you mark the answer.
We would like to understand what you say and see what you point to, so we need your
permission to record you on video as you work. None of your personal information will be
shared with anyone outside the study. The video recording will not be seen by anyone
outside the study. If you volunteer for the study, you can stop at any time.
We think it will take about 30 minutes to complete this activity. During the activity you will
be practicing vocabulary you need to know for class. Will some of you volunteer to do
this during your break time?

32. image interpretation
―The International Standards Organization (ISO) standard for correct
interpretations of public information symbols is 85% correct interpretations
(Olgay, 2001). While instructional illustrations, usually seen in context and with
accompanying text, may not need to reach this standard in order to be
useful, it is important to note that no images were interpreted correctly at 85%
or above across all sample groups (although no running - image 10 and
talking - image 8) come close, each with only one sample group missing the
85% mark).‖ (Boling, et al., p. 200, bolding added)
―Even when the threshold for interpretation is adjusted informally to account
for the additional support that may be present in an instructional situations,
designers may be disappointed to know that some sample groups averaged
as little as 47% interpretations consistent with their intentions across the 16
items, and that the highest average for a sample group was only 70%.‖
(Boling, et al., p. 201, bolding added)

33. meaning-making strategies
THEME DESCRIPTION EXAMPLE
Learner uses combined elements of PR, PL: Right, when we see the image then we
the image to activate culturally, could see that it must be a library as the
Lived
personally-situated schema. classroom does not contain as many books as the
Experience library in addition to the big tables.” (IEP2 4663-
5150)
Learner makes up a story related to, “PL: Um, well, uh we have—I know this word
but not literally depicted in, one or means “news” in Arabic, so I figure like since
several available images. they’re talking—there’s like a conversation about
Extended something, I assume like he’s asking whoever this
Narrative is like “did you see the news” or something.
Granted, I don’t know “to see” is in Arabic, but...”
(NELC3 2293-2579)
Image-text Learner changes the focus of “M: Being in the library, pen, So we need to look
attention from image to text and back for something with La Albeah [points at the third
Switching to to generate possible directions for image along the top of the sheet. Then he scans
Generate interpreting meaning. the sentences on his half of the paper]” (NELC2
Possibilities 1449-1635)

34. meaning-making strategies
THEME DESCRIPTION EXAMPLE
Learner discusses individual “M: She’s looking at something in the mirror. And
expectations to reach consensus on when I look close at that dot, I noticed that erh so
the meaning they will accept. wondered if they were talking about a zit.
Negotiating
F: I mean I didn’t even notice that until then. I was
Meaning trying to, I just noticed she was looking at herself.
Trying to find something that
M: Looking at her eyeball.” (NELC2 10341-10722)
Learner identifies salient features of “PR: I think something that like for me was kind of
an image based on its being part of a confusing was like I was looking at the [inaudible]
set. and like they were sitting, but there’s like quite a
Internal
few where they were sitting, but then I was like a
Context child, like which one of these is a child. That was
kind of the problem. You couldn’t like these two
differentiate” (NELC3 19563-19884)
Learner, directly or indirectly, “PR: Yeah, to see the two that were kind of similar
addresses the possible intentions of a PL: Yeah, and people always look for continuity
designer/writer who has produced the PR: Yeah—
Envisioning text. PL: So if you use like similar, like the same objects
the Designer or like similar objects throughout your images, it’s
like—keep people like [inaudible] I guess. Or like
say if this was a movie or something, they would—
” (NELC3 19246-19562)

35. decision-making strategies
THEME DESCRIPTION EXAMPLE
Learner eliminates items separately “PL: So, alright, let’s just—let’s just cross out the
from content of images or text (as in ones we have. OK. Cause I feel like that helps for
Narrowing
eliminating items already decided us to know what our options are.
the Field upon). PR: So we have F and C, OK” (NELC3 9169-
9332)
Learner figures out an answer by “LP: so this is tasahar, well this one is masculine,
relying on knowledge of grammar and so it has to be, so it should be C, right? Because
Language syntax. the other two are both girls.
Mechanics RP: umm , but this one is masculine too
LP: I know, dang” (NELC1 4390-4580)
Image-text Learner changes the focus of “PL: the answer is there is nothing but it, then we
Switching to attention from image to text and back connect it to mmmm…..someone … someone
Check to check or raise confidence in a listening to something, so maybe if he is listening
Interpretatio candidate decision. to something then it could be this one.” (IEP4
n 5705-5889)

36. decision-making strategies
THEME DESCRIPTION EXAMPLE
Learner asks about/confirms “PL: to tell you the truth, I am confused between F
Partner- candidate decision. and B
checking PR: it’s F I think, this how it looks.
PL: Ok then put it as F” (IEP3 2947-3139)
Learner selects an image element or “CG: So like say in C, what made you pick the
a word that allows him to act without blood instead of something else that was going on
explicitly making meaning. in the image?
Extrapolatio PL: Well, for some reason, this last word reminded
n from me of blood
Minimal CG: OK
Cues PL: I don’t know why—I—I had no idea why, it just
did. So I just went with it [laughs] And uh, I mean
they always say go with your first instinct, right?”
(NELC3 12491-12995)