In this study, the effects of spoken text and visual text as the two multimedia presentations in online educational videos on information overload, learning satisfaction, attention and recall are investigated.

1. Effects of Audio and Text
Multimedia Presentations in
Online Videos for Learning
Soo Young Shin | Irem Gokce Yildirim COM803 - SS17
MICHIGAN STATE UNIVERSITY

2. Outline
1. Introduction
2. Literature Review
3. Research Questions
4. Method
5. Results
6. Discussion
7. Conclusion

3. Introduction
Learning in the New Era

4. Introduction
YouTube as a new learning
platform
▸ User generated content (Web 2.0)
▸ Online educational market is growing:
$107 billion in 2015
▸ How to design and create online
learning video in different modes of
presentation
McCue, 2014; Salmon 2013

5. Introduction
Significance of the study
▸ Learning in online context
▸ Observing the relationship between multimedia
presentations and learning experience
▸ Cognitive Theory of Multimedia Learning and
Cognitive Load Theory
Mayer, 2005; Sweller, 1994

6. Introduction
Research Goal
Observing how different multimedia presentations of information
(auditory and visual) in online educational videos affect learning
experiences.
▸ Concepts:
▹ IV: Multimedia presentations (auditory or visual elements:
visual and spoken text)
▹ DVs: Recall, Information overload, Satisfaction and Attention

7. Literature Review
Concepts Related to Learning

8. Literature Review
Theoretical Approaches
Cognitive Theory of Multimedia Learning (Mayer, 2005) &
Cognitive Load Theory (Sweller, 1998)
Effective ways of learning in multimedia presentation?
▸ dual channel assumption: auditory and visual channels
▸ limited capacity assumption: If the information presented exceeds the capacity
of each channel, then information overload occurs.
▸ active processing assumption: learning is a process of filtering, selecting and
organizing and integrating and this process helps the learner to reduce the information
overload.

9. Literature Review
Multimedia Presentation (Instructional Mode)
▸ How information is presented (audio, text, graphics,
animation etc.)
▸ Cognitive Theory of Multimedia Learning: auditory
and visual channels are not mutually exclusive
process
▸ Audiovisual components are observed in studies

10. “Multimodality vs. Single modality and Learning
Seitz, Kim, & Shams, 2006: Audiovisual presentation.
Kim & Gilman, 2008: Audiovisual presentation (including graphics)
Sunder, 2000: Audiovisual presentation
Reasoning: psychological factors
Kalyuga, 2000: Audiovisual presentation
Reasoning: limited capacity of working memory
Single and multi-channels can be overloaded depending on the
demands for cognitive processing (Mayer and Moreno, 2003)

11. Literature Review
Recall in Learning
▸ Recall as an indicator for learning1
▸ Measured with questions asking what participants recall in written
exams and survey.
▸ Contributing factors (DeFleur, Davenport, & Cronin, 1992):
▹ Prior knowledge
▹ Proximity
▹ Importance
A short quiz consisting of 9 questions is used for this study.
1
Means, Toyama, Murphy, Bakia, & Jones, 2009; Zhang, Zhou, Briggs, & Nunamaker, 2006; Choi & Johnson, 2005

12. Literature Review
Information Overload
Based on Cognitive Load Theory (Sweller, 1994) and Cognitive Theory of
Multimedia Learning (Mayer & Moreno, 2003)
▸ The information presented exceeds the capacity of working
memory, information overload occurs.
▹ Redundancy: Same information is presented in different modes
at the same time, the risk of overloading of learners’ memory
increase.
▹ Ambiguity: If information is presented in a simpler way,
individuals should suffer less information overload.
▸ Perceived information overload
Agnew & Szykman, 2010; Chen et al., 2009; Kalyuga, 2000

13. Literature Review
Learner’s Motivation: Satisfaction and Attention
▸ Keller’s ARCS Model of Motivation (1987)
▹ To be motivated in the learning process
▹ Learners:
▹ are interested in and curious about learning: attention,
▹ believe that the learning is important personal goals or
motives: relevance
▹ expect that they will succeed and control their success:
confidence
▹ Satisfied with the process or result of the learning:
satisfaction

14. Literature Review
Learner’s Motivation: Satisfaction and Attention
Among four factors of motivation…
▸ Attention: first factors to catalyze learners’ motivation to learn
▸ Satisfaction: a direct and major factor to improve learning
outcome such as grades
Extrinsic motivational factors: the design of a multimedia presentation
Intrinsic motivational factors: “visual and auditory input of learning process”
▸ Visual input: text, pictures, diagrams, video, and animation
▸ Auditory input: narration or commentary, spoken instructions, cues, and
music.
Hede, 2002; Loorbach, Peters, Karreman, & Steehouder, 2015

15. Research Questions
Proposed Framework by the Study

16. Research Questions
▸ RQ1. Differences between multimedia presentations (visual text only, spoken text
only [audio], and visual and spoken text together[text and audio]) and recall
▸ RQ2. Differences between multimedia presentations (visual text only, spoken text
only [audio], and visual and spoken text together[text and audio]) and information
overload
▸ RQ3. Differences between multimedia presentations (visual text only, spoken text
only [audio], and visual and spoken text together[text and audio]) and learner’s
motivation (satisfaction and attention)
▸ RQ4. Relationship between multimedia presentations and recall vary according to
prior knowledge and proficiency in language?
▸ RQ5. Relationship between multimedia presentations and information overload
vary according to prior knowledge and proficiency in language?
▸ RQ6. Relationship between multimedia presentations and learner’s motivation
(satisfaction and attention) vary according to prior knowledge and proficiency in
language?

17. Research Questions

18. Method
Participants, Measures and Procedure

19. Method
Online Survey Instrument
Sample
▸ n = 34
▸ 29 undergraduate ; 5 graduate students
▸ 20 male ; 14 female
▸ 21 years old in average
▸ 23 White, 7 Asian, 2 Black or African American, 1 Hispanic, Latino, or Spanish, and 1 did
not disclose
▸ 17 CAS college, 5 Engineering, 5 Broad College of Business, 3 Social Sciences, 2
Education, 1 Natural Sciences, and 1 did not identify
▸ Recruited through SONA and email invitation

20. Method
Stimuli
▸ 3 different versions of the same video
▹ (Condition-1): Visual text only
▹ (Condition-2): Spoken text (audio) only
▹ (Condition-3): Visual and spoken text together
▸ 3 min
▸ Subject: Global warming
▸ Published on YouTube, embedded in survey

21. Method
Stimuli
(Condition-1): Visual text only (Condition-2): Spoken text (audio) only (Condition-3): Visual and spoken
text together

22. Method
Measures
▹ Recall: Short quiz with 9 questions
▹ Information overload: 11 items, 7-Likert, =.93
(Agnew and Szykman, 2010 and Chen et al., 2009)
▹ Satisfaction: 5 items, 7-Likert, =.92
(Keller’s Instructional Materials Motivation Survey-IMMS)
▹ Attention: 7 items, 7-Likert, =.74
(Keller’s Instructional Materials Motivation Survey-IMMS)
▹ Proficiency in Language and Prior Knowledge, 1 item, 5-Likert
“In the last century, how much
has the planet’s temperature
risen?”
“There were too many different
information in the video to
consider”
“Watching the video required a
great deal of thought”
“I enjoyed this video very much
that I would like to know more
about this topic”
“I would recommend this video to
others”
“There was something interesting
at the beginning of the video that
got my attention”
“This video has things that
stimulated my curiosity”

23. Method
Procedure
▸ Sign up through SONA (or email)
▸ Taking the online survey
▹ Rating prior knowledge about the subject
▹ Watching video task
▹ Used timer type question to control the duration of
watching
▹ Complete the survey
▹ 10 min. to complete

24. Results
Analysis

25. Results
▸ Descriptive statistics, ANOVA (no significance found!) and correlation analysis

26. Results

27. Discussion
Evaluation of Results

28. Discussion
Positive correlation between recall and information overload
Information overload
▸ Combination of visual and spoken text > visual text > spoken text (audio)
▹ When the same information is presented in different modes at the same
time, information overload occurs (Redundancy effect).
Ex. Redundancy effect in Kalyuga’s study (2000)
Diagram with visual text, audio text, and with visual text plus audio text
▸ The diagram with audio text group demonstrated a lower
subjective rating and higher test result

29. Discussion
Negative correlation between recall and satisfaction/attention
Satisfaction and attention
▸ Spoken text (audio) > visual text > the combination of visual and spoken text
▹ Video (or video and audio combination) is more preferable medium than
text (Choi and Johnson, 2007; Jonassen et al., 1999)
Personal factors:
Positive correlation between language proficiency and recall
Negative correlation between prior knowledge and recall and information overload

30. Conclusion
Limitations, Implications and Future Work

31. Conclusion
Limitations
▸ Survey Design: Starting with a watching task and being
questioned: frightening!
▸ Study Design: Experimental Study (in a lab) instead of Online
Survey
▸ Participation rate: Small sample size, no significant results!
Similar backgrounds
▸ Subject for Learning: Interesting and unfamiliar subject of the
video

32. Conclusion
Implications
▸ With more larger sample size, significant results in:
▹ Different multimedia presentations in their relations to DVs
▹ Possible mediating effects of information overload and
learner’s motivation on recall?
▹ Possible mediating effects of prior knowledge and language
proficiency on DVs?
▹ Additional variables: More Personal Characteristics
▹ Learning style preferences (Beccue, Whitley, & Vila, 2001; Krätzig & Arbuthnott, 2006)
▹ Learning goal orientations (Roedel, Schraw, & Plake, 1994)

33. Conclusion
Future Work
▸ This was a pilot study!
▹ Changing the design
▹ Repeating the experiment
▹ Generalizability with significant results
▸ With more larger sample size with more variety of
backgrounds
▸ Considering the additional variables

34. References
Chen, C. Y., Pedersen, S., & Murphy, K. L. (2011). Learners' perceived information overload in online learning via computer-mediated communication. Research in Learning Technology, 19(2), 101-116.
Choi, H. J., & Johnson, S. D. (2007). The effect of problem-based video instruction on learner satisfaction, comprehension and retention in college courses. British Journal of Educational Technology, 38(5),
885-895.
Hede, A. (2002). An integrated model of multimedia effects on learning. Journal of educational multimedia and hypermedia, 11(2), 177-191.
Kalyuga, S. (2000). When using sound with a text or picture is not beneficial for learning. Australasian Journal of Educational Technology, 16(2).
Keller, J. M. (2010). Motivational design for learning and performance: The ARCS model approach. Springer Science & Business Media.
Kim, D., & Gilman, D. A. (2008). Effects of text, audio, and graphic aids in multimedia instruction for vocabulary learning. Educational Technology & Society, 11(3), 114-126.
Loorbach, N., Peters, O., Karreman, J., & Steehouder, M. (2015). Validation of the Instructional Materials Motivation Survey (IMMS) in a self‐directed instructional setting aimed at working with technology.
British journal of educational technology, 46(1), 204-218.
Mayer, R. E. (Ed.). (2005). The Cambridge handbook of multimedia learning. Cambridge university press.
Mayer, R. E., & Moreno, R. (2003). Nine ways to reduce cognitive load in multimedia learning. Educational psychologist, 38(1), 43-52.
Seitz, A. R., Kim, R., & Shams, L. (2006). Sound facilitates visual learning. Current Biology, 16(14), 1422-1427.
Sundar, S. S. (2000). Multimedia effects on processing and perception of online news: A study of picture, audio, and video downloads. Journalism & Mass Communication Quarterly, 77(3), 480-499.
Sweller, J. (1994). Cognitive load theory, learning difficulty, and instructional design. Learning and instruction, 4(4), 295-312.

35. Appendix
Qualtrics Survey: https://msu.co1.qualtrics.com/jfe/form/SV_9zSe5wGWZ5Lmm5n
Email Invitation: https://goo.gl/i9OYVn
Videos produced: https://www.youtube.com/playlist?list=PLDiUCQjPqd2moRCHgRoD8aKJxHgeG44kg