This experiment investigated the effectiveness of five different video design techniques on learning, perception of the instructor, and cognitive load. The designs included instructor-only, slides-only, video-switching between instructor and slides, dual windows with instructor and slides displayed simultaneously, and superimposed slides over the instructor video. The results showed no significant differences in learning outcomes between the designs. Designs including both the instructor and slides received higher ratings for instructor credibility and nonverbal immediacy. There were no significant differences found for general immediacy, cognitive load, or desired to continue the course. Future research could explore more interactive designs or varying the content difficulty.

1. DDL – Multimedia & Video:
Empirical Effects of Multimedia Design on
Perception, Learning Effectiveness, and
Cognitive Load
10/19/2016 1

2. Agenda
 Introduction
 About ODUOnline
 Problem Statement
 Research Questions
 Methods
 Findings
 Conclusion
 Your thoughts? comments? revisions?
2

3. Old Dominion University: ODUOnline
3

4. General Problem
This experiment investigated the effectiveness of several video
design techniques to help determine how these multimedia
designs can be most effectively applied in authentic instructional
design contexts.
4

5.  Independent variable (multimedia presentation design)
 Instructor-only,
 Slides-only,
 Video-switching,
 Dual-windows, and
 Superimposed-slides
5
Research Questions

6. 6
“Instructor-Only” multimedia design:

7. 7
“Slides-Only” multimedia design:

8. 8
“Video-Switching” multimedia design:

9. 9
“Dual-Windows” multimedia design:

10. 10
“Superimposed-Slides” multimedia design:

11. Credibility and Immediacy
 Credibility
 The learner’s perception of the instructors expertise,
concern for students, & trustworthiness
 Immediacy
 Smiling, friendly, gestures, tone, eye-contact
 Ability to effectively communicate
 Both factors that enhance motivation and could impact
learning
11

12. Research Questions
What is the effect of instructor-only, slides-only, video-switching, dual-
windows, and superimposed-slides multimedia presentation designs on:
1. Learning effectiveness? (recall and comprehension pre & post-test)
2. Perceived instructor credibility? (McCroskey & Teven, 1999)
3. Nonverbal immediacy? (Richmond, Gorham, & McCroskey, 1997)
4. General immediacy? (Anderson, 1979)
5. Instructional environment design? (Jayasinghe, Morrison, & Ross, 1997)
6. Cognitive load? (NASA, 1986)
12

13. 13
Research Methods

14. Participants
 450 started the study: 226 = NASA TLX, 211 = survey, and 171 =
post-test.
 ~30 participants randomly assigned into each treatment group
 61.5% female and 30.5% male
 Average age = 27.1, with a median age = 22 years
 Range from 17 to 66
 120 participants reported using laptops, 62 = workstations or
desktops, 39 = phones, and 21 = tablets + mini tablets
14

15. Procedures
 Participant invitations
 Student Announcements (4 weeks)
 Full page ad Mace & Crown, and online ad, (1 week)
 $5 Starbucks or Amazon gift card, drawing for $80
Kindle
 Technical requirements
 Any Internet capable device
 Time required
 Approximately 45 minutes
 Tasks involved….
15

16. Participant Procedure
16
Initial
Data Collection
Website
1. Informed consent
2. Demographics &
20-item Pre-test
3. Assignment into
treatment &
watch video
Instructor-
Only Video
Superimposed-
Slides Video
TLX, Surveys,
And Post-Test
TLX, Surveys,
And Post-Test
Optional
Gift
Card
4. NASA TLX
5. Credibility
6. Nonverbal Immediacy
7. General Immediacy
8. Design
9. 20-item Post-test
10. Gift card selection
Slides-
Only Video
TLX, Surveys,
And Post-Test
Video-
Switching Video
TLX, Surveys,
And Post-Test
Dual-
Windows Video
TLX, Surveys,
And Post-Test

17. 17
Research Findings

18. Research Question 1: Learning Effectiveness
 No statistically significant difference
 ANCOVA, F(4,151) = .56, p = .7.
 All five groups appeared to recall and comprehend the subject
matter
 pre-test scores (M = 10.02, SD = 3.50), and post-test scores (M =
13.00, SD = 3.25), t(156) = 11.53, p < .01
 All five groups performed similarly on the multiple-choice post-test
 Design of each treatment too similar to differentiate?
 (include problem solving in future projects?)
18

19. Research Question 2: Instructor Credibility
 Inclusion of slides appeared to enhance instructor credibility
 ANOVA was significant, F(4,203) = 2.47, p < .05
19
5.75
5.53 5.51 5.49
5.1
4.5
4.7
4.9
5.1
5.3
5.5
5.7
5.9
6.1
6.3
Dual-Windows Superimpossed-Slides Video-Switching Slides-Only Instructor-Only
Source Credibility

20. Research Question 2: Instructor Credibility
 Inclusion of slides appeared to enhance instructor credibility
 ANOVA was significant, F(4,203) = 2.47, p < .05
20
5.75
5.53 5.51 5.49
5.1
4.5
4.7
4.9
5.1
5.3
5.5
5.7
5.9
6.1
6.3
Dual-Windows Superimpossed-Slides Video-Switching Slides-Only Instructor-Only
Source Credibility

21. Research Question 3: Nonverbal Immediacy
 Inclusion of instructor appeared to enhance immediacy
 ANOVA indicated a significant difference between treatment groups, F(4,206) =
4.68, p <.01
21
4.37
4.29 4.27 4.24
3.83
3.5
3.7
3.9
4.1
4.3
4.5
4.7
4.9
Superimpossed-Slides Video-Switching Instructor-Only Dual-Windows Slides-Only
Nonverbal Immediacy

22. Research Question 3: Nonverbal Immediacy
 Inclusion of instructor appeared to enhance immediacy
 ANOVA indicated a significant difference between treatment groups, F(4,206) =
4.68, p <.01
22
4.37
4.29 4.27 4.24
3.83
3.5
3.7
3.9
4.1
4.3
4.5
4.7
4.9
Superimpossed-Slides Video-Switching Instructor-Only Dual-Windows Slides-Only
Nonverbal Immediacy

23. Research Question 4: General Immediacy
 No statistically significant difference
 superimposed-slides group perceived the general immediacy of the
instructor highest,
 though the results were not statistically significant,
F(4,205) = .82, p = .51
 Design of survey may have impacted results
 Participants first required to read a short paragraph that defines
“immediacy”
 Survey items did not ‘fit’ in online survey tool format
 Exclude in future research projects?
23

24. Research Question 5: Instructor Evaluation Measure
 Item 9: desire to continue in the course as presented:
 ANOVA analysis indicated that there was a significant difference between treatments on
this instrument, F (4,206) = 4.03, p < .01
24
4.35
3.44
3.27
3 2.97
2.2
2.4
2.6
2.8
3
3.2
3.4
3.6
3.8
4
4.2
4.4
4.6
4.8
5
Instructor-Only Slides-Only Video-Switching Superimposed-Slides Dual-Windows
Desire to Drop Course

25. Research Question 5: Instructor Evaluation Measure
 Item 9: desire to continue in the course as presented:
 ANOVA analysis indicated that there was a significant difference between treatments on
this instrument, F (4,206) = 4.03, p < .01
25
4.35
3.44
3.27
3 2.97
2.2
2.4
2.6
2.8
3
3.2
3.4
3.6
3.8
4
4.2
4.4
4.6
4.8
5
Instructor-Only Slides-Only Video-Switching Superimposed-Slides Dual-Windows
Desire to Drop Course

26. Research Question 6: Cognitive Load
 No statistically significant differences
 (average TLX score), F(4,221) = .56, p = .69
 HD video/slides + studio grade audio + user centered interface
design = low extraneous cognitive load?
 Subject matter not challenging enough? (low intrinsic cognitive
load?)
 Instructional design not engaging enough? (low application of
germane resources?)
 Make design more interactive? (use different subject matter in
future projects?)
26

27. 27
Other Findings

28. Media Comparison:
 No statistically significant difference
 post-test, F=(3,167) = 1.19 p = .32
28
13.3
12.72 12.57
14.56
11
11.5
12
12.5
13
13.5
14
14.5
15
15.5
16
16.5
17
Desktop/Workstation Laptop Phone Tablet
Device Type compared to Post-test

29. 29
Conclusions

30. Key Findings
 All five designs appear applicable in terms of recall and
comprehension learning effectiveness
 HD studio produced audio and video
 Following our ID’s PowerPoint Guidelines
 Instructor credibility and immediacy can be enhanced by including
both slides/presentation and visuals of the instructor
 Less experienced students require additional support and
motivation
 The pedagogical design of the course is more important than the
technology/media used to deliver that course
30

31. Future Research Directions?
 Subject matter?
 Audio only no visuals?
 Visuals only no audio?
 Text density?
 Graphics only?
 Let students choose?
 TelePresence?
 Interactive instructor?
 Problem solving post-test?
 Standing/walking instructor?
 Shorten survey/post-test
 Transcript window?
 More than 20 minutes?
 Less than 20 minutes?
 Eye-tracking?
31

32. Thank You!
10/19/2016 32
Dr. Miguel Ramlatchan, mramlatc@odu.edu
Dr. Ginger Watson, gswatson@odu.edu
Dr. John Baaki, jbaaki@odu.edu
Dr. Peter Baker, pbaker@odu.edu

33. References
33
Anderson, J. F. (1979). Teacher immediacy as a predictor of teaching effectiveness. In D.
Nimmo (Ed.), Communication Yearbook 3. New Brunswick, NJ: Transaction Books.
Jayasinghe, M. G., Morrison, G. R., & Ross, S. M. (1997). The effect of distance learning
classroom design on student perceptions. Educational Technology Research and
Development, 45(4), 5-19.
McCroskey, J. C., & Teven, J. J. (1999). Goodwill: A reexamination of the construct and its
measurement. Communication Monographs, 66, 90-103.
National Aeronautics and Space Administration, Human Performance Research Group. (1986).
NASA task load index (TLX) v.1.0: Paper and pencil package. Retrieved from
http://humansystems.arc.nasa.gov/groups/tlx/paperpencil.html
Richmond, V. P., Gorham, J. S., & McCroskey, J. C. (1987). The relationship between selected
immediacy behaviors and cognitive learning. Communication Yearbook, 10, 574-590.