An invited presentation given at the University of the Sunshine Coast Learning and Teaching Week. Please cite: Sankey, M. (2015). ‘Multimodal design and using technology for effective learning’, Teaching Visions: See teaching in new ways. University of the Sunshine Coast. 14 – 18 September.

1. Using multimodal design as an enabler
for technology enhanced learning
A/Prof Michael Sankey
Director, Learning Environments and Media
Sankey, M. (2015). ‘Multimodal design and using technology for effective learning’, Teaching
Visions: See teaching in new ways. University of the Sunshine Coast. 14 – 18 September.

2. Abstract
• In HE today we have uni’s using the affordances of
VLEs to increase flexibility.
• We also have an increasingly diverse student body (non-traditional
learners, low SES, 1st in family, international students).
• We have a responsibility to cater for this diversity.
• Age, learning styles, cultural differences, etc., all play a part in this.
• Typically we record the lectures, provide a bit of multimedia, maybe
some audio, lot’s of readings and some quizzes for good measure.
• Not to mention fitting in some social media just to mix-it-up a bit.
• But when designing for diversity we can easily provide too much,
resulting in cognitive overload - limiting the opportunity for learning.
• We will look at some research in this field, provide some examples of
good practice and highlight the key role visualization plays in this.

3. Why multimedia
• Increasingly, we are using multimedia
and apps to develop more engaging interactive and
inclusive, mobile ready online course environments.
• Well designed, these can often be enduring, lasting
for many years.
• Multimedia can be used to represent the content
knowledge using multiple presentation modes (text,
visual, aural and kinaesthetic elements) to
potentially assist different modal preferences.

4. Multimedia learning
• In simple terms Multimedia learning involves
learning from pictures and spoken words (Mayer, 2014).
• e.g., learning from a narrated PowerPoints, computer
game or visualisations.
• Research has identified lots of principles for
presenting words and pictures to optimise learning.
• Generally agreed that materials that use both visual
and words allow two processing subsystems of the
brain to work concurrently, reducing the load on
working, or short-term memory.

5. Three example principles
• Signaling: Students learn better when a multimedia
lesson contains verbal and visual cues to direct cognitive
processing toward the most relevant information.
• Temporal contiguity: Students learn better from narrated
animations when this is presented at the same time as the
corresponding visuals, rather than before or after the
visuals are presented.
• Segmenting: People learn better when a multimedia
lesson is presented in manageable parts (chunks) rather
than as a continuous unit.

6. A practical example
• Observing an instructor drawing
• Signaling
– Directs attention to relevant information
– The hand directs attention to the relevant visual information
• Temporal contiguity
– Integrates words and pictures in time
– Visuals drawn concurrently with verbal explanation
• Segmenting
– Breaks down material into manageable parts
– Visuals drawn in sequence rather than presented all at once

7. The Flipping the classroom
• F-C have become increasing popular for the
re-visioning of student learning opportunities,
particularly since the widespread adoption of VLEs has made
it easier for students to access information online and study
independent of traditional classrooms. (Sankey & Hunt 2013)
• We do a standard lecture, the student goes home to do their
problems, some of them get lost. They spend to much time
being frustrated or, even worse, doing it wrong.
• Lectures at night, “homework” during the day – with support.
• F-C allows us to work on what we used to do as homework
when the teacher is there to help students and/or they are
there to help each other. (Daniel H Pink, 2012)

8. Three examples
Moodle LMS:
• Base content
• Audio enhanced
PowerPoint's
• Mini lectures
• Activities
• Discussion forums
• Library databases
External content:
• YouTube videos
• Case study websites
Classroom:
• Field work
• Visiting experts
• On-campus tutorials
• Off-campus
mediated forums
The Steven Goh flipped model

9. The Jill Lawrence flipped model
Moodle LMS:
• Activity/eTivities
all linked to
assessment
• Base content
• Audio enhanced
PowerPoint's
• Mini lectures
• Links to OERs
• Discussion forums
• Library databases
External content:
• TED Talks videos
• YouTube Videos
• Webpages
ePortfolio:
• Student reflections
• Student resources
On-campus tutorials or off-campus mediated forums
Formative feedback on:
• Early engagement
• Activities/eTivities
• Student reflections

10. The Ellie Kiernan flipped model
Moodle LMS:
• Separate spaces
for on and off
campus students
• Discussion forums
• Extra information
only
• Links to additional
resources and
Library databases
Online package:
• Audio enhanced
PowerPoint's
• Videos interviews
with experts
• Audio recordings
• Interactive
multimedia
• Animations
• Inline-exercises
• Webpages
On-campus students:
• 2 hour tutorial
• Limited discussion
forums
• Required to engage
with Study package
Off-campus students:
• Early online
engagement
• Heavy use of forums

11. Scenario
• You have been told that USC is moving all its courses
into a Flipped Classroom mode of delivery over the next couple
of years.
• This includes increasing the amount of blended delivery
options, to make it possible for students to participate from
off-campus during tutorials.
• At your tables discuss (for a couple of minutes) what this
would mean in the following scenarios.
• Report back in just a few minutes.

12. Scenarios
Table Unit Scenario Table Unit Scenario
1 1st Yr Economics 8 2nd Yr 3D Design
2 2nd Yr Accounting 9 2nd Yr Special Science
3 1st Yr Nursing (core unit) 10 1st Yr Communications
4 3rd Yr Marketing specialisation 11 3rd Yr Tourism
5 2nd Yr Civil Engineering 12 Pathway program
6 1st Yr Early Childhood 13 1st Yr IT foundations
7 1st Yr Intro to Law 14 2nd Yr Statistics

13. • Large bodies of research
indicate that visual cues
help us to better retrieve
and remember information.
• Logical when you consider
the brain is mainly an
image processor (much of
our sensory cortex is
devoted to vision), not a
word processor.
• The part of the brain used
to process words is quite
small in comparison to the
part that processes visuals.
(Kouyoumdjian 2012)

14. Single coding

15. Dual coding

16. • Words are abstract and
rather difficult for the brain
to retain, whereas visuals
are concrete and, as such,
more easily remembered.
• The effective use of visuals
can decrease learning time,
improve comprehension,
enhance retrieval, and
increase retention.
• Visual cues presented in a
book, on the slides, on the
web, are able to more
accurately retrieve and
associated with other
content. (Kouyoumdjian 2012)

17. Design heuristics
• Based on this and other research conducted
over the years by, Ayres, Clark, Fiorella,
Kalyuga, Mayer, Moreno, Paas, Sweller, etc,
etc
• We have developed the Design Heuristics
for Flipped Classrooms, when using
Visualisations.

18. Design heuristics for flipped
classrooms: When using visualisations
Compiled by A/Professor Michael Sankey
University of Southern Queensland.
Multimodal Design Heuristic 1
– “Less is more”. Lean, précised text gets the point across better
than lengthy elaborated text (particularly in PowerPoint
presentations). Students should not have to read large amounts
of information from the screen. If extensive reading is required,
make this available in a form that does not need to be read on
screen, i.e. in an additional printable document.

19. Multimodal Design Heuristic 2
Incorporate images only where they tell a story, providing the learner a
reference point, or anchor for the information being transmitted. Do not use
images for images’ sake. Pedagogical benefit must be present for their use.
Multimodal Design Heuristic 3
Avoid including additional music or sounds with visuals, unless these are an
essential component of the learning interaction, as this can add extra cognitive
load for no reason.
Multimodal Design Heuristic 4
Provide the learner with some control over the learning environment, ensuring
the instructional strategy is made clear. In doing so, have a rich set of
resources available (as an option) to help the learner make decisions. When
providing these alternative representations, allow ample opportunity for
learners to make decisions as to how to access these materials.

20. Multimodal Design Heuristic 5
When creating animation, use image and spoken text. The two sources of
information can then be processed concurrently in working memory. If text
has to be used with spoken text, keep it simple, preferably in dot point form
only.
Multimodal Design Heuristic 6
When using animation, allow access to an alternate version of the material,
thereby catering for those learners who prefer to read instruction rather than
view or listen. This is useful for learners with extensive prior knowledge who
may choose not to access the further information.
Multimodal Design Heuristic 7
Build knowledge gradually with stepwise segments of information
(sequentially), not in one long presentation. The e-learning environment is
especially useful to learners when information is presented in smaller
“chunks” to hold interest. Where this includes audio components
presentations should not exceed 12 minutes, 7 is ideal.

21. Multimodal Design Heuristic 8
Ensure that background image or colour does not interfere with the clarity of
information presented in the foreground. Use contracting colours; light on dark,
dark on light; as against complimentary colours. Use variations in colour or
intensity to highlight important information.
Multimodal Design Heuristic 9
Use simple graphics initially where possible, then add to complexity as the
learning sequence progresses. Scaffold visual learning where appropriate, by
building an illustration sequentially with animation or by a series of still images.
Multimodal Design Heuristic 10
Prevent the need for visual search, that is, make it obvious where to find certain
elements by placing all related information together. Learners will then not have
to hunt for this information. For example, pictures and text should be presented
together on the screen, (simultaneously), rather than separately. The two
representations can then be processed in working memory concurrently. Use
captioned images or incorporate the text into the image, where appropriate.
Avoid referring to an image or diagram that appears on another page or screen
unless it is hyperlinked.

22. Multimodal Design Heuristic 11
The presence of additional multimedia enhancements should be made explicit
by the use of markers or easily recognisable icons. Clear instruction should be
given on how to use these elements for maximum benefit. It should be clearly
indicated whether these elements are an alternative representation or stand-
alone.
Multimodal Design Heuristic 12
The use of video may be preferred for a lecture style presentation. However, if
the speaker is not confident in front of a camera only audio should be used. If
only audio is used this should be made lively. This may be achieved by the use
of vibrant voice, or multiple voices. More visual material should also be
integrated into this style of presentation to reinforce concepts and allow
students to concentrate on something that complements the voice.
Multimodal Design Heuristic 13
Information expressed through, or supported by, human gesture, that is
complimented by speech, help students integrate their knowledge based on
observational cues based in familiarity.

23. Multimodal Design Heuristic 14
The use of alternate printable resources is important, particularly for mature
age students and those with a read/write learning preference. Make printable
materials easy to access and give clear guidance as to what these materials
contain.
Multimodal Design Heuristic 15
If extra elements such as formative quizzes are used, these should be
incorporated and contextualised within the environment, rather than
requiring students to link to an external learning environment. Programmed
feedback should also be provided with formative quizzes.
Multimodal Design Heuristic 16
If audio and video components are used, transcripts of the text should be
made available, if subsequent content does not replicate the audio content.
This will allow students to highlight and make notes on a printed copy, if
required. This transcript does not necessarily need to be word-for-word, but
must contain all the key elements of the audio content and appear in the
same sequence.

24. Some examples
• Heuristic 1
• Heuristic 3
• Heuristic 4
• Heuristic 5
• Heuristic 6
• Heuristic 7
• Heuristic 11
• Heuristic 14
• Heuristic 16

25. • Heuristic 1
• Heuristic 2
• Heuristic 3
• Heuristic 4
• Heuristic 7
• Heuristic 9
• Heuristic 10
• Heuristic 16

26. • Heuristic 1
• Heuristic 2
• Heuristic 3
• Heuristic 4
• Heuristic 5
• Heuristic 6
• Heuristic 7
• Heuristic 8
• Heuristic 11

27. • Heuristic 1
• Heuristic 2
• Heuristic 3
• Heuristic 4
• Heuristic 5
• Heuristic 6
• Heuristic 7
• Heuristic 8
• Heuristic 11

28. • Heuristic 1
• Heuristic 3
• Heuristic 7
• Heuristic 10
• Heuristic 11
• Heuristic 12
• Heuristic 13
• Heuristic 14
• Heuristic 15

30. The role of the teacher
 They are our interface
 In most cases they want
to
 But are not confident
 And not overly aware of
what others are doing
 Staff need to be using the same tools they use to
teach to also learn

31. Example short course
• A Key: Get the staff using the same tools as the students
(if it’s good for the students it’s good for us)
Image accessed from http://www.fusionhq.com/package/templates/0/eatingdogfood.jpg

32. Conclusion
• When designing for blended learning its
important to incorporate visuals into the mix.
• Not because you want them there.
• It’s Because your students want them there.
• The right visuals can help make abstract and
difficult concepts more tangible and welcoming,
as well as make learning more effective and
long lasting. (Kouyoumdjian, 2012)

33. • Questions
• Just remember
Steve is watching
over us.

34. References
• Ayres, P. (2015). State-of-the-Art Research into Multimedia Learning: A Commentary on Mayer’s
Handbook of Multimedia Learning. Applied Cognitive Psychology. 29: pp. 631–636 DOI:
10.1002/acp.3142
• Fiorella, L., & Mayer, R. E. (2015, August 17). Effects of Observing the Instructor Draw Diagrams
on Learning From Multimedia Messages. Journal of Educational Psychology. Advance online
publication. http://dx.doi.org/10.1037/edu0000065
• Kouyoumdjian, H. (2012) Learning Through Visuals: Visual imagery in the classroom. Psychology
Today. Posted Jul 20, https://www.psychologytoday.com/blog/get-psyched/201207/learning-
through-visuals
• Mayer, R. E. (2014). The Cambridge handbook of multimedia learning (2nd edn). New York, NY:
Cambridge University Press.
• Paas, F., & Sweller, J. (2012) An Evolutionary Upgrade of Cognitive Load Theory: Using the Human
Motor System and Collaboration to Support the Learning of Complex Cognitive Tasks. Educational
Psychology Review. Volume 24, Issue 1, pp 27-45
• Pink, D.H. (2012) The Flip Manifesto. The essential Principle. Retrieved July 12, 2013 from
http://theessentialprincipal.blogspot.com.au/2012/02/flip-manifesto-from-d-pink-my-
thoughts.html
• Sankey, M.D, & Hunt, L. (2013). Using technology to enable flipped classrooms whilst sustaining
sound pedagogy. In H. Carter, M. Gosper and J. Hedberg (Eds.), Electric Dreams. Proceedings
ascilite 2013 Sydney. pp.785-795.