This document discusses digital multimodal composing (DMC) in English language teaching. It begins by outlining how digital tools have changed communication and the skills needed for language learning. DMC is then defined as going beyond traditional writing to include other modes made possible by digital media, like videos and infographics. The benefits of DMC are that it meets real-world communication needs, engages students, and increases autonomy. However, some argue it could distract from language learning. A case study from a Hong Kong university is presented where students produced scientific videos and reports. The project was scaffolded over several weeks and assessed composition and language use. Analyzing model documentaries helped address multimodality and rhetorical strategies.

1. Digital multimodal composing:
How to address multimodal
communication forms in ELT
Christoph A. Hafner
Department of English
City University of Hong Kong
2020 KATE International Conference
*참고: 발표의 영상 녹화 및 무단 복제, 유포를 하지 말아주세요.
**copyright disclaimer: Any attempt to copy, modify or distribute this presentation is void.

3. Overview
1. Emerging needs: ELT in the digital age
2. Digital multimodal composing
3. How to ‘do’ DMC: A case study from Hong Kong

4. Emerging needs
ELT in the digital age

5. Affordances
of digital
tools
Ways of
doing
Ways of
relating
Ways of
meaning
Ways of
being
Ways of
thinking
Jones & Hafner, 2012

6. Digital literacies and language skills
Speaking and listening
Communication
Reading
Research
Writing
Construction
Digital mediation
Shetzer & Warschauer, 2000
Synchronous CMC
Asynchronous CMC
Online netiquette
‘De-individuation’
Critical literacy
Hypertext
Hypertext
Remixing
Multimodality
Collaboration

7. Engaging with digital literacies
…we argue that literacy pedagogy now must account for the burgeoning
variety of text forms associated with information and multimedia
technologies. This includes understanding and competent control of
representational forms that are becoming increasingly significant in the
overall communications environment, such as visual images and their
relationship to the written word – for instance, visual design in desktop
publishing or the interface of visual and linguistic meaning in
multimedia.
(New London Group, 1996)

8. Digital multimodal
composing (DMC)

9. A definition
• Digital multimodal composing in ELT classes means going beyond
traditional writing forms to include other modes made available by
digital media.
Digital videos Infographics Posters Brochures
Academic
text/image
interaction

10. Why ‘do’ DMC?
• Meets the real-world needs of learners in a digitally mediated world;
• Enhances state-of-the-art language teaching strategies like task-based language teaching
and learning, by allowing for the easy integration of multiple skills;
• Engages students with authentic audiences, providing real-world motivation;
• Is ‘voice-enhancing’ and can ‘embolden struggling writers to express themselves’ (p. 82);
• Heightens genre awareness as a result of transformation processes that occur in the
composition process;
• Increases learner autonomy and encourages independent language practice.
(Belcher, 2017; see also Jiang, 2017)

11. Does DMC ‘distract’
learners from the ‘real’
work of language
learning?
“…for multimodal
writing to be valuable
for language learning
there must be a true
demand for
formulation, i.e., a real
struggle to transform
ideas into language”
(Manchón, 2017)
“multimodality use does
not lessen attention to
language” but “may be
unnecessarily burdensome
for students” (Kim &
Belcher, 2020)
“the use of monomodal
writing as a pre-
multimodal task
production step… might
address Manchón’s (2017)
concern that multimodal
tasks may not facilitate
acquisition” (Lim &
Polio, 2020)DMC: The controversy

12. How to ‘do’ DMC
A case study from Hong Kong

13. A project-based course in English for science
• Applied Biology
• Applied Chemistry
• Applied Physics
• Architectural Studies
• Computing Mathematics
• Environmental Science and
Management
• Surveying
• Veterinary Medicine

14. English for science project
A. Digital video project
B. Written scientific
report
English
for
science
project
A1. Reading/data
collection
A2. Scripting/
storyboarding
A3. Performing/
recording
A4. Editing
A5. Sharing
B2. Writing
B1. Reading/
outlining
B3. Editing/
proofreading

15. Sample topics
Dim the lights
(2013)
Working out
(2016)
Vitamin C
(2012)

16. Scaffolding the project
• Week 2, reading workshop; students to form groups for scientific
documentary;
• Week 3, video workshop, introduction to editing tools;
• Week 4, data collection for scientific documentary complete;
• Week 5, writing workshop, script complete;
• Week 6, video workshop, post production;
• Week 7, final video uploaded and shared through YouTube;
• Week 11, students to submit a draft scientific report for individual feedback.

17. Assessing(Hafner&Ho,2020)

18. Multimodal
pedagogy
The view from the classroom

19. Addressing
multimodality by
analysing ‘models’
• BBC documentary
• Al-Jazeera documentary
• Student documentary
Rhetorical
strategies
Linguistic analysis
Narrative images
Conceptual
representations
Offer and demand
Camera angle
Visual coherence

20. BBC mini-documentary
[This slide had a video clip embedded in it]

21. NARRATOR: In our BBC website survey, 85% of
people think they should drink 2 litres of pure
water on top of their normal diet. But in our quest
to stay young and look beautiful, is 2 litres really
the magic number? To find out, you'll need 1 set
of beauty conscious twins –
TWIN: Drink loads of water, it's going to make
your skin amazing you'll be a goddess.
NARRATOR: – one skin analyzer, and two extra
litres of plain water each day. Then simply add
the extra water to one of the twins.
First and
second person
pronouns
Question
forms
Imperatives
Informal
vocabulary

22. Blind as a bat
[This slide had a video clip embedded in it]

23. Narrative images
What is the (very simple) story of these images?
Who are the participants?
What are they doing?
Why are they doing it? What is their goal?

24. Conceptual representations
What information do these two images provide us with?
What section of the documentary do they come from? (e.g. conclusion)

25. Engaging the audience: Offer and demand
Which image do you feel engages the viewer more? Why?

26. Addressing
multimodality by
analysing ‘models’
• BBC documentary
• Al-Jazeera documentary
• Student documentary
Rhetorical
strategies
Linguistic analysis
Narrative images
Conceptual
representations
Offer and demand
Camera angle
Visual coherence

27. A recent example
‘Working out’

28. [This slide had a video clip embedded in it]

29. Christoph A. Hafner
Department of English, City University of Hong Kong
c.hafner@cityu.edu.hk
*참고: 발표자의 동의없이 발표의 영상 녹화 및 무단 복제, 수정, 유포를 하지 말아주세요.
**copyright disclaimer: you may not copy, modify or distribute this presentation without (a) presenter(s)’s permission.
Contact Information
for Q & A