This presentation focuses on the use of Audiovisual Translation in foreign language learning and it presents ClipFlair, a web platform specifically designed for this purpose. Language teachers often resort to video to present their students with linguistic and cultural aspects of communication in their context. Since learning-by-doing is generally considered more effective than learning-by-viewing, they try to find active tasks for their learners, such as note-taking, answering questions, summarizing or discussing the video with peers. Familiar Audiovisual Translation modalities, such as subtitling and dubbing, can be used in this context as multimodal resources that can account for a very active and motivating educational framework. ClipFlair proposes an authentic way of working with audiovisual material which results in a product, valuable in its own right: a subtitled or dubbed clip. Learners are asked to add to the clip their own subtitles, captions for the Deaf and Hard-of-Hearing, annotations or intertitles. Alternatively, they can record their voice to simulate foreign film dubbing, voice-over, free commentary, or audio description for the blind. Clips can be short video or audio files, including documentaries, film scenes, news pieces, animations and songs.

1. Audiovisual translation for
foreign language learning:
new multimodal resources
Stavroula Sokoli
Universitat Pompeu Fabra
TISLID Conference
Ávila, 8 May 2014

2. • New educational tools and ideas for foreign
language learning
• Active ways to work with audiovisual materials such
as films
• Authentic tasks to enhance authentic materials
2
Learner and teacher needs

3. Because the audiovisual mode:
• offers variety and flexibility
• contextualizes linguistic items
• provides exposure to cultural and nonverbal
elements
• is closer to natural ideal communication than the
written or oral modes
3
Why Clips?

4. • Learning by doing
▫ more effective than learning by watching
• Active approach, hands-on experience
▫ increases motivation
• “Tangible”, shareable result: the captioned or
revoiced clip
▫ as opposed to note-taking
4
Why ClipFlair?

5. What is ClipFlair?
• A web platform for foreign language learning
through interactive revoicing and captioning of clips
• A community of learners and teachers who are
interested in using audiovisual material for language
learning in an active and motivating way
• Watch this one minute animation
5

6. • The Gallery: for uploading, accessing and sharing
activities, clips and other materials
http://gallery.clipflair.net
• The Studio: for creating and using activities through
revoicing and captioning tools
http://studio.clipflair.net
• The Social Network: for interacting, collaborating
etc. in forums and groups http://social.clipflair.net
• The Conceptual Framework establishing guidelines
for activity creation and evaluation criteria
6
ClipFlair offers:

7. 7
ClipFlair basics
Click here to watch a three-minute video

8. • To add speech to a clip
▫ Dubbing: a new sound track and especially dialogue
in a different language
▫ Voice over: words that are spoken in a movie or
television program by a person who is not seen
▫ Audio description: description of what is happening
in the film - for the blind and visually impaired
▫ Free commentary, karaoke singing, reciting
8
What is revoicing?

9. • To add writing to a clip
9
What is captioning?
Intertitles Foreign film subtitles
Also: subtitles for the deaf and Hard-of-Hearing, annotations,
speech bubbles, comments

10. • En casa de los padres de Rosa
• For ClipFlair beginners
• Does not require subtitling knowledge
• Fill-in-the-gaps exercise
resulting in an authentic
product
10
Subtitling activity example

11. 11
Audio description activity example
• Edinburgh tourist guide for the blind
• Raises awareness of accessibility issues
• Promotes audiovisual literacy
• For practicing speaking skills

12. 12
The Social Network:
http://social.clipflair.net

13. • ClipFlair is not only about standard subtitling
and dubbing
• The possible kinds of activities are multiplied
depending on combinations of:
▫ Skills “From X to Y”
▫ Language combinations
▫ Types of learner response
▫ Audiovisual skills developed
• Available here: http://clipflair.net/outcomes
13
The Conceptual Framework

14. • Prompt: Stimulation, input. Teacher-driven or activity-
authored.
The term from. Learners write, speak and AV produce,
responding to prompts. We can extend the concept of
from to mean involves doing (in pursuit of final goal: to,
output), including prompts and what the learner may
do in the process of getting to their target, i.e. to.
• X-to-Y: An activity provides X as a prompt for learner to
produce Y.
X could be one of the four skills; Y, too.
Y can also specify: verbatim, gist, or react. “from-X-to-
Y” can also be expressed as: X-to-Y.
14
From-X-to-Y prompts

15. • L0 >> Language of instructions/description: could be L1 or
L2 but an activity may be introduced or described in any
number of languages. For example, the same intralingual
activity for learning Polish can be used for speakers of
different languages.
• L1 >> Learner’s language: A language the learner knows well
enough to work “from” (the prompt).
• L2 >> Language being learned: The target language to learn.
• L3 >> Other languages: Any other language that might
appear in a clip being used in an activity (when it might be
important to know such information).There may be different
language combinations in an activity.
15
Languages

16. • Intralingual: activity developed (from) and carried out (to) in
one language only, usually designed regardless of the
learners’ L1, L2; L3 level. Intralingual has no interlinguistic
transfer (e.g. translation).
• Interlingual: the clip (from), is in one language; the learner’s
production (to), is in another. Includes activities made
specifically for a given L1. It may (not) be reusable for other
learners. A translation exercise or other interlinguistic
transfer is typical , but our methodology is not rooted in
traditional grammar-translation. We wish to highlight the
rich possibilities of combining the intra- v. interlingual
variable with repeat-rephrase-react options.
intralingual L2-to-L2 // interlingual L1-to-L2 / L2-to-L1.
16
Multilingual combination

17. • Multilingual: scenarios and combinations which are
not strict L1-to-L2 or L2-to-L1 interlingual exercises;
▫ either because an L3 (some other language) is involved
(e.g. in the clip)
▫ or because the learner’s expected output is a mixture of
L1 and L2.
▫ “multilingual” can also signal that the clip is not entirely
monolingual even if it only includes the learner’s L1 and
L2 (the combination is not FROM a purely monolingual
clip TO a purely monolingual learner output).
17
Language combinations

18. • Repeat/verbatim: essentially, the same message in the
same words (or quite closely followed), “faithful” lexico-
semantic rendering, paraphrase, or translation.
• Rephrase/gist: a noticeable, deliberate change of
wording; “loose” paraphrase, free rendering or account;
a report; a summary, “free” translation.
• React/respond: a new communicative contribution
(e.g. utterance) prompted or elicited by a previous one.
Including diglossic, multilingual (contribution to a)
conversation.
18
Types of learner response (to-Y)

19. Visual literacy defined
• Literacy refers to the ability to read for knowledge, write
coherently, and think critically about the written word.
• Visual literacy includes in addition the ability to understand all
forms of communication, be it body language, pictures, maps, or
video ...
• Literacy encompasses a complex set of abilities to understand and
use the dominant symbol systems of a culture for personal and
community development.
• In a technological society, the concept of literacy is expanding to
include the media and electronic text, in addition to alphabetic and
number systems. These abilities vary in different social and cultural
contexts.
19

20. Deep understanding of text
• The primary sense of literacy still represents the lifelong,
intellectual process of gaining meaning from a critical
interpretation of the written or printed text.
• Key to all literacy is reading development, a progression of
skills that begins with the ability to understand spoken words
and decode written words, and culminates in the deep
understanding of text.
• Full language literacy, including:
• the abilities to approach printed material with critical analysis,
inference and synthesis;
• to write with accuracy and coherence;
• and to use information and insights from text as the basis for informed
decisions and creative thought.
20

21. 1. Watch: AV interpreting skill for AV texts (e.g. film) as a
whole, single, complex semiotic communication act, to find
meaning and sense from a combination of verbal and non-
verbal signs. It includes interpreting non-verbal pictures,
icons, symbols, metaphors, cultural elements, etc., and any
combination of verbal and non-verbal textual items.
2. AV listen: linguistic oral comprehension with the combined
effect of other elements of the AV text.
3.-AV read: linguistic written comprehension with the
combined effect of other elements of the AV text, and the
ability to read according to the requirements of the screen
(speed, focus, etc.).
21
6 Audiovisual skills

22. 4. AV speak: revoicing (dubbing, karaoke, voice-over, free
commentary), adapt to speed, voice quality, performance,
character-portrayal, etc. Including a variety of prompts, such as
improvising, reading from a script, repeating, mimicking, etc.
5. AV write: effective script writing or captioning in its various
forms. A specific component of “full” AV-production skills. Include
storyboard skills, and visual narrative skills involving the use of the
camera
6. AV produce: film making, film directing. In the context of
education, students and learners being able to produce video
films and clips to a certain standard, displaying a combination of
other semiotic and communicative, linguistic and non-linguistic,
technical and artistic skills.
22
6 Audiovisual skills contd.

23. Read-to-speak:
• (a) repeat/verbatim: read aloud e.g. also interlingual “sight
translation”
• (b) rephrase/gist: e.g. oral report or summary;
• (c) react/answer: e.g. answer, analyze, criticize, agree, etc.
Listen-to-speak:
• (a) repeat: intralingual or interlingual (more or less verbatim); e.g.
traditional “listen-and-repeat” exercises. Liaison and consecutive
interpreting
• (b) rephrase: also intra- or inter-lingual; e.g. simultaneous
interpreting, relay, or paraphrase, or report (e.g. reported speech)
• (c) react: answer questions, turn taking in spontaneous
conversation or debates. Dialogue would involve minimum two
instances of “listen to speak” by two people to each other.
23
X-to-Y combinations between skills

24. Read-to-write: Reading prompts >> elicits writing:
• (a) repeat: intra- or inter-lingually (translation);
intertextuality
• (b) rephrase: summarize; report; adapt;
• (c) react: e.g. book review. Film analysis.
Listen-to-write: Listening prompts >> elicits:
• (a) repeat: intralingual dictation, also possibly on-the-
spot translation, transcription.
• (b) rephrase: e.g. intra-/inter- lingual note-taking.
• (c) react: radio-listener’s letters to the editor or
program, viewers’ texting during TV or radio program.
24
X-to-Y combinations between skills

25. How to create a captioning
activity
1) Plan the activity
a) Decide the learning objectives
b) Decide the task(s)
2) Select a video from the Gallery
http://gallery.clipflair.net
3) Add text component for activity instructions
4) Add captioning component and do the spotting
5) Save the activity as ClipFlair file

26. How to create a revoicing
activity
1) Plan the activity
a) Decide the learning objectives
b) Decide the task(s)
2) Select a video from the Gallery
http://gallery.clipflair.net/
http://clipflair.net/ > Social Network > Gallery > Video
3) Add text component for activity instructions
4) Add captioning component and do the spotting
5) Go to options and enable “Audio visible”
6) Save the activity as ClipFlair file

27. Learner feedback
• One year (since April 2013)
• 967 users
• The questionnaire is available here:
• http://tinyurl.com/clipflairfeedback
27

28. The ClipFlair activity was… (1/2)
28

29. The ClipFlair activity was… (2/2)
29

30. The ClipFlair Studio is…
30

31. About the learners (1/2)
31

32. About the learners (2/2)
32

33. 33
The Partnership
Spain
UK
Greece
Romania
Ireland
Portugal
Poland
Estonia

34. Arabic, Basque, Catalan, Chinese, English, Estonian,
Greek, Irish, Japanese, Polish, Portuguese,
Romanian, Russian, Spanish, Ukrainian
34
Languages targeted in the project

35. More info?
clipflair.net
facebook.com/ClipFlair
twitter.com/ClipFlair
clipflair@cti.gr
Join us at the ClipFlair Conference, Barcelona, 18-19 July 2014
http://clipflair.net/conference2014
35