An overview of transLectures, plus results at the end of the second year of the project. Internet of Education 2013, 11-12 Nov 2013, Ljubljana, Slovenia. Video of the presentation available online at: http://videolectures.net/internetofeducation2013_diaz_munio_translectures/

1. Internet of Education 2013
12 November 2013
Gonçal Garcés Díaz-Munío
Universitat Politècnica de València
EC FP7 ICT project #287755

2. Outline
• transLectures (tL): motivation and goals
• Video demo
• tL technologies
• Progress and results
• Scientific evaluations
• User evaluations
• Quality control (expert evaluations)
• Implementation and integration
• tL open source tools
12 Nov 2013
2

3. Motivation
• Video lecture repositories and MOOCs
• Thousands of hours of video lectures available
• Hundreds of hours of video lectures recorded
every week
• Most video lectures only available in their
original language
• No subtitles
12 Nov 2013
3

4. Motivation
• Transcriptions and translations are needed
•
•
•
•
•
Accessibility for people with disabilities
Accessibility for speakers of different languages
Search and analysis functions
Automated topic finding
…
12 Nov 2013
4

5. Motivation
• Transcriptions and translations are needed
•
•
•
•
•
Accessibility for people with disabilities
Accessibility for speakers of different languages
Search and analysis functions
Automated topic finding
…
• How do we get there?
12 Nov 2013
5

6. The transLectures approach
1. Automatic Speech Recognition (ASR)
and Machine Translation (MT)
• Adaptation: Taking advantage of the characteristics
of video lecture repositories
• High-quality automatic transcriptions and translations
2. Interactive postediting:
intelligent interaction for reduced effort
12 Nov 2013
6

7. Goals
• Massive adaptation
• Intelligent interaction
• Implementation
• Case studies: Videolectures.NET & Polimedia
• Real-life evaluation
• Integration into Opencast Matterhorn
http://opencast.org/matterhorn/
12 Nov 2013
7

8. The transLectures partners
1
2
Name
Universitat Politècnica de València
Xerox SAS
Country
Spain
France
3
3+
4
Institut Jožef Stefan
Knowledge for All Foundation
RWTH Aachen University
Slovenia
UK
Germany
5
6
EML – European Media Laboratory
DDS – Deluxe Digital Studios
Germany
UK
12 Nov 2013
8

9. Languages
• Transcription (ASR)
• EN
• SL
• ES
• Translation (MT)
•
•
•
•
EN>SL , SL>EN
EN>ES , ES>EN
EN>FR
EN>DE
12 Nov 2013
9

10. transLectures: video demo
http://www.translectures.eu/new-demo-video/
12 Nov 2013
10

11. Massive adaptation
• Characteristics
of video lectures
Just one person
Known speaker
Clear talking
No interruptions
Focused on a topic
Slides
12 Nov 2013
11

12. Massive adaptation
• Known speaker and topic
• Slides
• Related documents
12 Nov 2013
12

13. Scientific evaluations (Y2)
• Transcription results
Worse
• WER: Word Error Rate (%)
• Goal: WER < 25%
• EN, SL, ES
Better
12 Nov 2013
13

14. Scientific evaluations (Y2)
• Translation results
Better
• BLEU
• Goal: BLEU > 30
•
•
•
•
EN>SL , SL>EN
EN>ES , ES>EN
EN>FR
EN>DE
Worse
12 Nov 2013
14

15. Y1 results and comparison
12 Nov 2013
15

16. Y1 results and comparison
12 Nov 2013
16

17. Y1 results and comparison
12 Nov 2013
17

18. Intelligent interaction
• Postediting automatic transcriptions/translations
• The user invests the least possible effort
• The system learns the most from it
• Confidence measures
• Fast constrained search
12 Nov 2013
18

19. Intelligent interaction
12 Nov 2013
19

20. Intelligent interaction
12 Nov 2013
20

21. Intelligent interaction
• Batch interaction vs. Intelligent interaction
12 Nov 2013
21

22. Intelligent interaction
• User evaluations
• UPV (Polimedia)
• JSI (Videolectures.NET)
12 Nov 2013
22

23. User evaluations
• User evaluations at UPV
• Users: lecturers
• Revising their own lectures
• 3 different experiments
1. Complete supervision
2. Intelligent interaction
3. Two-round supervision
12 Nov 2013
23

24. User evaluations
1. Complete supervision
12 Nov 2013
24

25. User evaluations
2. Intelligent interaction
12 Nov 2013
25

26. User evaluations
3. Two-round supervision
12 Nov 2013
26

27. User evaluations
• User evaluations at UPV: results
12 Nov 2013
27

28. User evaluations
• User evaluations at UPV: results
12 Nov 2013
28

29. User evaluations
• User evaluations at UPV: results
12 Nov 2013
29

30. Quality control: expert evaluations
• Transcription quality (EN, ES, SL)
• UPV: Representative set of Spanish transcriptions
• Avg. WER: 23.2 ; Avg. RTF: 3.8
12 Nov 2013
30

31. Quality control: expert evaluations
• Transcription quality (EN, ES, SL)
• UPV: Representative set of Spanish transcriptions
• Avg. WER: 23.2 ; Avg. RTF: 3.8
• Translation quality (EN<>SL, EN<>ES, EN>FR, EN>DE)
• UPV: Representative set of Spanish>English translations
• Avg. BLEU: 46.6 ; Avg. RTF: 14.8 ; Avg. Score: 3.6 out of 5
12 Nov 2013
31

32. Implementation and integration
• Videolectures.NET
• Polimedia
• Opencast Matterhorn
12 Nov 2013
32

33. • Polimedia
12 Nov 2013
33

34. transLectures: Open source tools
• The tL player (& editor)
• Coming soon (www.translectures.eu)
• The transLectures-UPV Toolkit (TLK) for ASR
• www.translectures.eu/tlk
• RWTH Aachen: rASR, Jane (MT)
• http://www-i6.informatik.rwth-aachen.de/web/Software/
12 Nov 2013
34

35. Next steps for transLectures
• Keep improving ASR and MT results
• Keep improving tL open source tools (TLK, tL player)
• External user evaluations (VL.NET and polimedia)
• External trials: implementation in other universities
12 Nov 2013
35

36. www.translectures.eu
• About this presentation:
Gonçal Garcés
• Project coordinator:
Alfons Juan-Ciscar
ggarces@dsic.upv.es
ajuan@dsic.upv.es
EC FP7 ICT Programme – Project Number 287755
12 Nov 2013
36