Real Time Translation for Live Captioning Workflows
1. Leveraging AWS Translate:
Real Time Translation for
Live Captioning Workflows
Presented by: Giovanni Galvez
VP of Business Development
Syncwords
Lydy Pinzón-Dadley
Director, Sales
National Captioning Institute
2. Your online captioning solution
Since 2010, SyncWords is the trusted partner for
captioning and subtitling service providers and media
companies.
The SyncWords platform combines automation and
authoring tools for high-quality closed captioning and
subtitling. Helping customers reach new audiences by
leveraging machine translation.
Using advanced artificial intelligence technology,
SyncWords delivers dramatic productivity gains,
expanded workload capacity, and improvements in
caption quality and accuracy.
3. For four decades, the National Captioning Institute has
pioneered excellence in accessibility by developing
technologies and solutions for the broadcast TV industry.
• 200,000 hours of media captioned per year
• Developed the closed caption decoder box
• First caption-decoding microchip for TV sets
• 40th anniversary airing the first closed captioned
broadcast with the “Wonderful World of Disney” and
“Masterpiece Theatre” (March 16th, 1980)
In 2011, NCI and Syncwords launched the innovative Recapd
system for live captions delivered to internet enabled
devices and online meetings.
More recently, there was an opportunity to develop AI
powered machine translation for live captioning in order to
localize live news and sporting events.
4. Use Case Challenges for Live Event Translation
• Constant 24/7 live feeds translated in real-time
• Sourcing Multilingual Human Captioners
• Scheduling Workforce
• Scaling live human captioners on-demand
• Expense could be a non-starter and varies
• Requirements for delivery to simultaneous multiple
languages
• Quality and Accuracy of real-time translation
• Latency of the translated captions
5. Syncwords’ Role in Solving the Problem
NCI approached Syncwords as their technology partner to help
automate live translations by connecting the existing live TV
broadcast workflow to AWS Translate.
This includes development of the following:
• A mechanism for receiving input from captioning software
• Real-time text formatting and machine-translation
• Delivery to broadcast TV hardware
• Live OTT delivery of translated subtitles to streaming player
platform
7. Typical AWS Translate in Offline Workflows
English text
Document or
Subtitle file
French text
document
Spanish
subtitling
Japanese
Timed Script
Most workflows use case for machine translation for existing archived documents
Amazon Translate
8. LanguageSync
Mechanism
NCI
Realtime 24/7
Caption Feeds
Leveraging AWS Translate for 24/7 Real-Time Workflows
Ingest and Setup Real-time Translation Real-time Delivery
Service provider
SyncWords
Live Live Broadcast
Translated
Captions
(608/708)
Live OTT
Translated
Captions
(WebVTT Text Tracks)
Language selection
and configuration.
Recapd
Translated
Live Online
Meetings
Translates
Protocols and
manages text
formatting
Amazon Translate
9. Outcome and Benefits
Syncwords Live Machine Translation Delivers:
ROI from Existing Production-Ready Live Workflows
Multiply Subscription Viewership
Increase Ad Revenue
Reach new Demographics and global markets
Expand Accessibility to 40+ Languages
Note: Closed Captioning is required by law in the U.S.A,
Canada, and Mexico
10. Live Translation for Live OTT
Example of Actual Live Stream in 6 languages
Increase Viewer Subscriptions with Translation
11. Live Translation for Live OTT
Challenges that we solved
• Integrating with existing production-ready Live OTT workflows
• Convert text to streaming format subtitles
• No need for transcoding video to add translated subtitles
• Compatibility with streaming platforms
• Simultaneous language outputs
• Scaling up to 40+ languages on output
12. LanguageSync vs. Human Translation
LanguageSync Live Translation reduces latency vs. Live Human Translation
13. LanguageSync vs. Human Translation
LanguageSync Live Translation increases coverage of translation to 100%
14. Automatic Live Translation Relies on Human Accuracy
Real-Time
Human Captioner
From NCI
Cloud AI
Translation
(40+ Languages)
Mountain Summit
of Accuracy
Hybrid Human-AI Approach:
• UX is our focus
• Input language must be accurately
captioned by a human
• Grammar and punctuation is
leveraged to boost accuracy
• This is ready for Primetime
15. IP Connection from
Real-time Captioner
Encoder / Inserter
Fitting into existing live captioning workflows
English
Text Input English Text
Input
Remote
Encoder / Inserter
Translated
Text
Internet
Streaming
TV Broadcast
LanguageSync is compatible with existing
live CC hardware and software
Amazon Translate
16. IP Connection from
Real-time Captioner
Live OTT Translation Workflows with Syncwords
Real-time Translated Text
for OTT Streaming
Live Translation
No need for expensive new hardware
for Live OTT-Only solution
NCI
Realtime
Caption Feed
Service provider
17. CDN
No transcoding of
video necessary
LanguageSync Workflow for Live OTT
LanguageSync real-time machine translation doesn’t
change the workflow and comes over the top to the CDN
Player
OTT
Real-time Translation with
AWS Translate
Receives live video
And subtitling
Español
中文
Française
Deutsche
Bahasa
日本語
…40+
Contribution
Side-Car Subtitle Streams
VTT
NCI
Realtime
Caption Feed
Service provider
AWS Elemental
MediaLive
18. LS takes
MPEG time stamps
to synchronize VTT files
CDN
Technical Requirements
The Master Playlist is refreshed continuously
m3u8
Origin
Server
For Each
Language
VTT
m3u8
Master
Playlist
Player
Series of VTT
Playlist Files
Series of VTT
Subtitle Files
Master m3u8
incorporates playlist
cues for each language
Español
中文
Française
Deutsche
Bahasa
日本語
…40+
NCI
Realtime
Caption Feed
Service provider
19. Leveraging AWS Translate for Live Captioning Workflows
For more info contact:
Giovanni Galvez
gio@syncwords.com
Lydy Pinzón-Dadley
LDadley@NCICAP.org
Editor's Notes
Timed Text Speech can dramatically speed up the captioning and subtitling process by delivering a timed transcription file back to the user. In many cases 90% accuracy can be achieved. Timed Text Speech uses a combination of IBM Watson and Google Speech to deliver auto-transcription in 100 languages.
Timed Text Speech can dramatically speed up the captioning and subtitling process by delivering a timed transcription file back to the user. In many cases 90% accuracy can be achieved. Timed Text Speech uses a combination of IBM Watson and Google Speech to deliver auto-transcription in 100 languages.
Timed Text Speech can dramatically speed up the captioning and subtitling process by delivering a timed transcription file back to the user. In many cases 90% accuracy can be achieved. Timed Text Speech uses a combination of IBM Watson and Google Speech to deliver auto-transcription in 100 languages.
Timed Text Speech can dramatically speed up the captioning and subtitling process by delivering a timed transcription file back to the user. In many cases 90% accuracy can be achieved. Timed Text Speech uses a combination of IBM Watson and Google Speech to deliver auto-transcription in 100 languages.
Timed Text Speech can dramatically speed up the captioning and subtitling process by delivering a timed transcription file back to the user. In many cases 90% accuracy can be achieved. Timed Text Speech uses a combination of IBM Watson and Google Speech to deliver auto-transcription in 100 languages.
Timed Text Speech can dramatically speed up the captioning and subtitling process by delivering a timed transcription file back to the user. In many cases 90% accuracy can be achieved. Timed Text Speech uses a combination of IBM Watson and Google Speech to deliver auto-transcription in 100 languages.
Timed Text Speech can dramatically speed up the captioning and subtitling process by delivering a timed transcription file back to the user. In many cases 90% accuracy can be achieved. Timed Text Speech uses a combination of IBM Watson and Google Speech to deliver auto-transcription in 100 languages.
Timed Text Speech can dramatically speed up the captioning and subtitling process by delivering a timed transcription file back to the user. In many cases 90% accuracy can be achieved. Timed Text Speech uses a combination of IBM Watson and Google Speech to deliver auto-transcription in 100 languages.
Timed Text Speech can dramatically speed up the captioning and subtitling process by delivering a timed transcription file back to the user. In many cases 90% accuracy can be achieved. Timed Text Speech uses a combination of IBM Watson and Google Speech to deliver auto-transcription in 100 languages.
Timed Text Speech can dramatically speed up the captioning and subtitling process by delivering a timed transcription file back to the user. In many cases 90% accuracy can be achieved. Timed Text Speech uses a combination of IBM Watson and Google Speech to deliver auto-transcription in 100 languages.
Timed Text Speech can dramatically speed up the captioning and subtitling process by delivering a timed transcription file back to the user. In many cases 90% accuracy can be achieved. Timed Text Speech uses a combination of IBM Watson and Google Speech to deliver auto-transcription in 100 languages.
Timed Text Speech can dramatically speed up the captioning and subtitling process by delivering a timed transcription file back to the user. In many cases 90% accuracy can be achieved. Timed Text Speech uses a combination of IBM Watson and Google Speech to deliver auto-transcription in 100 languages.
For this presentation we will focus on the text input to the CC encoder/inserter
For this presentation we will focus on the text input to the CC encoder/inserter
For this presentation we will focus on the text input to the CC encoder/inserter
For this presentation we will focus on the text input to the CC encoder/inserter
Timed Text Speech can dramatically speed up the captioning and subtitling process by delivering a timed transcription file back to the user. In many cases 90% accuracy can be achieved. Timed Text Speech uses a combination of IBM Watson and Google Speech to deliver auto-transcription in 100 languages.