Downloaded 28 times
Uploaded byKonstantin Savenkov
Progress in Commercial Machine Translation Systems
The document discusses advancements in commercial machine translation (MT) systems, focusing on various providers, supported language pairs, and the importance of MT quality evaluation. It highlights the evolution of MT technologies, including domain-adaptive NMT and custom terminology support, emphasizing the need for improvement beyond generic models. Additionally, it raises concerns about data collection practices related to MT services and the impact of enterprise versus user adoption.
More Related Content
Similar to Progress in Commercial Machine Translation Systems
Progress in Commercial Machine Translation Systems
- 1. PROGRESS IN COMMERCIAL MACHINE TRANSLATIONSYSTEMS by Konstantin Savenkov, Ph.D., CEO Intento October 29-30, 2019 Stanford University, Human-Centered Artificial Intelligence (HAI) and AI Index Workshop on Measurement in AI Policy: Opportunities and Challenges
- 2. Intento Alibaba Amazon Baidu Cloud Translate DeepLeBay Globalese Google GTCom IBM Iconic Kakao KantanMT Microsoft Mirai ModernMT Naver Niutrans Omniscien Pangea MT PROMT PrompsIT Rozetta SAP SDL Sogou Systran Tencent Tilde Yandex Youdao COMMERCIAL MT SYSTEMS 2 All product names, trademarks and registered trademarks are property of their respective owners. All company, product and service names used in this website are for identification purposes only. Use of these names, trademarks and brands does not imply endorsement. © Intento, Inc. / October 2019
- 3. Intento VENDOR DYNAMICS (STOCKMODELS) 3 Commercial Alibaba, Amazon, Baidu, CloudTranslate, DeepL, Google, GTCom, IBM, Mirai, Microsoft, ModernMT, Naver, Niutrans, PROMT, Rozetta, SAP, SDL, Sogou, Systran, Tilde, Tencent, Yandex, Youdao Preview / Limited eBay, Kakao, QCRI 0 5 10 15 20 25 Mar 18 Jul 18 Dec 18 Jun 19 Nov 19 Preview Commercial Intento, Inc. • June 2019 © Intento, Inc. / October 2019
- 4. Intento SUPPORTED LANGUAGE PAIRS 4 1 100 10000 N iutrans G oogle Yandex M icrosoftv3 Sogou Baidu Am azon Kakao Systran Tencent SDL PRO M T G TC om SAP DeepL M odernM TIBM W atson v3 N aver Youdao Alibaba eBay Tilde 1 3 2 54 6 8 272 2 202 1 2 20 24 38 5256 72 9090 111121122139 342 594 756 3 4223 782 7 482 10 50613 340 Total Unique *where possible, we have checked via API if all language pairs advertised by the documentation are supported and removed the pairs we were unable to locate in the API. ** as advertised (not validated via API) Unique language pairs - supported exclusively by one provider © Intento, Inc. / October 2019
- 5. Intento MT QUALITY EVALUATION 5 Intentomonitors MT Quality since May 2017 (public report every 4-6 months). — 48 popular language pairs, based on WMT and other public news corpora. — Reference-based evaluation using hLEPOR score (n=2000, statistically significant) © Intento, Inc. / October 2019
- 6. Intento BEST MT ENGINES (AS OF JUNE2019) 6 en ru ja de es fr pt it zh cs tr fi ro ko ar nl en ru ja de es fr pt it zh cs tr fi ro ko ar nl MT Engines deepl google amazon yandex systran-pnmt modernmt ibm promt microsoft tencent baidu 6 In several cases, there’s no statistically significant difference between the top engines. changed since Jan 2019: 19 pairs © Intento, Inc. / October 2019
- 7. Intento MORE INVESTMENT INMT QUALITY GOES INTO POPULAR LANGUAGE PAIRS 7 Intento data curation — new architectures — direct translation © Intento, Inc. / October 2019
- 8. Intento MT PROGRESS BEYONDLOW-IMPACT CONTENT REQUIRES MORE THAN GENERIC MODELS 8 Intento Cross-language NLP High-volume low impact Low-impact (inbound etc) High-impact generic High-impact in- domain MACHINES HUMANS © Intento, Inc. / October 2019
- 9. Intento MT PROGRESS BEYONDLOW-IMPACT CONTENT REQUIRES MORE THAN GENERIC MODELS 9 Intento Cross-language NLP High-volume low impact Low-impact (inbound etc) High-impact generic High-impact in- domain MACHINES HUMANS HOT TOPIC © Intento, Inc. / October 2019
- 10. Intento 2018: RAISE OFDOMAIN-ADAPTIVE NMT 10 Intento Sep 2017 Oct 2018 Nov 2017 May 2018 Jun 2018 Jul 2018 Globalese Custom NMT Lilt Adaptive NMT IBM Custom NMT Microsoft Custom Translator Google AutoML Translation SDL ETS 8.0 ModernMT Enterprise Apr 2018 Systran PNMT © Intento, Inc. / October 2019
- 11. Intento 2019: CUSTOM TERMINOLOGYSUPPORT 11 Intento Jun 2018 Oct 2019 Oct 2018 Jan 2019 Apr 2019 Amazon Translate Google Translate v3 SDL BeGlobal 4.1 Microsoft Custom Translator Nov 2018 Systran PNMT IBM Custom NMT “forced glossary customisation” “phrase dictionaries” “custom terminology” “syntax-aware custom terminology” May 2019 Yandex Cloud Translate v2 dynamic glossaries “glossaries” “glossary feature” © Intento, Inc. / October 2019
- 12. Intento IMPROVEMENT BEYOND STOCKMODELS 12 Intento Stock models define starting points — Adaptation based on Translation Memory and Terminology drives further improvement — Depends on architecture, data volume and quality © Intento, Inc. / October 2019
- 13. Intento GENERIC STOCK MODELS AlibabaAmazon Baidu DeepL eBay Google GTCom IBM Kakao Microsoft Mirai ModernMT Niutrans Naver Omniscien PROMT Rozetta SAP SDL Sogou Systran Tencent Tilde Yandex DOMAIN ADAPTATION CAPABILITIES 13© Intento, Inc. / October 2019 VERTICAL STOCK MODELS CUSTOM TERMINOLOGY SUPPORT AUTO DOMAIN ADAPTATION MANUAL DOMAIN ADAPTATION Youdao Alibaba Baidu Cloud Translate Microsoft Omniscien PROMT SAP Systran Amazon Baidu Google IBM Microsoft Rozetta SDL Systran Yandex Globalese Google IBM Kantan Microsoft ModernMT Omniscien SDL Systran Alibaba Baidu Cloud Translate Iconic Omniscien PangeaMT Prompsit PROMT SDL Systran Tilde Yandex All product names, trademarks and registered trademarks are property of their respective owners. All company, product and service names used in this website are for identification purposes only. Use of these names, trademarks and brands does not imply endorsement.
- 14. Intento DATA COLLECTION PRACTICESNEED TO MATCH GROWING MT UBIQUITY 14 Growing MT quality makes it ubiquitous — Enterprise adoption is far behind user adoption — Data collection policy remains in the fine print of “free” MT services — That’s more important than collecting cookies (we think) “We recently found that ~2Gb of confidential data monthly goes from our network to (free MT service)” company Y (2019) “We tried to block traffic to (free MT service), but SVP said it will stop the entire company’s operations” company X (2018) “We discovered text that had been typed in on (MT service) could be found by anyone conducting a web search.” Statoil (Sept 2017, link) © Intento, Inc. / October 2019
- 15. THANKS! by Konstantin Savenkov, Ph.D., CEO Intento October 29-30, 2019 Stanford University, Human-Centered Artificial Intelligence (HAI) and AI Index Workshop on Measurement in AI Policy: Opportunities and Challenges
- 16. THANK YOU! Konstantin Savenkov ks@inten.to 2150Shattuck Ave Berkeley CA 94704 INTENTO https://inten.to 16