Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

TAUS Global Content Summit Amsterdam 2019 / The Quantum Leap: Human Parity, Chris Wendt (Microsoft)

129 views

Published on

The Quantum Leap: Human Parity, Chris Wendt (Microsoft)

Published in: Data & Analytics
  • DOWNLOAD FULL ▶ ▶ ▶ ▶ http://bit.ly/37jCEgO ◀ ◀ ◀ ◀
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

TAUS Global Content Summit Amsterdam 2019 / The Quantum Leap: Human Parity, Chris Wendt (Microsoft)

  1. 1. Translator
  2. 2. Human Machine 1M words $250000 $70
  3. 3. Credit: Chris Manning 2016
  4. 4. Edited from Chris Manning 2016
  5. 5. Are human translations perfect? No!
  6. 6. Rank Z score Human eval score System ID 1 0.237 69.0 Combo-6 0.220 68.5 Human translation from scratch 0.216 68.9 Combo-5 0.211 68.6 Combo-4 0.141 67.3 Human post-edit of machine translation 2 -0.094 62.3 Sogou (winner of WMT2017 -0.115 62.1 Crowd-sourced human translation 3 -0.398 56.0 Online-A-1710 -0.468 54.1 Online-B-1710
  7. 7. Criticism: Claim of “Human Parity” Each time someone makes a claim like this, there is criticism. Here: 1. Test set includes the other direction Half of the test set were original English news sentences translated to Chinese 2. Sentences, not documents Translation as well as evaluation is per sentence, no document context given 3. Evaluators were not qualified translators Translators were, but evaluators were not qualified translators I say not valid. We are not translating for an audience of professional translators, we are translating for our audience.
  8. 8. • Processes the whole sentence • Can store additional information in hidden layers • Language ID • Domain • A phrase from the previous sentence • A representation of the document context • GPS location • Age of the speaker …
  9. 9. Microsoft Confidential – NDA Only
  10. 10. Microsoft Confidential – NDA Only
  11. 11. Microsoft Confidential – NDA Only
  12. 12. Classroom setup

×