Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

An MT Journey Intuit and Welocalize Localization World 2013


Published on

Insights how Intuit, working with Welocalize, architectures a machine translation (MT) program meeting an aggressive launch schedule that now supports the entire enterprise. Presentation given at Localization World 2013 in Silicon Valley

Published in: Business, Technology
  • Be the first to comment

An MT Journey Intuit and Welocalize Localization World 2013

  1. 1. Silver Linings Playbook: Intuit's MT Journey Fri Oct 11 9am Render Chiu, Intuit Group Manager, Global Content & Localization Tuyen Ho, Welocalize Senior Director All other product and service names mentioned are the trademarks of their respective companies. Data contained in this document serve informational and educational purposes only.
  2. 2. MT Journey Outcome?
  3. 3. MT in 3 Months? Silver Linings Playbook is a 2012 American romantic comedy-drama film written and directed by David O. Russell, adapted from the novel The Silver Linings Playbook by Matthew Quick. Reference is for informational purposes only.
  4. 4. • $4.15 billion rev in 2012 • Flagship products: QuickBooks, TurboTax and Quicken • New:, Intuit Money Manager • Markets: North America, Europe, Singapore, Australia, India
  5. 5. Globalization Business Driver: Opportunity to Serve a Global Ecosystem
  6. 6. Business & Technical Landscape • Focus: QuickBooks Online Software • Localization Readiness • Limited i18n of the codebase • In-house team for French Canadian only • Architecture • WorldServer SaaS • Mix of various DBs, authoring tools and CMS • GCL Platform • Go-to-Market Goals • Aggressive goal to SimShip 10 to 20 languages as fast as possible
  7. 7. Team & Products - Today Tax 2 FTE Payroll 2 FTE Mobile 1 FTE QBO 3 FTE SCM process – QBO/QBO-P Build Process -- 1 FTE Simplified Tools FTE Platform &English––21FTE Internal Translation (CA French) – 4 FTE External Translation – 2 FTE 9 Writers, 4 Translators, 2 ENG – 13 Products
  8. 8. Why MT? SPEED SCALE COST ✔ ✔ ✔ QUALITY ? (sure hope so)
  9. 9. Business Case for MT+Post-Editing Benefits Considerations • Efficiencies • Is our UI and UA content suitable • How much do we need to invest in engine training • What efficiency is needed to justify the investment • What about language pairs & productivity, e.g. FIGS higher than CJK? • What tradeoffs do we need to be prepared to make in terms of quality vs cost – 5-100% productivity increase • Target cost savings – 30% lower translation rates • Faster time to market – Needed to launch in less than 4 months • Quality – No compromising on UI content
  10. 10. Challenges (or Reality Check) How do you go global ASAP when you start from ground zero? Requirement Bilingual translations In-house MT expertise MT engine/technology TMS + MT connector Structured Content Status None, except for FR-CA None None None One Major Plus We Had Going for Us: STE
  11. 11. Why Simplified Technical English (STE)? • It’s the international standard • Widespread adoption; started in the aerospace industry, but not limited to that any more • Actively maintained and enhanced • Several checker tools that support it • More precision, less ambiguity • Easier to understand (esp. by non-native English speakers • Easier and cheaper to translate due to clear, unambiguous glossary and sentence structure 11
  12. 12. What Were Our Options Then? Extreme Options We Chose Collaboration • Lower cost by spreading the risk • Speed w/ immediate expertise • Scalability via deep supply chain
  13. 13. Comprehensive MT Approach Drives Quality Output Welocalize has a multi-tiered approach to machine translation (MT) implementation: 1. Evaluate content for MT readiness – source content audit – pre-translation editing – style and glossary verification 2. Assist in selection and integration of one or multiple MT engines into the localization technology ecosystem 3. Perform MT post-editing services – evaluation of MT output quality via workbench – human assessment and automated scoring – engine training feedback / engine improvement 4. Support transition from SaaS/hosted “black box” model to hosted glass box or in-house model
  14. 14. Ensuring Quality with MT+PE Req. gathering Solution Architecture Engine Training Feedback Loop(s) PE Metrics “Go Live” Intuit – Welocalize – MT Engine Coordination: 1) Client formulates the program requirements 2) MT provider, LSP and client define the solution architecture 3) MT or LSP provider trains the engine • • • • • linguistic training metadata analysis workflow architecture feedback loops with automated scores human PE measurement and assessment 4) LSP calculates PE metrics 5) MT-PE projects go “live”
  15. 15. Engine Strategy: SaaS, Trained Use Microsoft Translation Hub engine to achieve immediate cost savings and productivity gains • Automated engine training process, with minimal human involvement • No additional investment required Pros • Cost-effective • Rapid deployment Cons • Less control over engine training and tuning • Potentially lower productivity gains due to engine customization limitations
  16. 16. Engine Integration into L10N Ecosystem Source Source Source Files Files Translationn TMS 1 Segmentation & TM propagation 3 Translation Translation Project (XLIFF Project (XLIFF file w/TM file w/TM propagated propagated for X% for X% matches and matches and higher higher TM TM TM Translation Translation complete complete (TM + MT) (TM + MT) 2 5 7 Terminology Target Files 8 4 MT engine MT engine invoked for invoked for non-TM non-TM segments segments 5 5 MT server 6 2 Translated files uploaded; project complete MT with Post-Editing 7 Postediting 7 Linguistic settings
  17. 17. Post-Editing Philosophy • Language teams familiarized with MT environments • Talent selection and testing is the key • Human quality assessment is performed in a structured non-subjective environment • Post-editing throughput figures are captured by iOmegaT and subsequently analyzed • Translators realize the other benefits of the MT-based process: terminology consistency, predictability of errors, higher degree of control over the integrity of translation
  18. 18. Initial Results with 1 Engine Training BLEU GTM 70 70 60 60 50 50 40 40 30 Bing 30 Hub Hub 20 20 10 10 - Bing -
  19. 19. Bootstrap Approach Fast Cheap Let’s Give it a Try • Adopted SaaS MT ready-to-go engines with prepopulated financial domainspecific data • Created minimum training data with 3K glossary entries and 4.5K TU for first training • Leveraged pre-built MT connector • Applied automatic & human scoring to only a subset of translated data • Experimented with different free engines for branded and support site to gather feedback from customers, test markets, and identify quality gaps
  20. 20. MT Journey Recap 10 Engines & Post Editors Ready for Any Content Requirements or Scope Change Deployed MT Connector, Workflows, Engines + 1 Training 2.5 – 3 months Created Training Data 3 Months Confirmed Target Languages 4.5 months RFP Process 2 months May 2012 July 2012 Sep 2012 Nov 2012 Jan 2013 March 2013
  21. 21. Lessons Learned • Good wine comes from great grapes • You can hire a professional tennis player to play for you • You need a great team and a great partner
  22. 22. Looking Forward • Continue investment on MT quality • Evaluate maintenance & sustainability, e.g. re-training existing engines for improved performance • Expand beyond 10 languages • It’s not all about text
  23. 23. Questions? Contact: Tuyen Ho