TAUS USER CONFERENCE 2010
LANGUAGE BUSINESS INNOVATION
4 – 6 OCTOBER / PORTLAND (OR), USA




TUESDAY 5 OCTOBER / 11.00

M...
Data Needed to Build a Custom Engine


    Bilingual Source and Target Language
     Pre-Aligned              • Glossaries...
Basic SMT Steps: Data – Diagnostic - Release


                                                               Diagnostic  ...
Strategies for Scarce Data Scenarios


                              Partnership with Technology-Savvy LSP is Key to Succe...
Training Data: Volume vs. Quality

                               Data Volume                             Asia Online Clea...
Relative BLEU Score Comparisons

                                                            80
         •     The dataset...
With “Clean Data” Correction is Possible




           •     Typically about 10-20 examples for         •   Dirty data pr...
SMT in a Scarce Data Scenario


                           Create Strategic Linguistic Assets for Customization
          ...
Sample of 5-Gram Selection

               hong kong special administrative region         1446   the hong kong special ad...
LSP-Guided SMT Process


  1      Monolingual Data Analysis                   2      Multiple Diagnostic Engines   3   Rel...
Case Study – Leading Online Travel Site


             Rapidly make all web content available in multiple languages

     ...
FR Sample of Scarce Data Customized MT

        Source Material in en-US                                                  ...
DE Sample of Scarce Data Customized MT

        Source Material in en-US                                                  ...
Keys To Success


               •     Collaboration between the MT Vendor, LSP and the Client
               •     High Q...
Contact Information



             Machine Translation in the Imperfect World
    Practical Case Study of Using MT in Sit...
TAUS USER CONFERENCE 2010, Machine translation in the imperfect world - Practical Case Study of Using MT in Situations wit...
Upcoming SlideShare
Loading in …5
×

TAUS USER CONFERENCE 2010, Machine translation in the imperfect world - Practical Case Study of Using MT in Situations with Imperfect or Insufficient Language Data

1,208 views

Published on

Kirti Vashee, Vice-President Enterprise Translation Sales, Asia Online
Rustin Gibbs, Solutions Architect, Moravia Worldwide

Kirti and Rustin provide insights into an innovative approach to the practical use of MT in situations where the bilingual data is of insufficient volume and the monolingual data is of unclear relevance Kirti and Rustin provide examples from travel and publishing industries to show the individual steps of the process to equip participants with information on what language and language technology tools exist to build a high-quality translation engine.

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,208
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

TAUS USER CONFERENCE 2010, Machine translation in the imperfect world - Practical Case Study of Using MT in Situations with Imperfect or Insufficient Language Data

  1. 1. TAUS USER CONFERENCE 2010 LANGUAGE BUSINESS INNOVATION 4 – 6 OCTOBER / PORTLAND (OR), USA TUESDAY 5 OCTOBER / 11.00 MACHINE TRANSLATION IN THE IMPERFECT WORLD Kirti Vashee, Asia Online Rustin Gibbs, Moravia Worldwide
  2. 2. Data Needed to Build a Custom Engine Bilingual Source and Target Language Pre-Aligned • Glossaries Non-Aligned • Translation Memories Minimum: 20,000 Segments 250,000 Words Monolingual Target Language • Documents in target language Minimum: 500MB after cleaning – plain text Copyright © 2010, Asia Online Pte Ltd and Moravia IT a.s.
  3. 3. Basic SMT Steps: Data – Diagnostic - Release Diagnostic Release Version 1 Data 2 3 Engine Engine Clean & Prepare Correct & Improve Translate Copyright © 2010, Asia Online Pte Ltd and Moravia IT a.s.
  4. 4. Strategies for Scarce Data Scenarios Partnership with Technology-Savvy LSP is Key to Success Create & Develop Strategic Linguistic Assets – Identification of high frequency linguistic patterns – High quality monolingual data – Comprehensive domain focused glossaries The Quality of Data is Critical – Smaller amounts of clean data can produce high-quality systems – Clean data systems are more responsive to corrective feedback Copyright © 2010, Asia Online Pte Ltd and Moravia IT a.s.
  5. 5. Training Data: Volume vs. Quality Data Volume Asia Online Cleanliness Rating* Dataset A Dataset B Dataset C 0 5 10 70K, 18% Dataset A 192K, 48% 135K, 34% Dataset B Dataset C *Data optimized for TM tools may often not be suitable for SMT Copyright © 2010, Asia Online Pte Ltd and Moravia IT a.s.
  6. 6. Relative BLEU Score Comparisons 80 • The datasets that were cleaner at the outset produced better results and 70 tend to benefit and improve consistently from Asia Online’s light cleaning 60 efforts 50 Dirty • Dataset A had less data but still produced better results Clean than Dataset B that had 40 twice the data volume 30 • “Dirty” and noisy data has unpredictable results and is 20 much harder to correct and improve Dataset A Dataset B Dataset C Copyright © 2010, Asia Online Pte Ltd and Moravia IT a.s.
  7. 7. With “Clean Data” Correction is Possible • Typically about 10-20 examples for • Dirty data prohibits manual correction. each clean word of phrase. • Individual corrections would not be • Each correction has statistical statistically relevant. relevance and impact can be clearly • Manual corrections would compete seen. against 1,000’s of bad examples. • Corrections usually involve adding Impractical to create enough examples data to fill gaps. manually. • Far less correction of actual errors. • Understanding the cause of errors is difficult. • Clean data means cause of errors can be understood and corrected. • Slows training and overall processing time. Requires more resources. • Concordance used to create unbiased examples/phrases and ensure scope • Only solution is to acquire more dirty covered. data and hope problem is fixed. But may get worse or cause new errors. Copyright © 2010, Asia Online Pte Ltd and Moravia IT a.s.
  8. 8. SMT in a Scarce Data Scenario Create Strategic Linguistic Assets for Customization • Identify High Frequency Linguistic Patterns : N-Grams and translate to create bilingual text • Build comprehensive domain focused glossaries Gather Large Amount s of Target Language Monolingual Data in Domain Produce Frequent Expert Corrective Feedback for MT Output to Drive Quality Improvements Update and Train Engine Frequently in Initial Development Phase Use High Quality In-Domain Foundation Data Copyright © 2010, Asia Online Pte Ltd and Moravia IT a.s.
  9. 9. Sample of 5-Gram Selection hong kong special administrative region 1446 the hong kong special administrative region 1446 the hong kong special administrative 1428 the hong kong special administrative 1428 of the hong kong special 942 of the hong kong special administrative region 942 kong special administrative region shall 460 kong special administrative region shall 460 special administrative region shall be 241 special administrative region shall be 241 the government of the hong 195 the government of the hong 195 government of the hong kong 195 the government of the hong kong 195 in the hong kong special 189 in the hong kong special 189 kong special administrative region may 170 kong special administrative region may 170 council of the hong kong 151 council of the hong kong 151 law of the hong kong 131 law of the hong kong 131 the legislative council of the 130 the legislative council of the 130 basic law of the hong 130 basic law of the hong 130 legislative council of the hong 127 legislative council of the hong 127 the basic law of the 126 the basic law of the 126 standing committee of the national 115 standing committee of the national 115 committee of the national people 115 committee of the national people 115 the standing committee of the 113 the standing committee of the 113 the chief executive of the 113 the chief executive of the 113 kong special administrative region and 105 kong special administrative region and 105 executive of the hong kong 104 executive of the hong kong 104 chief executive of the hong 104 chief executive of the hong kong 104 of the government of the 95 of the government of the 95 the establishment of the hong 86 the establishment of the hong 86 establishment of the hong kong 86 establishment of the hong kong 86 of the legislative council of 84 of the legislative council of 84 for the basic law of 83 for the basic law of 83 committee for the basic law 83 committee for the basic law 83 the court of final appeal 81 the court of final appeal 81 of the court of final 81 of the court of final 81 administrative region of the people 73 administrative region of the people 73 the government of the region 72 the government of the region 72 Copyright © 2010, Asia Online Pte Ltd and Moravia IT a.s.
  10. 10. LSP-Guided SMT Process 1 Monolingual Data Analysis 2 Multiple Diagnostic Engines 3 Release Version Engine Gather Strategic Data Correct, Analyze & Update Translate & Monitor Translate Strategic Data Build, Refine & Prepare Data Copyright © 2010, Asia Online Pte Ltd and Moravia IT a.s.
  11. 11. Case Study – Leading Online Travel Site Rapidly make all web content available in multiple languages • Translate High Frequency N-Grams • Develop Named Entity Handling Strategy • Build Comprehensive Travel Domain Glossary • Gather Large Amounts of In-Domain Target Language Monolingual Data • Use High Quality Foundation Data to Augment Human Translated N-Gram Data • Frequent Corrective Feedback to Early System Output and Frequent Retraining Copyright © 2010, Asia Online Pte Ltd and Moravia IT a.s.
  12. 12. FR Sample of Scarce Data Customized MT Source Material in en-US MT Translation to fr-FR • This hotel is on Times Square and is 2 blocks from 49th Street • Cet hôtel se trouve sur Times Square, à 2 pâtés de maisons de la Subway Station. The property features 37-inch flat-screen TV and 49ème rue de la station de métro. L'établissement dispose de 37 provides an MP3 docking station in every room. pouces à plat-écran TV station d'accueil MP3 et une dans chaque • Crowne Plaza Times Square rooms include coffee maker and room chambre. service. Rooms are located on the 16th floor and above. • Crowne Plaza Times Square chambres comprennent cafetière et • The Times Square Crowne Plaza features a gym, which includes service en chambre. Les chambres sont situées à partir du 16ème cardio equipment, sauna and steam room. The hotel also has a 24- étage au dessus. hour business center. • Le Times Square Crowne Plaza dispose d'une salle de remise en • Guests at the Crowne Plaza can dine at Brasserie 1605, which forme, qui comprend des équipements de cardio, un sauna et un features views of the city and serves steak and seafood. The hotel hammam. L'hôtel dispose également d'un centre d'affaires 24h/24. also offers the Broadway 49 Bar. • Les clients du Crowne Plaza, vous pourrez dîner à la Brasserie 1605, • The Crowne Plaza Times Square is within a 10-minute walk of Radio qui offre des vues sur la ville et sert des steaks et des fruits de mer. City and Rockefeller Center. Central Park is 10 blocks from the hotel L'hôtel propose également la 49 bar Broadway. • Le Crowne Plaza Times Square est à 10 minutes à pied de la ville et le Rockefeller Center. Central Park se trouve 10 pâtés de maisons de l'hôtel. Known as the busiest commercial district in the United States, Midtown Connue des quartiers commerçants les plus fréquentés des États-Unis, Manhattan is home to such famous places as the Rockefeller Center, Midtown Manhattan abrite de nombreux lieux emblématiques tels que le Broadway and Times Square. Other landmarks include the remarkable Rockefeller Center, Broadway et Times Square. Des sites touristiques Empire State Building, Chrysler Building, Flatiron Building and the comme Le remarquable Empire State Building, le Chrysler Building, le beautifully restored Grand Central Station.If shopping is on your agenda, Flatiron Building et le bâtiment superbement restauré de la gare Grand you are certainly in the right neighborhood – just take a stroll down 5th Central. Si vous cherchez Shopping est sur, vous sont certainement dans le Avenue and you will come across names like Tiffany, FAO Schwarz (famous droit - dans le quartier seulement baladez-vous et la 5ème Avenue, des for the floor piano scene in the movie Big), and Saks. Or how about visiting noms comme Tiffany face, FAO Schwarz (fameuse pour l'étage piano dans the impressive Museum of Modern Art? In this diverse and buzzing area la grande scène de cinéma), et Saks. Et pourquoi ne pas aller visiter le the list of world famous attractions just goes on. Come see for yourself! MoMA, l'impressionnant musée d'art moderne de New York? Dans ce quartier animé et varié, la liste des attractions de renommée mondiale est inépuisable. Venez voir par vous-même! Copyright © 2010, Asia Online Pte Ltd and Moravia IT a.s.
  13. 13. DE Sample of Scarce Data Customized MT Source Material in en-US MT Translation to de-DE • This 3-star hotel is in a historic building in Hamburg's city centre, • Dieses 3 - Sterne-Hotel befindet sich in einem historischen Gebäude opposite the main railway station. It offers spacious rooms, free Wi- im Stadtzentrum Hamburgs, gleich gegenüber dem Hauptbahnhof. Es Fi internet access and free regional phone calls. bietet geräumige Zimmer, kostenfreie Wi-Fi-Internetzugang und • The rooms at the Hotel Kronprinz Novum are individually designed. kostenlose Telefonate innerhalb der Region. • In the morning, a rich buffet is served in the breakfast room. From • Die Zimmer im Hotel Kronprinz Novum sind individuell gestaltet. here, you have views of Hamburg's city centre. • Am Morgen, ein reichhaltiges Frühstücksbuffet wird im • The Kronprinz's sister hotel is only a 2-minute walk away and Frühstücksraum serviert. Von hier aus, haben Sie einen Blick auf die provides a spa area with a sauna. On presentation of your room Hamburger Innenstadt. key, you can use these facilities for a reduced price. • Das Kronprinz im Schwesterhotel Hotel ist nur 2 Gehminuten entfernt und bietet einen Wellnessbereich mit Sauna. Auf Vorlage • From the Kronprinz, Hamburg's city centre is easily explored on Ihrer Zimmerschlüssel, können Sie diese Einrichtungen zu einem foot. Alternatively, the public transport services from the reduzierten Preis. Hauptbahnhof (main train station) reach the rest of the city • Der Kronprinz, Hamburg bequem in die Innenstadt zu Fuß erkunden. Alternativ, mit den öffentlichen Verkehrsmitteln vom Hauptbahnhof (Hauptbahnhof) erreichen den Rest der Stadt. • The modern Golden Tulip Suites is situated on the bustling Sheikh Zayed Road, only 5-minutes walk from Mall of the Emirates. This • Die moderne Golden Tulip Suites liegt an der lebhaften Sheikh Zayed aparthotel offers spacious accommodation, great facilities and free Road, nur 5 Gehminuten von der Mall of the Emirates. Dieses parking. Aparthotel bietet geräumige Zimmer, hervorragende Einrichtungen • This 4-star hotel offers a great selection of studios and suite-style und kostenfreie Parkplätze. accommodation, most with kitchen facilities. All of the rooms are • Dieses 4 - Sterne-Hotel bietet eine große Auswahl an Studios und decorated in an elegant, contemporary style and include interactive Suiten-Stil, die meisten mit Küche. Alle Zimmer sind in einem TV systems, game consoles and a laptop for guests’ use. eleganten, modernen Stil eingerichtet und verfügen über ein • Leisure facilities at the hotel include a rooftop swimming pool, interaktives TV-System, Spielkonsolen und einen Laptop für Gäste. Jacuzzi and fitness centre. Guests wishing to eat at the hotel can • Zu den Freizeiteinrichtungen des Hotels gehören ein Swimmingpool enjoy international food 24-hours a day in the all-day dining auf dem Dach, einen Whirlpool und ein Fitnesscenter. Gäste restaurant. There is also a speciality Chinese restaurant. genießen möchten, können Sie die internationale Küche 24 - Stunden am Tag in der All-Tag Speisen. Es gibt auch eine Spezialität Chinese Restaurant. Copyright © 2010, Asia Online Pte Ltd and Moravia IT a.s.
  14. 14. Keys To Success • Collaboration between the MT Vendor, LSP and the Client • High Quality Foundation Data • Comprehensive and Consistent Terminology • Commitment to an Evolutionary Process • Quality and Skill of Linguistic Steering • Frequency and Volume of Corrective Feedback • Effective On-going Quality Measurement and Assessment Copyright © 2010, Asia Online Pte Ltd and Moravia IT a.s.
  15. 15. Contact Information Machine Translation in the Imperfect World Practical Case Study of Using MT in Situations with Imperfect or Insufficient Language Data Kirti Vashee Rustin Gibbs VP Enterprise Translation Sales Solution Architect Asia Online Moravia Worldwide 9 Shenton Way, #06-01 199 East Thousand Oaks Blvd., Suite 203 Singapore 068813 Thousand Oaks, CA 91360, USA Tel: +1-424- 229-2714 Tel: +1-206-200-6107 kirti.vashee@asiaonline.net RustinG@moraviaworldwide.com Copyright © 2010, Asia Online Pte Ltd and Moravia IT a.s.

×