Machine Translation vs. Common Language: !Effects on Idea Exchange in Cross-Lingual Groups Hao$Chuan)Wang       )Na#onalTs...
Globalizing Idea ExchangeIdeas)distributed)across)countries)and)cultures)   •  Ex.)Different)language)edi?ons)of)Wikipedia)...
Ways for Cross-Lingual CommunicationCommon)solu?ons:))   Common)language)(e.g.,)English))or)   human)transla?on.))•  Commo...
Problems in Machine Translation (MT)Concern)for)transla?on)quality)                             English•  “computerbug”may...
Can MT Still Be Useful?MT$based)communica?on)has)both)cost)and)benefit)   •  Cost:(Transla?on)errors)make)message)comprehen...
Experimental Design                       64)individuals)             32)two$person)cross$lingual)groups)Randomly)assign)c...
Experimental DesignRandomly)assign)cross$lingual)groups)to)media?on)condi?on)        MT-mediated                    Englis...
Measurement: ProductivityProduc-vity)        •  Manually)translated)Chinese)contribu?ons)to)English)for)           consist...
Results: Productivity                    14                                       English   MT                    12      ...
Results: Productivity                    14                                       English   MT                            ...
Measurement: ComprehensibilityPerceived(Comprehensibility)    •  Asked)par?cipants)aYer)each)brainstorming)trial))       h...
Results: Comprehensibility                              7   English    MT                        Both)American)and)Chinese...
Discussion                                   14                                                        English   MT       ...
Message versus Idea Production Behaviors                    cultureMT                                   Chinese           ...
#New Ideas                       Message versus Idea Production Behaviors              MT                                 ...
Style of CommunicationMT)appears)to)influence)people’s)style)of)communica?on)   •  Higher)propor?on)of)ideas)in)MT,)both)to...
Design ImplicationsAsymmetrical(design:(MT)benefits)produc?on,)but)harms)comprehension.))))                 “       ”      ...
Or, Interface Design for ComprehensionUse)highligh?ng)to)direct)people’s)aZen?on)to)keywords)in)  transla?ons.) [Gao)et)al...
Acknowledgement     Na?onal)Science)Founda?on)grants)#0845351)and)#1025425)             Undergraduate)assistants)Meaghan)F...
Upcoming SlideShare
Loading in …5
×

Machine translation vs. common language: Effects on idea exchange in cross-lingual groups (CSCW 2013)

656 views
580 views

Published on

CSCW 2013 Paper Presentation

1 Comment
0 Likes
Statistics
Notes
  • Full paper can be found at http://dl.acm.org/citation.cfm?doid=2441776.2441882
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

No Downloads
Views
Total views
656
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
3
Comments
1
Likes
0
Embeds 0
No embeds

No notes for slide

Machine translation vs. common language: Effects on idea exchange in cross-lingual groups (CSCW 2013)

  1. 1. Machine Translation vs. Common Language: !Effects on Idea Exchange in Cross-Lingual Groups Hao$Chuan)Wang )Na#onalTsingHuaUniversity,Taiwan Susan)R.)Fussell )CornellUniversity,NY,USA Dan)Cosley ) )CornellUniversity,NY,USA ) ACMCSCW2013.SanAntonio,TX,USA ! !
  2. 2. Globalizing Idea ExchangeIdeas)distributed)across)countries)and)cultures) •  Ex.)Different)language)edi?ons)of)Wikipedia)varying)in) content)of)the)same)topics.) [Hecht)et)al.,)2011] •  Valuable)source)for)intelligence)and)problem)solving.) [Wang)et)al.,)2011]CMC:)Opportunity)for)global)idea)exchange) •  Need)to)cross)the)language)barrier)first.) ! Wang, Fussell, Cosley ! 2)
  3. 3. Ways for Cross-Lingual CommunicationCommon)solu?ons:)) Common)language)(e.g.,)English))or) human)transla?on.))•  Common)language:)Second)language)to) some)individuals.)Learning)required.)•  Human)transla?on:)Expensive)and) inflexible.)Machine(transla-on((MT)(as)a)non$expensive,)flexible)tool)for)cross$lingual)idea)exchange.() ! Wang, Fussell, Cosley ! 3)
  4. 4. Problems in Machine Translation (MT)Concern)for)transla?on)quality) English•  “computerbug”may)become)“computer insect.”•  Transla?ons)of)complex)ideas)like) “needtoconfirmwithhertoseeif theresenough#metomakeit”can)be) unpredictable.) Chinese))People)oYen)communicate)beZer)over)a)common,)second)language)than)MT.) [Yamashita)&)Ishida,)2006;)Yamashita)et)al.,)2009] ! Wang, Fussell, Cosley ! 4)
  5. 5. Can MT Still Be Useful?MT$based)communica?on)has)both)cost)and)benefit) •  Cost:(Transla?on)errors)make)message)comprehension) and)repairing)difficult.( •  Benefit:()Producing)messages)in)na?ve)language)is)easier) than)non$na?ve)language.))Current)study:)American$Chinese)idea)exchange)over)MT)or) English)as)common)language)) •  H1:Chinese)produce)more)ideas)over)MT)than)English)) (MT(benefit(>(MT(cost)( •  H2:Americans)produce)less)ideas)over)MT)than)English) (MT(has(no(benefit(but(only(extra(cost)( ! Wang, Fussell, Cosley ! 5)
  6. 6. Experimental Design 64)individuals) 32)two$person)cross$lingual)groups)Randomly)assign)cross$lingual)groups)to)media?on)condi?on) MT-mediated English-mediated American-Chinese Dyad American-Chinese Dyad ! Wang, Fussell, Cosley ! 6)
  7. 7. Experimental DesignRandomly)assign)cross$lingual)groups)to)media?on)condi?on) MT-mediated English-mediated American-Chinese Dyad American-Chinese Dyad Brainstorm)two)trials)(15)min)each))) in)counterbalanced)order) “Benefits and costs “Benefits and costs for having an extra for having an extra thumb” eye” Measure) Produc?vity)and)Comprehensibility) ! Wang, Fussell, Cosley ! 8)
  8. 8. Measurement: ProductivityProduc-vity) •  Manually)translated)Chinese)contribu?ons)to)English)for) consistency)of)coding •  Count)numberoforiginalideasby)manually)coding) brainstorming)conversa?ons) Idea? Original?( How
about
sleeping,
with
one
eye
contac5ng
with
 A pillow? idea 1‐some
originality New
opportuni5es
for
business:
glasses
for
the
3
 B eyed


person. idea 1‐some
originality I
think
sleeping
on
your
side
would
become
the
 B norm. idea 0‐duplicate A Are
we
s5ll
talking
about
the
advan? no_idea B New
hats
would
need
to
be
made. idea 1‐some
originality B sure. no_idea A ok no_idea B haircuts
would
certainly
be
modified. idea 1‐some
originality ! Wang, Fussell, Cosley ! 9)
  9. 9. Results: Productivity 14 English MT 12 In)English)groups,)Americans) (na?ve)speaker))produced) 10 more)ideas)than)Chinese)Idea Productivity 8 (non$na?ve)speaker).) 6 4 2 0 American Chinese ! Wang, Fussell, Cosley ! 10)
  10. 10. Results: Productivity 14 English MT Chinese)produce)more)ideas) 12 using)MT)than)English.)) p<.05)for)the)second)trial;)) 10 p=.1)overall.)Idea Productivity 8 )(H1:MT(benefit(>(MT(cost)) 6 ) MT)did)not)affect)Americans’) 4 produc?vity.)) 2 )(H2notsupported:no)MT) 0 cost)on)produc?vity)) American Chinese ) ) ! Wang, Fussell, Cosley ! 11)
  11. 11. Measurement: ComprehensibilityPerceived(Comprehensibility) •  Asked)par?cipants)aYer)each)brainstorming)trial)) how)clear)were)the)messages)to)them.) •  “I)could)understand)other)members’)ideas.”) “I)am)confident)that)other)group)members)understand) my)ideas.”)) “I)didn’t)understand)what)other)group)members)said.”) (alpha=.89))( ! Wang, Fussell, Cosley ! 12)
  12. 12. Results: Comprehensibility 7 English MT Both)American)and)Chinese) par?cipants)rated))Perceived Comprehensibility 6 MT)less)comprehensible) 5 than)English.) 4 ) 3 MT)harms)comprehensibility) both)to)na?ve)and)non$ 2 na?ve)speakers.) 1 ) 0 Consistent)with)previous) American Chinese studies.) ! Wang, Fussell, Cosley ! 13)
  13. 13. Discussion 14 English MT 12 MT)benefits)idea)produc?on)by) 10 enabling)na?ve)language)use,)Idea Productivity 8 though)MT)harms) 6 comprehensibility.) 4 ) 2 ) 0 American Chinese How)does)MT)benefit)Chinese) 7 English MT par?cipants’)produc?on?) Perceived Comprehensibility 6 5 •  Was)it)because)Chinese)talking) 4 more?)) 3 •  Or)was)it)because)Chinese)being) 2 more)efficient)in)producing) 1 ideas?) 0 American Chinese ! Wang, Fussell, Cosley ! 14)
  14. 14. Message versus Idea Production Behaviors cultureMT Chinese Compare)number(of(new(ideas(English 12 and)number(of(chat(turns:) 10 ) #New Ideas 8 6 MT:)~40%)chat)turns)contain) 4 ideas) 2 English:)~33%)turns)contain) 0 ideas) 35 30 ) #Chat Turns Chinese)produce)ideas)more) 25 20 15 efficiently)over)MT)than)English.) ) 10 5 0 0 180 360 540 720 900 Time (seconds) #Words ! Wang, Fussell, Cosley ! 15)
  15. 15. #New Ideas Message versus Idea Production Behaviors MT culture English American Chinese Exploratory)analysis)of) 12 American’s)produc?on) 10 behaviors:)#Chat Turns#New Ideas 8 ) 6 4 MT:)~44%)chat)turns)contain) 2 ideas) 0 English:)~35%)turns)contain) 35 ideas) 30#Chat Turns 25 #Words 20 Americans)talked)less)over) 15 10 MT)than)English.) 5 0 0 180 360 540 720 900 0 180 360 540 720 900 Time (seconds) !#Words Wang, Fussell, Cosley ! 16)
  16. 16. Style of CommunicationMT)appears)to)influence)people’s)style)of)communica?on) •  Higher)propor?on)of)ideas)in)MT,)both)to)Chinese)and) Americans.) •  Chinese)produced)more)ideas;) Americans)produced)less)turns.))Incoming)messages)s?ll)have)impact.))Future)work:)People’s)prac?ces)and)strategies)for)dealing) with)poor)transla?on)in)different)language) communi?es.) ! Wang, Fussell, Cosley ! 17)
  17. 17. Design ImplicationsAsymmetrical(design:(MT)benefits)produc?on,)but)harms)comprehension.)))) “ ” “How are you” translated)) “I’m fine” “I’m fine” untranslated))For)bilinguals,)combine)MT)and)second)language)ability:)•  Translate)outgoing)messages)to)enable)produc?on)in)na?ve) language.)•  Leave)incoming)messages)untranslated.)People)use)second) language)proficiency)for)understanding.) ! Wang, Fussell, Cosley ! 18))
  18. 18. Or, Interface Design for ComprehensionUse)highligh?ng)to)direct)people’s)aZen?on)to)keywords)in) transla?ons.) [Gao)et)al.,)CHI2013])) ! Wang, Fussell, Cosley ! 19)
  19. 19. Acknowledgement Na?onal)Science)Founda?on)grants)#0845351)and)#1025425) Undergraduate)assistants)Meaghan)Frank,)) Johna)Seo)and)Monica)Chen)at)Cornell)University)Anonymous)CSCW)reviewers)for)their)valuable)comments)and)sugges?ons) ! Wang, Fussell, Cosley !

×