Crowdsoucing Based Mobile Image Translation

A Crowdsourcing Based Mobile Image Translation and Knowledge Sharing ServiceYefeng Liu, Vili Lehdonvirta1, Mieke Kleppe2, Todorka Alexandrova, Hiroaki Kimura, Tatsuo Nakajima Department of Computer Science Waseda University, Tokyo, Japan 1Helsinki Institute for Information Technology 2Eindhoven University of Technology yefeng@dcl.info.waseda.ac.jp

Outline• Introduction• Human Mobile Image Translation• Preliminary Study• Discussion• Future Directions A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 2

Introduction“...I can’t wear tie here?? Should Itake off my tie?..” A menu board outside a restaurant, Tokyo A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 3

Real World Problem• Digital pocket translators or online translation services are useless if you donʼt know how to input the characters. A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 4

(Typical) Mobile Image TranslationImage OCR MT EnglishText Optical Character Recognition Machine Translation Text Poor Irregular fonts or formats, handwriting, etc. performance A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 5

Our Solution: Human Mobile Image TranslationImage Text Translator English TextQuestion of Outsourcing Communitythe image Crowdsourcing • Better quality in text recognition and translation • Human worker can provide richer interpretations and responses in addition to literal answers. A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 6

Image Based Translator + Mobile Q&A • NOT only a translator • But also a knowledge broker that allows users to share high level information pertinent to the situation at hand, e.g. Q: “what’s the • advice difference between 1 and 2 in my electricity • explanations bill?” • instructions A: “1is basic charge, • suggestions 2 is additional fee ” A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 7

Basic work-ﬂow overview English Kanji Open call Scoring etc.Requester Best Translators Requester answer A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 8

Comparison Expert/Social Translation Mobility Quality Search(OCR based)Mobile Image Yes No Yes So-so TranslationHuman-based Search Yes & No Yes No Good Proposed Solution Yes Yes Yes Good A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 9

Preliminary study• A preliminary study and design research aims to • verify the feasibility of the design • identify real user requirements and design issuesA Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 10

Preliminary study - Method Collected pictures/questions from potential usersFifteen characteristic cases were selected from the collected imagesInterviewed the requesters what kind of answers they were expecting Assigned questions to invited translators Interviewed translators for their feedbacks Compared the results with the requesters’ expectations A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 11

Preliminary Study Cases - Example “...how long do I have to wait?”Information in the picture is insufﬁcient for collecting their feedback. we interviewed the translatorto answer this question.However, most of the repliers can stillsuggest an approximate waiting timeaccording to their life experiences. A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 12

Preliminary Study Cases - Example (2) “What are the events between 5th and 8th?” we interviewed the translator for collecting their feedback. Poor question text. Some translators misunderstood the question, thus provided useless answers. A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 13

Preliminary Study - Implication1. Communication between requester and worker. Better communication Better understanding Better result we interviewed the translator for collecting their feedback.2. Question/Answer style • Short, but clear (e.g clarify to what level of details is wanted); • Question with choices is better; • Asking for links (of image/web page/etc) is a good way to lower the difﬁculty and faster the response time. A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 14

Discussion (1)1. Quality of outcomeMisunderstanding between requester andworker strongly affects quality of outcome. - Workers often are not native English speakers. - Requesters may use unclear or too complicated English. - People always make mistakes. - Malicious replies.A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 15

Discussion (2) Kanji English Scoring open call BestRequester Translators Proofreaders Requester answer An additional proofreading phase. A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 16

Discussion (3)2. Different user types (user requirements) Client Users Short-term stay Long-term stayNeed immediate Waitable Need immediate Waitable answer answer A B C Dmay have different preference on the accuracy vs. timeliness trade-off A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 17

Future Directions (1)1. Dynamical task allocation with real time requirement • Task is better be assigned to worker who is: i. capable for the task ii. available for the task Not only about if the worker is free, but also involves other factors like expertise, properties of question, etc. A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 18

Future Directions (2)2. Motivation and Incentive Social and Intrinsic incentive: game play A location-based mobile game is designed A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 19

Conclusion• Conclusion • Human-based mobile image translation system • Preliminary study • Findings and future directions• Current Status Preliminary Prototype Early Usability/OnDesign Redesign Redesign Study Implementation Test ﬁeld study A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 20

Thank you for your attention! Yefeng Liu, PhD candidate yefeng@dcl.info.waseda.ac.jp Distributed & Ubiquitous Computing Lab. Depart. of Computer Science, Waseda University http://www.dcl.info.waseda.ac.jp/

“keywords” style answer is preferred a). “Pork, spicy, famous chinese food” we interviewed the translator for collecting their feedback. b). “Twice cooked pork (huiguo rou)” - meaningless if don’t know the name - Many translators use English as 2nd or 3rd language, they often face the problem of being unable to explain in long sentence. A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 22

we interviewed the translator for collecting their feedback. “what’re these two? can you provide links of“is it a show or training course?” pics” A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 23

we interviewed the translator for collecting their feedback.I wanna buy the ticket for swim! what divination result I got here? A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 24

Crowdsoucing Based Mobile Image Translation

by Yefeng Liu on Dec 14, 2010

Accessibility

Upload Details

Usage Rights

Statistics

Crowdsoucing Based Mobile Image Translation Presentation Transcript