The document evaluates various mobile Chinese text input interfaces, primarily comparing Sony Ericsson with competitors like Nokia and Samsung. It finds that Nokia offers the best usability and speed, while Sony Ericsson lags behind due to system lag and poor design choices. The report concludes with design recommendations aimed at improving text input efficiency and user satisfaction across mobile platforms.

1. OSE Chinese Text Input Benchmark Test Report Chen, John Sony Ericsson Experience Lab BJ 2008.Mar.-Apr.

2. Introduction Current T9 has broadly used for mobile text input, but there’re different interfaces on different mobile phones, which results in significantly different user experience. So it’s important to assess the impact of different text entry interfaces. Local mobile Chinese text entry research & development makes a significant progress: ZTA4 is widely downloaded and installed in Symbian S60 and windows mobile phones, which is becoming more and more popular.

3. Purpose To benchmark the Chinese text entry speed, accuracy (on 12 key pad phone) and user’s subjective satisfaction between Sony Ericsson (OSE) and main competitors’ mobile phones. To identify what are good designs and what are bad designs regarding Chinese text input on keypad phone. And to provide data support to push changes of Chinese text input on Central platform.

4. Executive Summary Nokia had the best usability because of the highly consistent UI, simple design and considerate association words, phrases and even symbols. A4 show ed the fastest texting speed, but a little lower on usability than Nokia due to the slight inconsistency and the way of choosing single candidate Pinyin. Sony Ericsson had both the slowest texting speed and also the worst usability due to the obvious system lag, lack of new features, and hidden functionalities for long press operations (switch input mode and choose candidates related number). Samsung had the relative fast texting speed, but lower usability ratings due to the hidden functionality on “1” key, and hidden long press on asterisk key to input symbols. Chinese users has very limited experience in Latin input. Almost all the test users got confused while entering uppercase “MSN” in SonyEricsson, Samsung and Motorola, due to the design of separated key to switch Latin input mode.

5. Content Methodology Results: Entry Speed Analysis Results: SUS Analysis Design Recommendations Recap

6. Methodology

7. Research Design Hardware keypad performance benchmark The results as a controlled factor 2 (user experience level) x 5 (text entry interfaces) x 4 (text type) design User experience level as between subject factor Text entry interfaces & text type as within subject factors E D C B p9,p10 E D C A p7,p8 E D B A p,5,p6 E C B A P3,p4 D C B A p1,p2 Test phones (the order of phone presented counterbalanced) Test participants (10 normal Pinyin users) E D C B p19,p20 E D C A p17,p18 E D B A p15,p16 E C B A p13,p14 D C B A p11,p12 Test phones Test participants (9 experienced Pinyin users)

8. Apparatus Test phones ZTA4 (Nokia 3230 S60 with ZTA4) Nokia 2610 S40 Motorola E398 (iTap) Samsung J218 Sony Ericsson K530 (OSE) Video camera / stopwatch Stopwatch used as primary timer Video recording used as backup timing ZTA4 Nokia Motorola Samsung SonyEricsson

9. Users Profile 19 participants in all, aged from 20-30, 8 females and 11 males, mixed 10 Normal Pinyin users, SMS usage: <10 SMS/day Text entry speed on own phone is 12 wpm (words per minute) on average, varies between 7wpm to 16wpm 9 Experienced Pinyin users, SMS usage: >15 SMS/day Text entry speed on own phone is 20 wpm on average, varies between 17wpm to 26wpm, average is 20 wpm

10. Measurement Chinese Pinyin Text entry speed Keystroke level Character level e.g. Ni xian zai mang ma (15 characters, 21 keystrokes in OSE phone K660) 你 现 在 忙 吗 (5 words) Note: 405 Pinyin strings are used for a total of 7513 common Chinese characters, the average Pinyin length is 3.21, and the average Pinyin length weighted by number of characters having the same pronunciation is 3.07 (US Patent No. US006822585B1) SUS (System Usability Scale) 10 items of 5 point Likert Scale (e.g. “I thought the input method was easy to use”) Qualitative data collection (observations & user’s comments etc.) Interview after test Affinity diagram to highlight user’s behavior and needs

11. Test tasks Text copy task Entry speed measurement Have user enter the same sentence in their own phone, and record time “ 王总，产品报价发到您的邮箱了，请查收，如有任何疑问，请随时联系我。” Practice task (have users get familiar with the key mapping & keypad feeling etc) Have user enter 2 messages which he/she actually sent in his daily life (keep the 2 messages the same for each phone practice task) Test task (have users complete the task as quick & accurate as possible) Chinese content: (from SMS investigation*) 先到附近的嘉禾一品或者肯德基吃点东西 (Only Chinese text) 我在单位呢 现在上不了 MSN (Mixed Chinese & letters) 晚上去哪儿 happy ？ (Mixed Chinese, English& symbols) 请回电话 58659394 (Mixed Chinese & numbers) * An online SMS content investigation was conducted before the test to extract typical SMS expressions/sentences for text copy task.

12. Results: Entry Speed analysis

13. Entry Speed - time spent on sentences ZTA4 has the fastest entry speed on all sentences SE spent longer time than competitors on all sentences

14. Entry Speed - time spent on Chinese only sentence Sony Ericsson Chinese entry speed was significantly lower than other competitors. ZTA4 was almost as fast as user’s entry speed on their own phones even it was new for them. “ 先到附近的嘉禾一品或者肯德基吃点东西”

15. Why? - Observations and comments Why Sony Ericsson was the slowest one? Lag of showing Pinyin (system response was slow). System lag made a lot of mistakes Unintuitive scrolling of candidates Pinyin candidates were hidden “ C” key and “Back” key confusion made mistakes Why ZTA4 was the fastest? Easy to get started in spite of previous texting experience due to the good compatibility Phrase input, even for non-dictionary phrase entry Fast word frequency adjustment Word frequency adjustment effect (due to NOT reset the phone after each single test session)

16. Entry speed - time spent on Chinese +”MSN” Sony Ericsson was significantly slower than other competitor phones. ZTA4 was fastest due to the uppercase and lowercase was in the input mode switch loop, which was quite clear for Chinese users. “ 我在单位呢 现在上不了 MSN”

17. Entry Speed - time spent on Chinese + “happy?” Sony Ericsson was significantly slower than other competitor phones. ZTA4 was the fastest, partly because after entering “happy”, the symbols bar appeared automatically. “ 晚上去哪儿 happy ？”

18. Why? - Observations and comments Most Chinese users got confused and very frustrated to enter uppercase “MSN” in some models. According to the SMS online survey, most Chinese had very limited experience with Latin input. SonyEricsson and Samsung has separated switch key for Latin input (Lowercase, uppercase and initial letter capitalized), while ZTA4 and Nokia has lowercase and uppercase in the input mode switch loop which was quite clear.

19. Entry Speed - time spent on “Chinese + 58659394” Sony Ericsson and Motorola was slower than the other 3 phones significantly. 请回电话 58659394

20. Why? - Observations and comments Phrase input, association words and the way of switch were the main influence factors. ZTA4, Nokia and Samsung had phrase input, good association, which made this simple Chinese sentence input very quick. SonyEricsson and Motorola lack these new features to facilitate texting speed. And SonyEricsson needed long press to switch input mode, which made it slower than Motorola.

21. Results: SUS analysis

22. SUS Scores Comparison Both ZTA4 and Nokia were rated as good usability. Normally SUS* scores above 60 shows good usability. No significant difference between experienced Pinyin users and normal Pinyin users. Although experienced users rated higher on Samsung than normal users. * SUS (System Usability Scale) is a ten item attitude scale, developed by John Brooke in 1996. Here it was used to measure the usability of several user interfaces of Chinese text input.

23. Why? - Observations and comments Why Nokia and ZTA4 had higher usability? Nokia had the most consistent operation. “ Anytime I can use asterisk key to input symbols and use pound key to switch input mode, it’s easy ” A4 was very easy to get started for most test users, in spite of his/her previous texting experience. e.g. allow users to choose candidates with 5-way Nav. key, numerical keypad, and also using number &quot;1&quot; to indicate number index etc (like Samsung).. “ I can use it in any way what I want. It’s just what I want.”

24. Why? - Observations and comments Why SonyEricsson got the worst usability? Obvious System lag “ I want to type quickly, but the system is too slow to support quick typing. I need to wait. ” Hidden functions due to long press Long press to switch between Chinese and Latin Long press to chose candidates related number

25. Perceived keypad Performance Impact There’s no statistical significance among the test phone’s perceived hardware performance impact on text entry speed. Although ZTA4 keypad was perceived to have slightly bigger impact on text entry speed, but it still showed the fastest texting speed. - SUS item 11 analysis “ Keypad handfeel , screen size etc have big impact on my text entry speed.”

26. Design Recommendations

27. How these recommendations comes - Affinity Diagram

28. 6 Most Important Consistence : Keep consistence of single key’s functionality and keep consistent with user’s former experience as well Features : Advanced features facilitate text speed Flexible : Allow users to make mistake and recover from mistakes easily and costless Simple : Less eye movement and fewer clicks Feedback : Provide clear and in-time feedback Help : Provide tips on-demand

29. Consistence Keep consistence of keys (especially *, #, and “1” key) in different input mode Keep consistent in different situation (especially for “space” key and switch input mode key) When there’re predictions on screen, I can still enter space or switch input mode without back or delete the predictions firstly. Enter symbols easily and keep consistent in different input mode

30. Consistence (cont.) Simplify uppercase and lowercase entry for Chinese Put uppercase and lowercase separately in the switch loop Do not use long press in frequently used functions, e.g. switch input mode, enter symbols, choose number index etc. Provide users more ways to choose candidates, e.g. navigation and number index. Simplify the definition of “C” key and “Back” key in Chinese text input Most users confused about “C” key and “back” key Whenever users need to ‘delete’ or ‘back’, they tend to press “C” Key

31. Features Phrasal input Capability of dictionary phrase input and also non-dictionary phrase input User dictionary: Memorize user’s non-dictionary phrase Fast frequency adjustment Abundant and smart association Not only word association, but also phrase and symbols association like Nokia does which was very appreciated by most test users.

32. Flexible Allow users to make mistakes and recover from mistakes easily and costless. Allow users choose candidate Pinyin and candidates word in a loop Allow users move forward and backward freely using Navigation keys Keep my association words after I back from typing mistakes. Samsung didn’t dismiss the association words which made users feel good

33. Simple Keep Pinyin and candidates bar close on the screen Don’t force user use menu to switch input mode like Samsung does. Fewer clicks to get the intended candidate Allow user to choose number index with short press

34. Feedback Provide in-time visual feedback when typing See exactly what the user is typing, no lag of word and phrase input Error alert for wrong typing is good (Sony Ericsson has it)) Provide clear indicator Clear indicator on the keypad “ # 中” is a clear indicator on the keys No toggling indicator on Sony Ericsson’s keypad Clear indicator of current input mode on the screen Pinyin, 拼音，笔画， 123 ，

35. Help Clear indicator or hint to guide user to get started if it’s not consistent with normal design. E.g. “*” as an indicator for choosing candidate Pinyin is perceived good. Samsung has no indicator for user to choose candidates by pressing “1” key, which is quite hidden for novice users. SonyEricsson didn’t has any indicator for users to long press related number to choose the corresponding candidate. (similar with toggling input mode on hash key) More advanced features could be introduced in help information. No user actually used long press to enter number in Chinese text input mode.

36. Recap

37. Why Chinese text input is so important? Statistical Significance between ZTA4, Nokia and SonyEricsson. Users are willing to send more messages via ZTA4 and Nokia phones, and significant less willingness to send SMS via Sony Ericsson. Important to operators to increase revenue. Important to Chinese market to increase market share. “ I think I would like to use this input method to send SMS frequently.” - SUS item 1 analysis

38. The rode to be No.1 in Chinese text input - 3 Most Important: Quick System Response No lag of showing Pinyin Support phrase input very well, either for phrase input or sentence based input Significant UI Consistence in key’s functionalities Consistency with user’s former experience Advanced Features Phrase input, including non-dictionary phrase input Fast frequency adjustment

39. THANKS Please direct all your questions to John, Chen john.chen@sonyericsson.com