Report

Share

•2 likes•470 views

•2 likes•470 views

Report

Share

Download to read offline

=== Invited talk at the International Conference on Operations Research, September 2017, Berlin === We present the optimization of the new French standard for keyboard layouts. The typical French keyboard layout made it overly complicated or even impossible to type all French characters, punctuation marks, and other symbols. The goal of this new standard was to enable and facilitate the correct spelling of French, and allow the entry of symbols common in programming languages, mathematical expressions, and other European languages. Therefore, we had to assign over 115 characters to 148 slots on the keyboard, optimizing their placement with respect to each other and to the numbers (0-9) and non-accentuated letters (a-z) that were kept unchanged. In interaction with an expert committee, we identified four objectives: input performance and ergonomics, similarity to the prior keyboard, and coherent placement of similar symbols. We collected extensive performance data, language statistics, expert ratings on character similarities, and ergonomic scores to implement and solve a multi-objective quadratic assignment problem. In an iterative process, we computed and adapted a range of solutions in interaction with language experts. The resulting design unifies mathematical bounds with expert opinions and constraints. The new French keyboard layout is the first modern standard where computational optimization methods were used in interaction with domain experts to implement an optimal keyboard design.

- 1. Optimizing Special Character Entry TheCase ofthe FrenchKeyboard Standard Anna Feit, Aalto University, Helsinki Joint work with Mathieu Nancel, Maximilian John, Andreas Karrenbauer, Daryl Weir, and Antti Oulasvirta
- 2. Anna Feit annafeit.de userinterfaces.aalto.fi Fun facts about the French keyboard • There is no way to type œuf, Été or FRANÇAIS (egg, Summer, FRENCH) • où (where) is the only word that contains ù, yet it has it’s own key • No french quotation marks « » • Typing . requires Shift, so do all numbers, and try finding the @ key… ” It is almost impossible to write correctly French with a keyboard marketed in France” French Ministry of Culture and Communications 08.09.2017 2
- 3. Anna Feit annafeit.de userinterfaces.aalto.fi Design Task 08.09.2017 3 Assign 129 keyslots to 122 special characters Normal Shift Alt Gr Alt Gr + Shift ! ? " # $ % & ' ( ) [ ] { } * + , - . / : ; < = > ≤ ≥ ÷ @ | ¿ ¡ ¢ £ ¤ ¥ § © ª « ® ° ± · º » ¼ ½ ¾ × ù Ù à æ Æ ç Ç è é ê œ Œ þ İ ı Ĳ ĳ Ŋ ŋ ſ Ə Ʒ ə ʒ μ ð Ð Þ ß ẞ _ - – — ‘ ’ ‚ “ ” „ † ‡ … ‰ ‹ › ₩ ™ √ ∞ ≃ · ´ ˋ ˆ ¨ ˉ ̲ ˘ ̑ ˇ ˜ ˙ ̣ ̊ ˝ ˵ ¸ ˛ ̦ ̵ ̷ > 𝟏𝟎 𝟐𝟏𝟑 𝒂𝒔𝒔𝒊𝒈𝒏𝒎𝒆𝒏𝒕𝒔
- 4. Anna Feit annafeit.de userinterfaces.aalto.fi Given: n characters – 𝑖, 𝑗 𝜖 Σ m keyslots – 𝑘, 𝑙 𝜖 𝑆 Let: 𝑥𝑖𝑘 = 1 if character 𝑖 is assigned to keyslot 𝑘, 𝑥𝑖𝑘 = 0 otherwise The (letter) assignment problem [Burkhard, 1977] 08.09.2017 4
- 5. Anna Feit annafeit.de userinterfaces.aalto.fi Given: n characters – 𝑖, 𝑗 𝜖 Σ m keyslots – 𝑘, 𝑙 𝜖 𝑆 Let: 𝑥𝑖𝑘 = 1 if character 𝑖 is assigned to keyslot 𝑘, 𝑥𝑖𝑘 = 0 otherwise Goal: Find the assignment of characters to keyslots that minimizes the cost of typing one character after another The (letter) assignment problem [Burkhard, 1977] 08.09.2017 5
- 6. Anna Feit annafeit.de userinterfaces.aalto.fi Given: n characters – 𝑖, 𝑗 𝜖 Σ m keyslots – 𝑘, 𝑙 𝜖 𝑆 Let: 𝑥𝑖𝑘 = 1 if character 𝑖 is assigned to keyslot 𝑘, 𝑥𝑖𝑘 = 0 otherwise Goal: Find the assignment of characters to keyslots that minimizes the cost of typing one character after another [Burkhard, 1977] The (letter) assignment problem Assign each symbol to exactly one input action 08.09.2017 6
- 7. Anna Feit annafeit.de userinterfaces.aalto.fi Given: n characters – 𝑖, 𝑗 𝜖 Σ m keyslots – 𝑘, 𝑙 𝜖 𝑆 Let: 𝑥𝑖𝑘 = 1 if character 𝑖 is assigned to keyslot 𝑘, 𝑥𝑖𝑘 = 0 otherwise Goal: Find the assignment of characters to keyslots that minimizes the cost of typing one character after another [Burkhard, 1977] The (letter) assignment problem Assign each character to exactly one keyslot Assign each keyslot not more than once 08.09.2017 7
- 8. Anna Feit annafeit.de userinterfaces.aalto.fi Optimization of Text Entry methods 08.09.2017 8 [Zhai, Hunter, Smith, 2000] [Karrenbauer and Oulasvirta, 2014] [Feit and Oulasvirta, 2014] [Sridhar, Feit, Theobalt and Oulasvirta, 2015] …
- 9. Anna Feit annafeit.de userinterfaces.aalto.fi Special character entry Quadratic assignment problem is NP hard, with 129 keyslots, 122 characters too complex to even approximate. Observation: • Special characters are infrequent in comparison to letters • Mostly entered before or after a letter We can simplify the quadratic problem to a linear assignment problem plus a sparse quadratic part 08.09.2017 9
- 10. Anna Feit annafeit.de userinterfaces.aalto.fi Given: n characters – 𝑖, 𝑗 𝜖 Σ m keyslots – 𝑘, 𝑙 𝜖 𝑆 Let: 𝑥𝑖𝑘 = 1 if character 𝑖 is assigned to keyslot 𝑘, 𝑥𝑖𝑘 = 0 otherwise Goal: Find the assignment of characters to keyslots that minimizes the cost of typing one character after another or before / after a letter Assignment problem 08.09.2017 10
- 11. Anna Feit annafeit.de userinterfaces.aalto.fi Given: n characters – 𝑖, 𝑗 𝜖 Σ m keyslots – 𝑘, 𝑙 𝜖 𝑆 Let: 𝑥𝑖𝑘 = 1 if character 𝑖 is assigned to keyslot 𝑘, 𝑥𝑖𝑘 = 0 otherwise Goal: Find the assignment of characters to keyslots that minimizes the cost of typing one character after another or before / after a letter Assignment problem 08.09.2017 11 sparse
- 12. Anna Feit annafeit.de userinterfaces.aalto.fi Objectives 08.09.2017 12 Standardization committee: ” The new keyboard should • facilitate typing of correct french • allow quick and easy access to all French characters • be easy to learn and intuitive to use ”
- 13. Anna Feit annafeit.de userinterfaces.aalto.fi Motor Performance: minimize time to move between frequent character-letter pairs Objectives Note: linear function 08.09.2017 13
- 14. Anna Feit annafeit.de userinterfaces.aalto.fi Objectives Note: linear function Ergonomics: minimize frequent extreme movements of wrist and fingers when entering a special character in combination with a letter Motor Performance: minimize time to move between frequent character-letter pairs Note: linear function 08.09.2017 14
- 15. Anna Feit annafeit.de userinterfaces.aalto.fi Learnability (Intuitiveness): minimize the distance between similar characters to ensure a consistent and intuitive layout that is easy to use and learn. Objectives 08.09.2017 15
- 16. Anna Feit annafeit.de userinterfaces.aalto.fi Learnability (Intuitiveness): minimize the distance between similar characters to ensure a consistent and intuitive layout that is easy to use and learn. Objectives Learnability (Familiarity): minimize the distance of a character to its position on the standard keyboard layout. Note: linear function 08.09.2017 16
- 18. Anna Feit annafeit.de userinterfaces.aalto.fi Assign each letter to exactly one key Optimization Problem 08.09.2017 18
- 19. Anna Feit annafeit.de userinterfaces.aalto.fi Assign each letter to exactly one key Assign each key to at most one letter Optimization Problem 08.09.2017 19
- 20. Anna Feit annafeit.de userinterfaces.aalto.fi Assign each character to exactly one key Assign each key to at most one character Capital character must be on shifted key of lower case character Optimization Problem 08.09.2017 20
- 21. Anna Feit annafeit.de userinterfaces.aalto.fi Letter and letter pair frequency Weighted combination of different corpora representing diverse tasks • Formal corpus: legal texts, newspaper articles, wikipedia, emails, facebook 10 different corpora averaged, largest coverage of symbols • Twitter corpus: 10000 tweets from 10 different twitter accounts, 6 "classic" (e.g. newspaper) and 4 "popular" accounts • Code corpus: 50 projects from github of most common programming languages (Java, C++, python, CSS, javascript, HTML) Input data 08.09.2017 21
- 22. Anna Feit annafeit.de userinterfaces.aalto.fi Formal: Twitter: Code: Input data 08.09.2017 22
- 23. Anna Feit annafeit.de userinterfaces.aalto.fi Input data Performance crowdsourced movement times of 7555 different key pairs embedded in nonsense words and repeated 10 times. 08.09.2017 23
- 24. Anna Feit annafeit.de userinterfaces.aalto.fi Input data Ergonomics: heuristic cost based on literature, equally punishing movement toward outer right and left, wrist and finger extension, use of modifiers (Alt, Shift) 08.09.2017 24
- 25. Anna Feit annafeit.de userinterfaces.aalto.fi Input data Similarity each letter-character and character-character pair rated with a heuristic score, maximum of multiple similarity criteria such as: • visual similarity, e.g. à and a, • semantic or functional similarity, e.g. $ and £ • phonetic similarity, e.g. s and ß • etc. Sparse input matrix 08.09.2017 25
- 26. Anna Feit annafeit.de userinterfaces.aalto.fi Input data Distance Manhattan distance quantifies number of rows and columns between two keyslots, additional cost for difference in modifier keys Similar to search behavior of users (row and column wise) 08.09.2017 26
- 27. Anna Feit annafeit.de userinterfaces.aalto.fi Optimization in the Iterative Design Process Problem and design space Objectives and constraints Input data Optimization Process Expert Comittee Optimal layouts 08.09.2017 27
- 29. Anna Feit annafeit.de userinterfaces.aalto.fi One solution French characters easily and quickly accessible 08.09.2017 29
- 30. Anna Feit annafeit.de userinterfaces.aalto.fi One solution Easy to learn: familiar position and groups of similar characters 08.09.2017 30
- 31. Anna Feit annafeit.de userinterfaces.aalto.fi Comparison with Azerty Our mapping AZERTY Difference of AZERTY to our mapping (smaller is worse) 08.09.2017 31
- 32. Anna Feit annafeit.de userinterfaces.aalto.fi Comparison with manual design Our mapping Manually created mapping Difference of manual mapping to our mapping (smaller is worse) 08.09.2017 32
- 33. Anna Feit annafeit.de userinterfaces.aalto.fi Optimization in the Iterative Design Process Problem and design space Objectives and constraints Input data Optimization Process Expert Comittee Optimal layouts Manual changes evaluated by objective functions 08.09.2017 33
- 34. Anna Feit annafeit.de userinterfaces.aalto.fi Current status Public feedback Nearly 3000 comments from French people (47% positive, 37% negative, 16% neutral) Revision of optimization model and layout to integrate feedback 08.09.2017 34
- 35. Anna Feit annafeit.de userinterfaces.aalto.fi Summary 08.09.2017 35 • Large real world instance of the (sparse) quadratic assignment problem • Relevant for millions of users and applicable to other languages • Problem models objectives for performance, ergonomics, and learnability of typing special characters • Optimization as part of an iterative design process