Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Optimizing Special Character Entry: the Case of the French Keyboard Standard

17 views

Published on

=== Invited talk at the International Conference on Operations Research, September 2017, Berlin ===

We present the optimization of the new French standard for keyboard layouts. The typical French keyboard layout made it overly complicated or even impossible to type all French characters, punctuation marks, and other symbols. The goal of this new standard was to enable and facilitate the correct spelling of French, and allow the entry of symbols common in programming languages, mathematical expressions, and other European languages.
Therefore, we had to assign over 115 characters to 148 slots on the keyboard, optimizing their placement with respect to each other and to the numbers (0-9) and non-accentuated letters (a-z) that were kept unchanged.
In interaction with an expert committee, we identified four objectives: input performance and ergonomics, similarity to the prior keyboard, and coherent placement of similar symbols.
We collected extensive performance data, language statistics, expert ratings on character similarities, and ergonomic scores to implement and solve a multi-objective quadratic assignment problem. In an iterative process, we computed and adapted a range of solutions in interaction with language experts. The resulting design unifies mathematical bounds with expert opinions and constraints.
The new French keyboard layout is the first modern standard where computational optimization methods were used in interaction with domain experts to implement an optimal keyboard design.

Published in: Science
  • Be the first to comment

  • Be the first to like this

Optimizing Special Character Entry: the Case of the French Keyboard Standard

  1. 1. Optimizing Special Character Entry TheCase ofthe FrenchKeyboard Standard Anna Feit, Aalto University, Helsinki Joint work with Mathieu Nancel, Maximilian John, Andreas Karrenbauer, Daryl Weir, and Antti Oulasvirta
  2. 2. Anna Feit annafeit.de userinterfaces.aalto.fi Fun facts about the French keyboard • There is no way to type œuf, Été or FRANÇAIS (egg, Summer, FRENCH) • où (where) is the only word that contains ù, yet it has it’s own key • No french quotation marks « » • Typing . requires Shift, so do all numbers, and try finding the @ key… ” It is almost impossible to write correctly French with a keyboard marketed in France” French Ministry of Culture and Communications 08.09.2017 2
  3. 3. Anna Feit annafeit.de userinterfaces.aalto.fi Design Task 08.09.2017 3 Assign 129 keyslots to 122 special characters Normal Shift Alt Gr Alt Gr + Shift ! ? " # $ % & ' ( ) [ ] { } * + , - . / : ; < = > ≤ ≥ ÷ @ | ¿ ¡ ¢ £ ¤ ¥ § © ª « ® ° ± · º » ¼ ½ ¾ × ù Ù à æ Æ ç Ç è é ê œ Œ þ İ ı IJ ij Ŋ ŋ ſ Ə Ʒ ə ʒ μ ð Ð Þ ß ẞ _ - – — ‘ ’ ‚ “ ” „ † ‡ … ‰ ‹ › ₩ ™ √ ∞ ≃ · ´ ˋ ˆ ¨ ˉ ̲ ˘ ̑ ˇ ˜ ˙ ̣ ̊ ˝ ˵ ¸ ˛ ̦ ̵ ̷ > 𝟏𝟎 𝟐𝟏𝟑 𝒂𝒔𝒔𝒊𝒈𝒏𝒎𝒆𝒏𝒕𝒔
  4. 4. Anna Feit annafeit.de userinterfaces.aalto.fi Given: n characters – 𝑖, 𝑗 𝜖 Σ m keyslots – 𝑘, 𝑙 𝜖 𝑆 Let: 𝑥𝑖𝑘 = 1 if character 𝑖 is assigned to keyslot 𝑘, 𝑥𝑖𝑘 = 0 otherwise The (letter) assignment problem [Burkhard, 1977] 08.09.2017 4
  5. 5. Anna Feit annafeit.de userinterfaces.aalto.fi Given: n characters – 𝑖, 𝑗 𝜖 Σ m keyslots – 𝑘, 𝑙 𝜖 𝑆 Let: 𝑥𝑖𝑘 = 1 if character 𝑖 is assigned to keyslot 𝑘, 𝑥𝑖𝑘 = 0 otherwise Goal: Find the assignment of characters to keyslots that minimizes the cost of typing one character after another The (letter) assignment problem [Burkhard, 1977] 08.09.2017 5
  6. 6. Anna Feit annafeit.de userinterfaces.aalto.fi Given: n characters – 𝑖, 𝑗 𝜖 Σ m keyslots – 𝑘, 𝑙 𝜖 𝑆 Let: 𝑥𝑖𝑘 = 1 if character 𝑖 is assigned to keyslot 𝑘, 𝑥𝑖𝑘 = 0 otherwise Goal: Find the assignment of characters to keyslots that minimizes the cost of typing one character after another [Burkhard, 1977] The (letter) assignment problem Assign each symbol to exactly one input action 08.09.2017 6
  7. 7. Anna Feit annafeit.de userinterfaces.aalto.fi Given: n characters – 𝑖, 𝑗 𝜖 Σ m keyslots – 𝑘, 𝑙 𝜖 𝑆 Let: 𝑥𝑖𝑘 = 1 if character 𝑖 is assigned to keyslot 𝑘, 𝑥𝑖𝑘 = 0 otherwise Goal: Find the assignment of characters to keyslots that minimizes the cost of typing one character after another [Burkhard, 1977] The (letter) assignment problem Assign each character to exactly one keyslot Assign each keyslot not more than once 08.09.2017 7
  8. 8. Anna Feit annafeit.de userinterfaces.aalto.fi Optimization of Text Entry methods 08.09.2017 8 [Zhai, Hunter, Smith, 2000] [Karrenbauer and Oulasvirta, 2014] [Feit and Oulasvirta, 2014] [Sridhar, Feit, Theobalt and Oulasvirta, 2015] …
  9. 9. Anna Feit annafeit.de userinterfaces.aalto.fi Special character entry Quadratic assignment problem is NP hard, with 129 keyslots, 122 characters too complex to even approximate. Observation: • Special characters are infrequent in comparison to letters • Mostly entered before or after a letter We can simplify the quadratic problem to a linear assignment problem plus a sparse quadratic part 08.09.2017 9
  10. 10. Anna Feit annafeit.de userinterfaces.aalto.fi Given: n characters – 𝑖, 𝑗 𝜖 Σ m keyslots – 𝑘, 𝑙 𝜖 𝑆 Let: 𝑥𝑖𝑘 = 1 if character 𝑖 is assigned to keyslot 𝑘, 𝑥𝑖𝑘 = 0 otherwise Goal: Find the assignment of characters to keyslots that minimizes the cost of typing one character after another or before / after a letter Assignment problem 08.09.2017 10
  11. 11. Anna Feit annafeit.de userinterfaces.aalto.fi Given: n characters – 𝑖, 𝑗 𝜖 Σ m keyslots – 𝑘, 𝑙 𝜖 𝑆 Let: 𝑥𝑖𝑘 = 1 if character 𝑖 is assigned to keyslot 𝑘, 𝑥𝑖𝑘 = 0 otherwise Goal: Find the assignment of characters to keyslots that minimizes the cost of typing one character after another or before / after a letter Assignment problem 08.09.2017 11 sparse
  12. 12. Anna Feit annafeit.de userinterfaces.aalto.fi Objectives 08.09.2017 12 Standardization committee: ” The new keyboard should • facilitate typing of correct french • allow quick and easy access to all French characters • be easy to learn and intuitive to use ”
  13. 13. Anna Feit annafeit.de userinterfaces.aalto.fi Motor Performance: minimize time to move between frequent character-letter pairs Objectives Note: linear function 08.09.2017 13
  14. 14. Anna Feit annafeit.de userinterfaces.aalto.fi Objectives Note: linear function Ergonomics: minimize frequent extreme movements of wrist and fingers when entering a special character in combination with a letter Motor Performance: minimize time to move between frequent character-letter pairs Note: linear function 08.09.2017 14
  15. 15. Anna Feit annafeit.de userinterfaces.aalto.fi Learnability (Intuitiveness): minimize the distance between similar characters to ensure a consistent and intuitive layout that is easy to use and learn. Objectives 08.09.2017 15
  16. 16. Anna Feit annafeit.de userinterfaces.aalto.fi Learnability (Intuitiveness): minimize the distance between similar characters to ensure a consistent and intuitive layout that is easy to use and learn. Objectives Learnability (Familiarity): minimize the distance of a character to its position on the standard keyboard layout. Note: linear function 08.09.2017 16
  17. 17. Anna Feit annafeit.de userinterfaces.aalto.fi Optimization Problem 08.09.2017 17
  18. 18. Anna Feit annafeit.de userinterfaces.aalto.fi Assign each letter to exactly one key Optimization Problem 08.09.2017 18
  19. 19. Anna Feit annafeit.de userinterfaces.aalto.fi Assign each letter to exactly one key Assign each key to at most one letter Optimization Problem 08.09.2017 19
  20. 20. Anna Feit annafeit.de userinterfaces.aalto.fi Assign each character to exactly one key Assign each key to at most one character Capital character must be on shifted key of lower case character Optimization Problem 08.09.2017 20
  21. 21. Anna Feit annafeit.de userinterfaces.aalto.fi Letter and letter pair frequency Weighted combination of different corpora representing diverse tasks • Formal corpus: legal texts, newspaper articles, wikipedia, emails, facebook 10 different corpora averaged, largest coverage of symbols • Twitter corpus: 10000 tweets from 10 different twitter accounts, 6 "classic" (e.g. newspaper) and 4 "popular" accounts • Code corpus: 50 projects from github of most common programming languages (Java, C++, python, CSS, javascript, HTML) Input data 08.09.2017 21
  22. 22. Anna Feit annafeit.de userinterfaces.aalto.fi Formal: Twitter: Code: Input data 08.09.2017 22
  23. 23. Anna Feit annafeit.de userinterfaces.aalto.fi Input data Performance crowdsourced movement times of 7555 different key pairs embedded in nonsense words and repeated 10 times. 08.09.2017 23
  24. 24. Anna Feit annafeit.de userinterfaces.aalto.fi Input data Ergonomics: heuristic cost based on literature, equally punishing movement toward outer right and left, wrist and finger extension, use of modifiers (Alt, Shift) 08.09.2017 24
  25. 25. Anna Feit annafeit.de userinterfaces.aalto.fi Input data Similarity each letter-character and character-character pair rated with a heuristic score, maximum of multiple similarity criteria such as: • visual similarity, e.g. à and a, • semantic or functional similarity, e.g. $ and £ • phonetic similarity, e.g. s and ß • etc. Sparse input matrix 08.09.2017 25
  26. 26. Anna Feit annafeit.de userinterfaces.aalto.fi Input data Distance Manhattan distance quantifies number of rows and columns between two keyslots, additional cost for difference in modifier keys Similar to search behavior of users (row and column wise) 08.09.2017 26
  27. 27. Anna Feit annafeit.de userinterfaces.aalto.fi Optimization in the Iterative Design Process Problem and design space Objectives and constraints Input data Optimization Process Expert Comittee Optimal layouts 08.09.2017 27
  28. 28. Anna Feit annafeit.de userinterfaces.aalto.fi One solution 08.09.2017 28
  29. 29. Anna Feit annafeit.de userinterfaces.aalto.fi One solution French characters easily and quickly accessible 08.09.2017 29
  30. 30. Anna Feit annafeit.de userinterfaces.aalto.fi One solution Easy to learn: familiar position and groups of similar characters 08.09.2017 30
  31. 31. Anna Feit annafeit.de userinterfaces.aalto.fi Comparison with Azerty Our mapping AZERTY Difference of AZERTY to our mapping (smaller is worse) 08.09.2017 31
  32. 32. Anna Feit annafeit.de userinterfaces.aalto.fi Comparison with manual design Our mapping Manually created mapping Difference of manual mapping to our mapping (smaller is worse) 08.09.2017 32
  33. 33. Anna Feit annafeit.de userinterfaces.aalto.fi Optimization in the Iterative Design Process Problem and design space Objectives and constraints Input data Optimization Process Expert Comittee Optimal layouts Manual changes evaluated by objective functions 08.09.2017 33
  34. 34. Anna Feit annafeit.de userinterfaces.aalto.fi Current status Public feedback Nearly 3000 comments from French people (47% positive, 37% negative, 16% neutral) Revision of optimization model and layout to integrate feedback 08.09.2017 34
  35. 35. Anna Feit annafeit.de userinterfaces.aalto.fi Summary 08.09.2017 35 • Large real world instance of the (sparse) quadratic assignment problem • Relevant for millions of users and applicable to other languages • Problem models objectives for performance, ergonomics, and learnability of typing special characters • Optimization as part of an iterative design process

×