Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Optimizing Special
Character Entry
TheCase ofthe FrenchKeyboard Standard
Anna Feit, Aalto University, Helsinki
Joint work ...
Anna Feit
annafeit.de
userinterfaces.aalto.fi
Fun facts about the French keyboard
• There is no way to type œuf, Été or FR...
Anna Feit
annafeit.de
userinterfaces.aalto.fi
Design Task
08.09.2017
3
Assign 129 keyslots to 122
special characters
Norma...
Anna Feit
annafeit.de
userinterfaces.aalto.fi
Given: n characters – 𝑖, 𝑗 𝜖 Σ
m keyslots – 𝑘, 𝑙 𝜖 𝑆
Let: 𝑥𝑖𝑘 = 1 if charact...
Anna Feit
annafeit.de
userinterfaces.aalto.fi
Given: n characters – 𝑖, 𝑗 𝜖 Σ
m keyslots – 𝑘, 𝑙 𝜖 𝑆
Let: 𝑥𝑖𝑘 = 1 if charact...
Anna Feit
annafeit.de
userinterfaces.aalto.fi
Given: n characters – 𝑖, 𝑗 𝜖 Σ
m keyslots – 𝑘, 𝑙 𝜖 𝑆
Let: 𝑥𝑖𝑘 = 1 if charact...
Anna Feit
annafeit.de
userinterfaces.aalto.fi
Given: n characters – 𝑖, 𝑗 𝜖 Σ
m keyslots – 𝑘, 𝑙 𝜖 𝑆
Let: 𝑥𝑖𝑘 = 1 if charact...
Anna Feit
annafeit.de
userinterfaces.aalto.fi
Optimization of Text Entry methods
08.09.2017
8
[Zhai, Hunter, Smith, 2000] ...
Anna Feit
annafeit.de
userinterfaces.aalto.fi
Special character entry
Quadratic assignment problem is NP hard, with 129 ke...
Anna Feit
annafeit.de
userinterfaces.aalto.fi
Given: n characters – 𝑖, 𝑗 𝜖 Σ
m keyslots – 𝑘, 𝑙 𝜖 𝑆
Let: 𝑥𝑖𝑘 = 1 if charact...
Anna Feit
annafeit.de
userinterfaces.aalto.fi
Given: n characters – 𝑖, 𝑗 𝜖 Σ
m keyslots – 𝑘, 𝑙 𝜖 𝑆
Let: 𝑥𝑖𝑘 = 1 if charact...
Anna Feit
annafeit.de
userinterfaces.aalto.fi
Objectives
08.09.2017
12
Standardization committee:
” The new keyboard shoul...
Anna Feit
annafeit.de
userinterfaces.aalto.fi
Motor Performance: minimize time to move between frequent
character-letter p...
Anna Feit
annafeit.de
userinterfaces.aalto.fi
Objectives
Note: linear function
Ergonomics: minimize frequent extreme movem...
Anna Feit
annafeit.de
userinterfaces.aalto.fi
Learnability (Intuitiveness): minimize the distance between similar
characte...
Anna Feit
annafeit.de
userinterfaces.aalto.fi
Learnability (Intuitiveness): minimize the distance between similar
characte...
Anna Feit
annafeit.de
userinterfaces.aalto.fi
Optimization Problem
08.09.2017
17
Anna Feit
annafeit.de
userinterfaces.aalto.fi
Assign each letter to exactly one key
Optimization Problem
08.09.2017
18
Anna Feit
annafeit.de
userinterfaces.aalto.fi
Assign each letter to exactly one key
Assign each key to at most one letter
...
Anna Feit
annafeit.de
userinterfaces.aalto.fi
Assign each character to exactly one key
Assign each key to at most one char...
Anna Feit
annafeit.de
userinterfaces.aalto.fi
Letter and letter pair frequency
Weighted combination of different corpora r...
Anna Feit
annafeit.de
userinterfaces.aalto.fi
Formal: Twitter: Code:
Input data
08.09.2017
22
Anna Feit
annafeit.de
userinterfaces.aalto.fi
Input data
Performance crowdsourced movement times of 7555 different key pai...
Anna Feit
annafeit.de
userinterfaces.aalto.fi
Input data
Ergonomics: heuristic cost based on literature, equally punishing...
Anna Feit
annafeit.de
userinterfaces.aalto.fi
Input data
Similarity
each letter-character and character-character pair rat...
Anna Feit
annafeit.de
userinterfaces.aalto.fi
Input data
Distance
Manhattan distance quantifies number of rows and columns...
Anna Feit
annafeit.de
userinterfaces.aalto.fi
Optimization in the Iterative Design Process
Problem and design
space
Object...
Anna Feit
annafeit.de
userinterfaces.aalto.fi
One solution
08.09.2017
28
Anna Feit
annafeit.de
userinterfaces.aalto.fi
One solution
French characters easily and
quickly accessible
08.09.2017
29
Anna Feit
annafeit.de
userinterfaces.aalto.fi
One solution
Easy to learn: familiar position and groups of
similar characte...
Anna Feit
annafeit.de
userinterfaces.aalto.fi
Comparison with Azerty
Our mapping
AZERTY
Difference of AZERTY to our
mappin...
Anna Feit
annafeit.de
userinterfaces.aalto.fi
Comparison with manual design
Our mapping
Manually created mapping
Differenc...
Anna Feit
annafeit.de
userinterfaces.aalto.fi
Optimization in the Iterative Design Process
Problem and design
space
Object...
Anna Feit
annafeit.de
userinterfaces.aalto.fi
Current status
Public feedback
Nearly 3000 comments from French people (47% ...
Anna Feit
annafeit.de
userinterfaces.aalto.fi
Summary
08.09.2017
35
• Large real world instance of the (sparse) quadratic ...
Upcoming SlideShare
Loading in …5
×

of

Optimizing Special Character Entry: the Case of the French Keyboard Standard Slide 1 Optimizing Special Character Entry: the Case of the French Keyboard Standard Slide 2 Optimizing Special Character Entry: the Case of the French Keyboard Standard Slide 3 Optimizing Special Character Entry: the Case of the French Keyboard Standard Slide 4 Optimizing Special Character Entry: the Case of the French Keyboard Standard Slide 5 Optimizing Special Character Entry: the Case of the French Keyboard Standard Slide 6 Optimizing Special Character Entry: the Case of the French Keyboard Standard Slide 7 Optimizing Special Character Entry: the Case of the French Keyboard Standard Slide 8 Optimizing Special Character Entry: the Case of the French Keyboard Standard Slide 9 Optimizing Special Character Entry: the Case of the French Keyboard Standard Slide 10 Optimizing Special Character Entry: the Case of the French Keyboard Standard Slide 11 Optimizing Special Character Entry: the Case of the French Keyboard Standard Slide 12 Optimizing Special Character Entry: the Case of the French Keyboard Standard Slide 13 Optimizing Special Character Entry: the Case of the French Keyboard Standard Slide 14 Optimizing Special Character Entry: the Case of the French Keyboard Standard Slide 15 Optimizing Special Character Entry: the Case of the French Keyboard Standard Slide 16 Optimizing Special Character Entry: the Case of the French Keyboard Standard Slide 17 Optimizing Special Character Entry: the Case of the French Keyboard Standard Slide 18 Optimizing Special Character Entry: the Case of the French Keyboard Standard Slide 19 Optimizing Special Character Entry: the Case of the French Keyboard Standard Slide 20 Optimizing Special Character Entry: the Case of the French Keyboard Standard Slide 21 Optimizing Special Character Entry: the Case of the French Keyboard Standard Slide 22 Optimizing Special Character Entry: the Case of the French Keyboard Standard Slide 23 Optimizing Special Character Entry: the Case of the French Keyboard Standard Slide 24 Optimizing Special Character Entry: the Case of the French Keyboard Standard Slide 25 Optimizing Special Character Entry: the Case of the French Keyboard Standard Slide 26 Optimizing Special Character Entry: the Case of the French Keyboard Standard Slide 27 Optimizing Special Character Entry: the Case of the French Keyboard Standard Slide 28 Optimizing Special Character Entry: the Case of the French Keyboard Standard Slide 29 Optimizing Special Character Entry: the Case of the French Keyboard Standard Slide 30 Optimizing Special Character Entry: the Case of the French Keyboard Standard Slide 31 Optimizing Special Character Entry: the Case of the French Keyboard Standard Slide 32 Optimizing Special Character Entry: the Case of the French Keyboard Standard Slide 33 Optimizing Special Character Entry: the Case of the French Keyboard Standard Slide 34 Optimizing Special Character Entry: the Case of the French Keyboard Standard Slide 35
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

1 Like

Share

Download to read offline

Optimizing Special Character Entry: the Case of the French Keyboard Standard

Download to read offline

=== Invited talk at the International Conference on Operations Research, September 2017, Berlin ===

We present the optimization of the new French standard for keyboard layouts. The typical French keyboard layout made it overly complicated or even impossible to type all French characters, punctuation marks, and other symbols. The goal of this new standard was to enable and facilitate the correct spelling of French, and allow the entry of symbols common in programming languages, mathematical expressions, and other European languages.
Therefore, we had to assign over 115 characters to 148 slots on the keyboard, optimizing their placement with respect to each other and to the numbers (0-9) and non-accentuated letters (a-z) that were kept unchanged.
In interaction with an expert committee, we identified four objectives: input performance and ergonomics, similarity to the prior keyboard, and coherent placement of similar symbols.
We collected extensive performance data, language statistics, expert ratings on character similarities, and ergonomic scores to implement and solve a multi-objective quadratic assignment problem. In an iterative process, we computed and adapted a range of solutions in interaction with language experts. The resulting design unifies mathematical bounds with expert opinions and constraints.
The new French keyboard layout is the first modern standard where computational optimization methods were used in interaction with domain experts to implement an optimal keyboard design.

Related Books

Free with a 30 day trial from Scribd

See all

Optimizing Special Character Entry: the Case of the French Keyboard Standard

  1. 1. Optimizing Special Character Entry TheCase ofthe FrenchKeyboard Standard Anna Feit, Aalto University, Helsinki Joint work with Mathieu Nancel, Maximilian John, Andreas Karrenbauer, Daryl Weir, and Antti Oulasvirta
  2. 2. Anna Feit annafeit.de userinterfaces.aalto.fi Fun facts about the French keyboard • There is no way to type œuf, Été or FRANÇAIS (egg, Summer, FRENCH) • où (where) is the only word that contains ù, yet it has it’s own key • No french quotation marks « » • Typing . requires Shift, so do all numbers, and try finding the @ key… ” It is almost impossible to write correctly French with a keyboard marketed in France” French Ministry of Culture and Communications 08.09.2017 2
  3. 3. Anna Feit annafeit.de userinterfaces.aalto.fi Design Task 08.09.2017 3 Assign 129 keyslots to 122 special characters Normal Shift Alt Gr Alt Gr + Shift ! ? " # $ % & ' ( ) [ ] { } * + , - . / : ; < = > ≤ ≥ ÷ @ | ¿ ¡ ¢ £ ¤ ¥ § © ª « ® ° ± · º » ¼ ½ ¾ × ù Ù à æ Æ ç Ç è é ê œ Œ þ İ ı IJ ij Ŋ ŋ ſ Ə Ʒ ə ʒ μ ð Ð Þ ß ẞ _ - – — ‘ ’ ‚ “ ” „ † ‡ … ‰ ‹ › ₩ ™ √ ∞ ≃ · ´ ˋ ˆ ¨ ˉ ̲ ˘ ̑ ˇ ˜ ˙ ̣ ̊ ˝ ˵ ¸ ˛ ̦ ̵ ̷ > 𝟏𝟎 𝟐𝟏𝟑 𝒂𝒔𝒔𝒊𝒈𝒏𝒎𝒆𝒏𝒕𝒔
  4. 4. Anna Feit annafeit.de userinterfaces.aalto.fi Given: n characters – 𝑖, 𝑗 𝜖 Σ m keyslots – 𝑘, 𝑙 𝜖 𝑆 Let: 𝑥𝑖𝑘 = 1 if character 𝑖 is assigned to keyslot 𝑘, 𝑥𝑖𝑘 = 0 otherwise The (letter) assignment problem [Burkhard, 1977] 08.09.2017 4
  5. 5. Anna Feit annafeit.de userinterfaces.aalto.fi Given: n characters – 𝑖, 𝑗 𝜖 Σ m keyslots – 𝑘, 𝑙 𝜖 𝑆 Let: 𝑥𝑖𝑘 = 1 if character 𝑖 is assigned to keyslot 𝑘, 𝑥𝑖𝑘 = 0 otherwise Goal: Find the assignment of characters to keyslots that minimizes the cost of typing one character after another The (letter) assignment problem [Burkhard, 1977] 08.09.2017 5
  6. 6. Anna Feit annafeit.de userinterfaces.aalto.fi Given: n characters – 𝑖, 𝑗 𝜖 Σ m keyslots – 𝑘, 𝑙 𝜖 𝑆 Let: 𝑥𝑖𝑘 = 1 if character 𝑖 is assigned to keyslot 𝑘, 𝑥𝑖𝑘 = 0 otherwise Goal: Find the assignment of characters to keyslots that minimizes the cost of typing one character after another [Burkhard, 1977] The (letter) assignment problem Assign each symbol to exactly one input action 08.09.2017 6
  7. 7. Anna Feit annafeit.de userinterfaces.aalto.fi Given: n characters – 𝑖, 𝑗 𝜖 Σ m keyslots – 𝑘, 𝑙 𝜖 𝑆 Let: 𝑥𝑖𝑘 = 1 if character 𝑖 is assigned to keyslot 𝑘, 𝑥𝑖𝑘 = 0 otherwise Goal: Find the assignment of characters to keyslots that minimizes the cost of typing one character after another [Burkhard, 1977] The (letter) assignment problem Assign each character to exactly one keyslot Assign each keyslot not more than once 08.09.2017 7
  8. 8. Anna Feit annafeit.de userinterfaces.aalto.fi Optimization of Text Entry methods 08.09.2017 8 [Zhai, Hunter, Smith, 2000] [Karrenbauer and Oulasvirta, 2014] [Feit and Oulasvirta, 2014] [Sridhar, Feit, Theobalt and Oulasvirta, 2015] …
  9. 9. Anna Feit annafeit.de userinterfaces.aalto.fi Special character entry Quadratic assignment problem is NP hard, with 129 keyslots, 122 characters too complex to even approximate. Observation: • Special characters are infrequent in comparison to letters • Mostly entered before or after a letter We can simplify the quadratic problem to a linear assignment problem plus a sparse quadratic part 08.09.2017 9
  10. 10. Anna Feit annafeit.de userinterfaces.aalto.fi Given: n characters – 𝑖, 𝑗 𝜖 Σ m keyslots – 𝑘, 𝑙 𝜖 𝑆 Let: 𝑥𝑖𝑘 = 1 if character 𝑖 is assigned to keyslot 𝑘, 𝑥𝑖𝑘 = 0 otherwise Goal: Find the assignment of characters to keyslots that minimizes the cost of typing one character after another or before / after a letter Assignment problem 08.09.2017 10
  11. 11. Anna Feit annafeit.de userinterfaces.aalto.fi Given: n characters – 𝑖, 𝑗 𝜖 Σ m keyslots – 𝑘, 𝑙 𝜖 𝑆 Let: 𝑥𝑖𝑘 = 1 if character 𝑖 is assigned to keyslot 𝑘, 𝑥𝑖𝑘 = 0 otherwise Goal: Find the assignment of characters to keyslots that minimizes the cost of typing one character after another or before / after a letter Assignment problem 08.09.2017 11 sparse
  12. 12. Anna Feit annafeit.de userinterfaces.aalto.fi Objectives 08.09.2017 12 Standardization committee: ” The new keyboard should • facilitate typing of correct french • allow quick and easy access to all French characters • be easy to learn and intuitive to use ”
  13. 13. Anna Feit annafeit.de userinterfaces.aalto.fi Motor Performance: minimize time to move between frequent character-letter pairs Objectives Note: linear function 08.09.2017 13
  14. 14. Anna Feit annafeit.de userinterfaces.aalto.fi Objectives Note: linear function Ergonomics: minimize frequent extreme movements of wrist and fingers when entering a special character in combination with a letter Motor Performance: minimize time to move between frequent character-letter pairs Note: linear function 08.09.2017 14
  15. 15. Anna Feit annafeit.de userinterfaces.aalto.fi Learnability (Intuitiveness): minimize the distance between similar characters to ensure a consistent and intuitive layout that is easy to use and learn. Objectives 08.09.2017 15
  16. 16. Anna Feit annafeit.de userinterfaces.aalto.fi Learnability (Intuitiveness): minimize the distance between similar characters to ensure a consistent and intuitive layout that is easy to use and learn. Objectives Learnability (Familiarity): minimize the distance of a character to its position on the standard keyboard layout. Note: linear function 08.09.2017 16
  17. 17. Anna Feit annafeit.de userinterfaces.aalto.fi Optimization Problem 08.09.2017 17
  18. 18. Anna Feit annafeit.de userinterfaces.aalto.fi Assign each letter to exactly one key Optimization Problem 08.09.2017 18
  19. 19. Anna Feit annafeit.de userinterfaces.aalto.fi Assign each letter to exactly one key Assign each key to at most one letter Optimization Problem 08.09.2017 19
  20. 20. Anna Feit annafeit.de userinterfaces.aalto.fi Assign each character to exactly one key Assign each key to at most one character Capital character must be on shifted key of lower case character Optimization Problem 08.09.2017 20
  21. 21. Anna Feit annafeit.de userinterfaces.aalto.fi Letter and letter pair frequency Weighted combination of different corpora representing diverse tasks • Formal corpus: legal texts, newspaper articles, wikipedia, emails, facebook 10 different corpora averaged, largest coverage of symbols • Twitter corpus: 10000 tweets from 10 different twitter accounts, 6 "classic" (e.g. newspaper) and 4 "popular" accounts • Code corpus: 50 projects from github of most common programming languages (Java, C++, python, CSS, javascript, HTML) Input data 08.09.2017 21
  22. 22. Anna Feit annafeit.de userinterfaces.aalto.fi Formal: Twitter: Code: Input data 08.09.2017 22
  23. 23. Anna Feit annafeit.de userinterfaces.aalto.fi Input data Performance crowdsourced movement times of 7555 different key pairs embedded in nonsense words and repeated 10 times. 08.09.2017 23
  24. 24. Anna Feit annafeit.de userinterfaces.aalto.fi Input data Ergonomics: heuristic cost based on literature, equally punishing movement toward outer right and left, wrist and finger extension, use of modifiers (Alt, Shift) 08.09.2017 24
  25. 25. Anna Feit annafeit.de userinterfaces.aalto.fi Input data Similarity each letter-character and character-character pair rated with a heuristic score, maximum of multiple similarity criteria such as: • visual similarity, e.g. à and a, • semantic or functional similarity, e.g. $ and £ • phonetic similarity, e.g. s and ß • etc. Sparse input matrix 08.09.2017 25
  26. 26. Anna Feit annafeit.de userinterfaces.aalto.fi Input data Distance Manhattan distance quantifies number of rows and columns between two keyslots, additional cost for difference in modifier keys Similar to search behavior of users (row and column wise) 08.09.2017 26
  27. 27. Anna Feit annafeit.de userinterfaces.aalto.fi Optimization in the Iterative Design Process Problem and design space Objectives and constraints Input data Optimization Process Expert Comittee Optimal layouts 08.09.2017 27
  28. 28. Anna Feit annafeit.de userinterfaces.aalto.fi One solution 08.09.2017 28
  29. 29. Anna Feit annafeit.de userinterfaces.aalto.fi One solution French characters easily and quickly accessible 08.09.2017 29
  30. 30. Anna Feit annafeit.de userinterfaces.aalto.fi One solution Easy to learn: familiar position and groups of similar characters 08.09.2017 30
  31. 31. Anna Feit annafeit.de userinterfaces.aalto.fi Comparison with Azerty Our mapping AZERTY Difference of AZERTY to our mapping (smaller is worse) 08.09.2017 31
  32. 32. Anna Feit annafeit.de userinterfaces.aalto.fi Comparison with manual design Our mapping Manually created mapping Difference of manual mapping to our mapping (smaller is worse) 08.09.2017 32
  33. 33. Anna Feit annafeit.de userinterfaces.aalto.fi Optimization in the Iterative Design Process Problem and design space Objectives and constraints Input data Optimization Process Expert Comittee Optimal layouts Manual changes evaluated by objective functions 08.09.2017 33
  34. 34. Anna Feit annafeit.de userinterfaces.aalto.fi Current status Public feedback Nearly 3000 comments from French people (47% positive, 37% negative, 16% neutral) Revision of optimization model and layout to integrate feedback 08.09.2017 34
  35. 35. Anna Feit annafeit.de userinterfaces.aalto.fi Summary 08.09.2017 35 • Large real world instance of the (sparse) quadratic assignment problem • Relevant for millions of users and applicable to other languages • Problem models objectives for performance, ergonomics, and learnability of typing special characters • Optimization as part of an iterative design process
  • MosBar

    Apr. 29, 2020

=== Invited talk at the International Conference on Operations Research, September 2017, Berlin === We present the optimization of the new French standard for keyboard layouts. The typical French keyboard layout made it overly complicated or even impossible to type all French characters, punctuation marks, and other symbols. The goal of this new standard was to enable and facilitate the correct spelling of French, and allow the entry of symbols common in programming languages, mathematical expressions, and other European languages. Therefore, we had to assign over 115 characters to 148 slots on the keyboard, optimizing their placement with respect to each other and to the numbers (0-9) and non-accentuated letters (a-z) that were kept unchanged. In interaction with an expert committee, we identified four objectives: input performance and ergonomics, similarity to the prior keyboard, and coherent placement of similar symbols. We collected extensive performance data, language statistics, expert ratings on character similarities, and ergonomic scores to implement and solve a multi-objective quadratic assignment problem. In an iterative process, we computed and adapted a range of solutions in interaction with language experts. The resulting design unifies mathematical bounds with expert opinions and constraints. The new French keyboard layout is the first modern standard where computational optimization methods were used in interaction with domain experts to implement an optimal keyboard design.

Views

Total views

351

On Slideshare

0

From embeds

0

Number of embeds

3

Actions

Downloads

0

Shares

0

Comments

0

Likes

1

×