SlideShare a Scribd company logo
Optimization ofText Input
Anna Feit, Doctoral student, Aalto University
Dagstuhl Seminar on Computational Interactivity, 06.06.2017
…
Mid-air hand gestures for text input
A
B
C
…
Z
space
?
…
27 letters32 gestures
1033 mappings
Which hand gesture to use for which letter?
Decisions:
For each letter and gesture,
assign the letter to the gesture
or not.
Constraints:
No letter / gesture is assigned
more than once.
Evaluation criteria:
Evaluate each design and pick
the best one.
1033 feasible designs
Mathematical or algorithmic method to find the best design in
the space
• Search very large design spaces
• Efficient and rigorous process
• Quantitative guarantees on the goodness of the outcome
• Explicitly trade-off different criteria and constraints
Optimization
Challenges
Formulation Objectives Optimization
Formulation of the design
problem and space,
identification of design
variables and constraints
argmin
𝐾
෍
𝑘𝜖𝐾
෍
𝑙𝜖𝐾
𝐶 𝐾(𝑘, 𝑙)
Modeling of evaluation criteria
and combination into a fast to
compute objective function
(interaction cost)
Mathematical solver or
approximation algorithm to
efficiently and thoroughly
search the design space
Objective 1
Objective2
The (quadratic) letter assignment problem
Given: n letters – 𝑖, 𝑗 𝜖 𝛴
m gestures – 𝑘, 𝑙 𝜖 𝐺
Let: 𝑥𝑖𝑘 = 1 if letter 𝑖 is assigned to gesture 𝑘, 𝑥𝑖𝑘 = 0 otherwise
𝐗 = {𝑥𝑖𝑘 | ∀ 𝑖 𝜖 𝛴, 𝑘 𝜖 𝐺, 𝑥𝑖𝑘 𝜖 0,1 } characterises the full design space
[Burkhard, 1977]
Formulation
Goal: Find the assignment of letters to gestures that minimize the cost of typing
one character after another
Formulation
[Zhai, Hunter & Smith, 2000]
[Light & Anderson, 1993]
Formulation
[Oulasvirta & Karrenbauer, 2014]
” It is almost impossible to write correctly French
with a keyboard marketed in France”
French Ministry of Culture and Communications
[Feit, Nancel,Weir, John, Bailly,
Karrenbauer, Oulasvirta, upcoming]
Formulation
[Feit, Nancel,Weir, John, Bailly,
Karrenbauer, Oulasvirta, upcoming]
Formulation
é è à ù ê Ê É È À ç Ç æ Æ œ Œ ß ẞ þ Þ ð Ð ŋ Ŋ ij IJ ə Ə ʒ Ʒ & θ ı İ @ ™ ® ©
ſ º ª · ´ ˋ ˆ ¨ ˉ ̲ ˘ ̑ ˇ ˜ ˙ ̣ ̊ ˝ ˵ ¸ ˛ ̦ ̵ ̷ + < > = ± × ÷ ≤ ≥ ≃ % ‰ √ ∞ ¼ ½ ¾
# /  | . , ; : ! ? ¡ ¿ … - - — – _ * † ‡ § ( ) [ ] { } “ ” ‘ ’ « » ‚ „ ‹ › € $ £ ¢ ¤ ¥ ₩
?? ?
> 𝟏𝟎 𝟐𝟏𝟑
𝒂𝒔𝒔𝒊𝒈𝒏𝒎𝒆𝒏𝒕𝒔
[Feit, Nancel,Weir, John, Bailly,
Karrenbauer, Oulasvirta, upcoming]
Formulation
è
Objectives
• Performance – Fitts’ law weighted by letter pair frequency
• QWERTY similarity [Dunlop & Levine 2012]
• Word or gesture clarity [Dunlop & Levine 2012, Smith, Bi & Zhai 2015]
Objectives
Text entry is a complex task involving cognitive and motor processes.
Fast performance involves more than quickly pointing from one key to
another
• Different performance factors
• Different tasks
• Different skill levels
• Different strategies
Objectives
[How we type,
Feit, Weir, Oulasvirta, CHI 2016]
Objectives
…
|C6| = 0.38
Middle vs. Ring, participant 2046
Non-instructed:Ring
Instructed: Middle
Gesture performance models
based on Fitts’ law and theories
of motor control
Anatomical comfort:
Individuation index for each
finger
[Investigating the Dexterity of Multi-Finger Input for Mid-AirText Entry,
Sridhar, Feit,Theobalt, Oulasvirta, CHI 2015]
Objectives
[Feit, Nancel,Weir, John, Bailly,
Karrenbauer, Oulasvirta, upcoming]
Standardization committee:
”The new keyboard should facilitate typing of correct french, should be easy to learn and intuitive to use”
• Performance and ergonomics of typing a
special character before or after a letter
• Intuitive and easy to learn:
• Grouping similar characters
• Position similar to QWERTY
• Language statistics take into account different
typing tasks, e.g. programming, social media
usage, formal writing, etc.
Optimization
Mathematical, exact methods
Linear or Integer Programming, Branch and Bound
methods
Pro:
• Explicit bounds and guarantees on optimality
• Fast solvers available, e.g. Gurobi, CPLEX (IBM)
Con:
• Objective function in closed mathematical form
• Not so flexible (e.g. noisy input data, interactive
optimization, multi-objectives, etc. )
Heuristic approximation algorithms
• Simulated annealing, Genetic algorithms, Biology
inspired algorithms etc.
Pro:
• Straightforward to implement and standard
implementations available
• Flexible, e.g. combine with simulation models
Con:
• No bounds or guarantee to find the global
optimum
• Potentially slow
• Formulation of design space and constraints
Optimization
[Feit, Nancel,Weir, John, Bailly,
Karrenbauer, Oulasvirta, upcoming]
Mathematical solver: Gurobi
• Guarantees to cover the full
design space
• Gives explicit bounds
• Nevertheless: cannot solve to
the global optimum
Challenge: integrate
optimization with stakeholders’
opinions
Challenges
• Multi-objective optimization: weighted sum versus pareto
optimization
• No ”one size fits all”: trade-off different tasks, skills, strategies, etc.
or optimally adapt
• Models and input data: efficient, mathematical models, noisy data
(App) Menus
(Web)
Layouts
UI elements
Gestures
www.annafeit.de @AnnaFeit
Anna Feit
Doctoral student,
finishing end 2017
Research topics:
• Text entry
• UI Optimization
I also know a bit about:
• User modeling
• Mid-air input
• Eye tracking

More Related Content

Similar to Optimization of Text Input

Chounta avouris arv2011
Chounta avouris arv2011Chounta avouris arv2011
Chounta avouris arv2011
Irene-Angelica Chounta
 
Analytics Boot Camp - Slides
Analytics Boot Camp - SlidesAnalytics Boot Camp - Slides
Analytics Boot Camp - Slides
Aditya Joshi
 
BDD Primer
BDD PrimerBDD Primer
BDD Primer
Attila Bertók
 
PL Lecture 01 - preliminaries
PL Lecture 01 - preliminariesPL Lecture 01 - preliminaries
PL Lecture 01 - preliminaries
Schwannden Kuo
 
Lepor: augmented automatic MT evaluation metric
Lepor: augmented automatic MT evaluation metricLepor: augmented automatic MT evaluation metric
Lepor: augmented automatic MT evaluation metric
Lifeng (Aaron) Han
 
LEPOR: an augmented machine translation evaluation metric - Thesis PPT
LEPOR: an augmented machine translation evaluation metric - Thesis PPT LEPOR: an augmented machine translation evaluation metric - Thesis PPT
LEPOR: an augmented machine translation evaluation metric - Thesis PPT
Lifeng (Aaron) Han
 
VOC real world enterprise needs
VOC real world enterprise needsVOC real world enterprise needs
VOC real world enterprise needs
Ivan Berlocher
 
Applying Mobile Device Soft Keyboards to Collaborative Multitouch Tabletop Di...
Applying Mobile Device Soft Keyboards to Collaborative Multitouch Tabletop Di...Applying Mobile Device Soft Keyboards to Collaborative Multitouch Tabletop Di...
Applying Mobile Device Soft Keyboards to Collaborative Multitouch Tabletop Di...
Niklas Elmqvist
 
Training at AI Frontiers 2018 - Lukasz Kaiser: Sequence to Sequence Learning ...
Training at AI Frontiers 2018 - Lukasz Kaiser: Sequence to Sequence Learning ...Training at AI Frontiers 2018 - Lukasz Kaiser: Sequence to Sequence Learning ...
Training at AI Frontiers 2018 - Lukasz Kaiser: Sequence to Sequence Learning ...
AI Frontiers
 
Part 1
Part 1Part 1
Part 1
butest
 
How good of developer are you?
How good of developer are you?How good of developer are you?
How good of developer are you?
Roy Wasse
 
Real-time DirectTranslation System for Sinhala and Tamil Languages.
Real-time DirectTranslation System for Sinhala and Tamil Languages.Real-time DirectTranslation System for Sinhala and Tamil Languages.
Real-time DirectTranslation System for Sinhala and Tamil Languages.
Sheeyam Shellvacumar
 
week1-module1-introduction-merged.pdf
week1-module1-introduction-merged.pdfweek1-module1-introduction-merged.pdf
week1-module1-introduction-merged.pdf
ObedurRahman1
 
Definitive answer to developer productivity v1 .2 Devworld.pptx
Definitive answer to developer productivity v1 .2 Devworld.pptxDefinitive answer to developer productivity v1 .2 Devworld.pptx
Definitive answer to developer productivity v1 .2 Devworld.pptx
Roy Wasse
 
CUHK intern PPT. Machine Translation Evaluation: Methods and Tools
CUHK intern PPT. Machine Translation Evaluation: Methods and Tools CUHK intern PPT. Machine Translation Evaluation: Methods and Tools
CUHK intern PPT. Machine Translation Evaluation: Methods and Tools
Lifeng (Aaron) Han
 
Statistics Concepts and Descriptive Measures Grading Guide.docx
Statistics Concepts and Descriptive Measures Grading Guide.docxStatistics Concepts and Descriptive Measures Grading Guide.docx
Statistics Concepts and Descriptive Measures Grading Guide.docx
whitneyleman54422
 
How good of developer are you v1 .1 ASML.pptx
How good of developer are you v1 .1 ASML.pptxHow good of developer are you v1 .1 ASML.pptx
How good of developer are you v1 .1 ASML.pptx
Roy Wasse
 
Statistics Concepts and Descriptive Measures Grading Guide.docx
Statistics Concepts and Descriptive Measures Grading Guide.docxStatistics Concepts and Descriptive Measures Grading Guide.docx
Statistics Concepts and Descriptive Measures Grading Guide.docx
susanschei
 
R programmingmilano
R programmingmilanoR programmingmilano
R programmingmilano
Ismail Seyrik
 
Artificial Intelligence for Automated Software Testing
Artificial Intelligence for Automated Software TestingArtificial Intelligence for Automated Software Testing
Artificial Intelligence for Automated Software Testing
Lionel Briand
 

Similar to Optimization of Text Input (20)

Chounta avouris arv2011
Chounta avouris arv2011Chounta avouris arv2011
Chounta avouris arv2011
 
Analytics Boot Camp - Slides
Analytics Boot Camp - SlidesAnalytics Boot Camp - Slides
Analytics Boot Camp - Slides
 
BDD Primer
BDD PrimerBDD Primer
BDD Primer
 
PL Lecture 01 - preliminaries
PL Lecture 01 - preliminariesPL Lecture 01 - preliminaries
PL Lecture 01 - preliminaries
 
Lepor: augmented automatic MT evaluation metric
Lepor: augmented automatic MT evaluation metricLepor: augmented automatic MT evaluation metric
Lepor: augmented automatic MT evaluation metric
 
LEPOR: an augmented machine translation evaluation metric - Thesis PPT
LEPOR: an augmented machine translation evaluation metric - Thesis PPT LEPOR: an augmented machine translation evaluation metric - Thesis PPT
LEPOR: an augmented machine translation evaluation metric - Thesis PPT
 
VOC real world enterprise needs
VOC real world enterprise needsVOC real world enterprise needs
VOC real world enterprise needs
 
Applying Mobile Device Soft Keyboards to Collaborative Multitouch Tabletop Di...
Applying Mobile Device Soft Keyboards to Collaborative Multitouch Tabletop Di...Applying Mobile Device Soft Keyboards to Collaborative Multitouch Tabletop Di...
Applying Mobile Device Soft Keyboards to Collaborative Multitouch Tabletop Di...
 
Training at AI Frontiers 2018 - Lukasz Kaiser: Sequence to Sequence Learning ...
Training at AI Frontiers 2018 - Lukasz Kaiser: Sequence to Sequence Learning ...Training at AI Frontiers 2018 - Lukasz Kaiser: Sequence to Sequence Learning ...
Training at AI Frontiers 2018 - Lukasz Kaiser: Sequence to Sequence Learning ...
 
Part 1
Part 1Part 1
Part 1
 
How good of developer are you?
How good of developer are you?How good of developer are you?
How good of developer are you?
 
Real-time DirectTranslation System for Sinhala and Tamil Languages.
Real-time DirectTranslation System for Sinhala and Tamil Languages.Real-time DirectTranslation System for Sinhala and Tamil Languages.
Real-time DirectTranslation System for Sinhala and Tamil Languages.
 
week1-module1-introduction-merged.pdf
week1-module1-introduction-merged.pdfweek1-module1-introduction-merged.pdf
week1-module1-introduction-merged.pdf
 
Definitive answer to developer productivity v1 .2 Devworld.pptx
Definitive answer to developer productivity v1 .2 Devworld.pptxDefinitive answer to developer productivity v1 .2 Devworld.pptx
Definitive answer to developer productivity v1 .2 Devworld.pptx
 
CUHK intern PPT. Machine Translation Evaluation: Methods and Tools
CUHK intern PPT. Machine Translation Evaluation: Methods and Tools CUHK intern PPT. Machine Translation Evaluation: Methods and Tools
CUHK intern PPT. Machine Translation Evaluation: Methods and Tools
 
Statistics Concepts and Descriptive Measures Grading Guide.docx
Statistics Concepts and Descriptive Measures Grading Guide.docxStatistics Concepts and Descriptive Measures Grading Guide.docx
Statistics Concepts and Descriptive Measures Grading Guide.docx
 
How good of developer are you v1 .1 ASML.pptx
How good of developer are you v1 .1 ASML.pptxHow good of developer are you v1 .1 ASML.pptx
How good of developer are you v1 .1 ASML.pptx
 
Statistics Concepts and Descriptive Measures Grading Guide.docx
Statistics Concepts and Descriptive Measures Grading Guide.docxStatistics Concepts and Descriptive Measures Grading Guide.docx
Statistics Concepts and Descriptive Measures Grading Guide.docx
 
R programmingmilano
R programmingmilanoR programmingmilano
R programmingmilano
 
Artificial Intelligence for Automated Software Testing
Artificial Intelligence for Automated Software TestingArtificial Intelligence for Automated Software Testing
Artificial Intelligence for Automated Software Testing
 

Recently uploaded

Describing and Interpreting an Immersive Learning Case with the Immersion Cub...
Describing and Interpreting an Immersive Learning Case with the Immersion Cub...Describing and Interpreting an Immersive Learning Case with the Immersion Cub...
Describing and Interpreting an Immersive Learning Case with the Immersion Cub...
Leonel Morgado
 
fermented food science of sauerkraut.pptx
fermented food science of sauerkraut.pptxfermented food science of sauerkraut.pptx
fermented food science of sauerkraut.pptx
ananya23nair
 
The binding of cosmological structures by massless topological defects
The binding of cosmological structures by massless topological defectsThe binding of cosmological structures by massless topological defects
The binding of cosmological structures by massless topological defects
Sérgio Sacani
 
11.1 Role of physical biological in deterioration of grains.pdf
11.1 Role of physical biological in deterioration of grains.pdf11.1 Role of physical biological in deterioration of grains.pdf
11.1 Role of physical biological in deterioration of grains.pdf
PirithiRaju
 
Microbiology of Central Nervous System INFECTIONS.pdf
Microbiology of Central Nervous System INFECTIONS.pdfMicrobiology of Central Nervous System INFECTIONS.pdf
Microbiology of Central Nervous System INFECTIONS.pdf
sammy700571
 
Applied Science: Thermodynamics, Laws & Methodology.pdf
Applied Science: Thermodynamics, Laws & Methodology.pdfApplied Science: Thermodynamics, Laws & Methodology.pdf
Applied Science: Thermodynamics, Laws & Methodology.pdf
University of Hertfordshire
 
Gadgets for management of stored product pests_Dr.UPR.pdf
Gadgets for management of stored product pests_Dr.UPR.pdfGadgets for management of stored product pests_Dr.UPR.pdf
Gadgets for management of stored product pests_Dr.UPR.pdf
PirithiRaju
 
Summary Of transcription and Translation.pdf
Summary Of transcription and Translation.pdfSummary Of transcription and Translation.pdf
Summary Of transcription and Translation.pdf
vadgavevedant86
 
Direct Seeded Rice - Climate Smart Agriculture
Direct Seeded Rice - Climate Smart AgricultureDirect Seeded Rice - Climate Smart Agriculture
Direct Seeded Rice - Climate Smart Agriculture
International Food Policy Research Institute- South Asia Office
 
JAMES WEBB STUDY THE MASSIVE BLACK HOLE SEEDS
JAMES WEBB STUDY THE MASSIVE BLACK HOLE SEEDSJAMES WEBB STUDY THE MASSIVE BLACK HOLE SEEDS
JAMES WEBB STUDY THE MASSIVE BLACK HOLE SEEDS
Sérgio Sacani
 
CLASS 12th CHEMISTRY SOLID STATE ppt (Animated)
CLASS 12th CHEMISTRY SOLID STATE ppt (Animated)CLASS 12th CHEMISTRY SOLID STATE ppt (Animated)
CLASS 12th CHEMISTRY SOLID STATE ppt (Animated)
eitps1506
 
Methods of grain storage Structures in India.pdf
Methods of grain storage Structures in India.pdfMethods of grain storage Structures in India.pdf
Methods of grain storage Structures in India.pdf
PirithiRaju
 
Farming systems analysis: what have we learnt?.pptx
Farming systems analysis: what have we learnt?.pptxFarming systems analysis: what have we learnt?.pptx
Farming systems analysis: what have we learnt?.pptx
Frédéric Baudron
 
Authoring a personal GPT for your research and practice: How we created the Q...
Authoring a personal GPT for your research and practice: How we created the Q...Authoring a personal GPT for your research and practice: How we created the Q...
Authoring a personal GPT for your research and practice: How we created the Q...
Leonel Morgado
 
HOW DO ORGANISMS REPRODUCE?reproduction part 1
HOW DO ORGANISMS REPRODUCE?reproduction part 1HOW DO ORGANISMS REPRODUCE?reproduction part 1
HOW DO ORGANISMS REPRODUCE?reproduction part 1
Shashank Shekhar Pandey
 
Immersive Learning That Works: Research Grounding and Paths Forward
Immersive Learning That Works: Research Grounding and Paths ForwardImmersive Learning That Works: Research Grounding and Paths Forward
Immersive Learning That Works: Research Grounding and Paths Forward
Leonel Morgado
 
Candidate young stellar objects in the S-cluster: Kinematic analysis of a sub...
Candidate young stellar objects in the S-cluster: Kinematic analysis of a sub...Candidate young stellar objects in the S-cluster: Kinematic analysis of a sub...
Candidate young stellar objects in the S-cluster: Kinematic analysis of a sub...
Sérgio Sacani
 
23PH301 - Optics - Optical Lenses.pptx
23PH301 - Optics  -  Optical Lenses.pptx23PH301 - Optics  -  Optical Lenses.pptx
23PH301 - Optics - Optical Lenses.pptx
RDhivya6
 
Compexometric titration/Chelatorphy titration/chelating titration
Compexometric titration/Chelatorphy titration/chelating titrationCompexometric titration/Chelatorphy titration/chelating titration
Compexometric titration/Chelatorphy titration/chelating titration
Vandana Devesh Sharma
 
Travis Hills of MN is Making Clean Water Accessible to All Through High Flux ...
Travis Hills of MN is Making Clean Water Accessible to All Through High Flux ...Travis Hills of MN is Making Clean Water Accessible to All Through High Flux ...
Travis Hills of MN is Making Clean Water Accessible to All Through High Flux ...
Travis Hills MN
 

Recently uploaded (20)

Describing and Interpreting an Immersive Learning Case with the Immersion Cub...
Describing and Interpreting an Immersive Learning Case with the Immersion Cub...Describing and Interpreting an Immersive Learning Case with the Immersion Cub...
Describing and Interpreting an Immersive Learning Case with the Immersion Cub...
 
fermented food science of sauerkraut.pptx
fermented food science of sauerkraut.pptxfermented food science of sauerkraut.pptx
fermented food science of sauerkraut.pptx
 
The binding of cosmological structures by massless topological defects
The binding of cosmological structures by massless topological defectsThe binding of cosmological structures by massless topological defects
The binding of cosmological structures by massless topological defects
 
11.1 Role of physical biological in deterioration of grains.pdf
11.1 Role of physical biological in deterioration of grains.pdf11.1 Role of physical biological in deterioration of grains.pdf
11.1 Role of physical biological in deterioration of grains.pdf
 
Microbiology of Central Nervous System INFECTIONS.pdf
Microbiology of Central Nervous System INFECTIONS.pdfMicrobiology of Central Nervous System INFECTIONS.pdf
Microbiology of Central Nervous System INFECTIONS.pdf
 
Applied Science: Thermodynamics, Laws & Methodology.pdf
Applied Science: Thermodynamics, Laws & Methodology.pdfApplied Science: Thermodynamics, Laws & Methodology.pdf
Applied Science: Thermodynamics, Laws & Methodology.pdf
 
Gadgets for management of stored product pests_Dr.UPR.pdf
Gadgets for management of stored product pests_Dr.UPR.pdfGadgets for management of stored product pests_Dr.UPR.pdf
Gadgets for management of stored product pests_Dr.UPR.pdf
 
Summary Of transcription and Translation.pdf
Summary Of transcription and Translation.pdfSummary Of transcription and Translation.pdf
Summary Of transcription and Translation.pdf
 
Direct Seeded Rice - Climate Smart Agriculture
Direct Seeded Rice - Climate Smart AgricultureDirect Seeded Rice - Climate Smart Agriculture
Direct Seeded Rice - Climate Smart Agriculture
 
JAMES WEBB STUDY THE MASSIVE BLACK HOLE SEEDS
JAMES WEBB STUDY THE MASSIVE BLACK HOLE SEEDSJAMES WEBB STUDY THE MASSIVE BLACK HOLE SEEDS
JAMES WEBB STUDY THE MASSIVE BLACK HOLE SEEDS
 
CLASS 12th CHEMISTRY SOLID STATE ppt (Animated)
CLASS 12th CHEMISTRY SOLID STATE ppt (Animated)CLASS 12th CHEMISTRY SOLID STATE ppt (Animated)
CLASS 12th CHEMISTRY SOLID STATE ppt (Animated)
 
Methods of grain storage Structures in India.pdf
Methods of grain storage Structures in India.pdfMethods of grain storage Structures in India.pdf
Methods of grain storage Structures in India.pdf
 
Farming systems analysis: what have we learnt?.pptx
Farming systems analysis: what have we learnt?.pptxFarming systems analysis: what have we learnt?.pptx
Farming systems analysis: what have we learnt?.pptx
 
Authoring a personal GPT for your research and practice: How we created the Q...
Authoring a personal GPT for your research and practice: How we created the Q...Authoring a personal GPT for your research and practice: How we created the Q...
Authoring a personal GPT for your research and practice: How we created the Q...
 
HOW DO ORGANISMS REPRODUCE?reproduction part 1
HOW DO ORGANISMS REPRODUCE?reproduction part 1HOW DO ORGANISMS REPRODUCE?reproduction part 1
HOW DO ORGANISMS REPRODUCE?reproduction part 1
 
Immersive Learning That Works: Research Grounding and Paths Forward
Immersive Learning That Works: Research Grounding and Paths ForwardImmersive Learning That Works: Research Grounding and Paths Forward
Immersive Learning That Works: Research Grounding and Paths Forward
 
Candidate young stellar objects in the S-cluster: Kinematic analysis of a sub...
Candidate young stellar objects in the S-cluster: Kinematic analysis of a sub...Candidate young stellar objects in the S-cluster: Kinematic analysis of a sub...
Candidate young stellar objects in the S-cluster: Kinematic analysis of a sub...
 
23PH301 - Optics - Optical Lenses.pptx
23PH301 - Optics  -  Optical Lenses.pptx23PH301 - Optics  -  Optical Lenses.pptx
23PH301 - Optics - Optical Lenses.pptx
 
Compexometric titration/Chelatorphy titration/chelating titration
Compexometric titration/Chelatorphy titration/chelating titrationCompexometric titration/Chelatorphy titration/chelating titration
Compexometric titration/Chelatorphy titration/chelating titration
 
Travis Hills of MN is Making Clean Water Accessible to All Through High Flux ...
Travis Hills of MN is Making Clean Water Accessible to All Through High Flux ...Travis Hills of MN is Making Clean Water Accessible to All Through High Flux ...
Travis Hills of MN is Making Clean Water Accessible to All Through High Flux ...
 

Optimization of Text Input

  • 1. Optimization ofText Input Anna Feit, Doctoral student, Aalto University Dagstuhl Seminar on Computational Interactivity, 06.06.2017
  • 2.
  • 3. … Mid-air hand gestures for text input
  • 4. A B C … Z space ? … 27 letters32 gestures 1033 mappings Which hand gesture to use for which letter?
  • 5. Decisions: For each letter and gesture, assign the letter to the gesture or not. Constraints: No letter / gesture is assigned more than once. Evaluation criteria: Evaluate each design and pick the best one. 1033 feasible designs
  • 6. Mathematical or algorithmic method to find the best design in the space • Search very large design spaces • Efficient and rigorous process • Quantitative guarantees on the goodness of the outcome • Explicitly trade-off different criteria and constraints Optimization
  • 7. Challenges Formulation Objectives Optimization Formulation of the design problem and space, identification of design variables and constraints argmin 𝐾 ෍ 𝑘𝜖𝐾 ෍ 𝑙𝜖𝐾 𝐶 𝐾(𝑘, 𝑙) Modeling of evaluation criteria and combination into a fast to compute objective function (interaction cost) Mathematical solver or approximation algorithm to efficiently and thoroughly search the design space Objective 1 Objective2
  • 8. The (quadratic) letter assignment problem Given: n letters – 𝑖, 𝑗 𝜖 𝛴 m gestures – 𝑘, 𝑙 𝜖 𝐺 Let: 𝑥𝑖𝑘 = 1 if letter 𝑖 is assigned to gesture 𝑘, 𝑥𝑖𝑘 = 0 otherwise 𝐗 = {𝑥𝑖𝑘 | ∀ 𝑖 𝜖 𝛴, 𝑘 𝜖 𝐺, 𝑥𝑖𝑘 𝜖 0,1 } characterises the full design space [Burkhard, 1977] Formulation
  • 9. Goal: Find the assignment of letters to gestures that minimize the cost of typing one character after another Formulation
  • 10. [Zhai, Hunter & Smith, 2000] [Light & Anderson, 1993] Formulation [Oulasvirta & Karrenbauer, 2014]
  • 11. ” It is almost impossible to write correctly French with a keyboard marketed in France” French Ministry of Culture and Communications [Feit, Nancel,Weir, John, Bailly, Karrenbauer, Oulasvirta, upcoming] Formulation
  • 12. [Feit, Nancel,Weir, John, Bailly, Karrenbauer, Oulasvirta, upcoming] Formulation é è à ù ê Ê É È À ç Ç æ Æ œ Œ ß ẞ þ Þ ð Ð ŋ Ŋ ij IJ ə Ə ʒ Ʒ & θ ı İ @ ™ ® © ſ º ª · ´ ˋ ˆ ¨ ˉ ̲ ˘ ̑ ˇ ˜ ˙ ̣ ̊ ˝ ˵ ¸ ˛ ̦ ̵ ̷ + < > = ± × ÷ ≤ ≥ ≃ % ‰ √ ∞ ¼ ½ ¾ # / | . , ; : ! ? ¡ ¿ … - - — – _ * † ‡ § ( ) [ ] { } “ ” ‘ ’ « » ‚ „ ‹ › € $ £ ¢ ¤ ¥ ₩ ?? ? > 𝟏𝟎 𝟐𝟏𝟑 𝒂𝒔𝒔𝒊𝒈𝒏𝒎𝒆𝒏𝒕𝒔
  • 13. [Feit, Nancel,Weir, John, Bailly, Karrenbauer, Oulasvirta, upcoming] Formulation è
  • 14. Objectives • Performance – Fitts’ law weighted by letter pair frequency • QWERTY similarity [Dunlop & Levine 2012] • Word or gesture clarity [Dunlop & Levine 2012, Smith, Bi & Zhai 2015]
  • 15. Objectives Text entry is a complex task involving cognitive and motor processes. Fast performance involves more than quickly pointing from one key to another • Different performance factors • Different tasks • Different skill levels • Different strategies
  • 16. Objectives [How we type, Feit, Weir, Oulasvirta, CHI 2016]
  • 17. Objectives … |C6| = 0.38 Middle vs. Ring, participant 2046 Non-instructed:Ring Instructed: Middle Gesture performance models based on Fitts’ law and theories of motor control Anatomical comfort: Individuation index for each finger [Investigating the Dexterity of Multi-Finger Input for Mid-AirText Entry, Sridhar, Feit,Theobalt, Oulasvirta, CHI 2015]
  • 18. Objectives [Feit, Nancel,Weir, John, Bailly, Karrenbauer, Oulasvirta, upcoming] Standardization committee: ”The new keyboard should facilitate typing of correct french, should be easy to learn and intuitive to use” • Performance and ergonomics of typing a special character before or after a letter • Intuitive and easy to learn: • Grouping similar characters • Position similar to QWERTY • Language statistics take into account different typing tasks, e.g. programming, social media usage, formal writing, etc.
  • 19. Optimization Mathematical, exact methods Linear or Integer Programming, Branch and Bound methods Pro: • Explicit bounds and guarantees on optimality • Fast solvers available, e.g. Gurobi, CPLEX (IBM) Con: • Objective function in closed mathematical form • Not so flexible (e.g. noisy input data, interactive optimization, multi-objectives, etc. ) Heuristic approximation algorithms • Simulated annealing, Genetic algorithms, Biology inspired algorithms etc. Pro: • Straightforward to implement and standard implementations available • Flexible, e.g. combine with simulation models Con: • No bounds or guarantee to find the global optimum • Potentially slow • Formulation of design space and constraints
  • 20. Optimization [Feit, Nancel,Weir, John, Bailly, Karrenbauer, Oulasvirta, upcoming] Mathematical solver: Gurobi • Guarantees to cover the full design space • Gives explicit bounds • Nevertheless: cannot solve to the global optimum Challenge: integrate optimization with stakeholders’ opinions
  • 21. Challenges • Multi-objective optimization: weighted sum versus pareto optimization • No ”one size fits all”: trade-off different tasks, skills, strategies, etc. or optimally adapt • Models and input data: efficient, mathematical models, noisy data
  • 23. www.annafeit.de @AnnaFeit Anna Feit Doctoral student, finishing end 2017 Research topics: • Text entry • UI Optimization I also know a bit about: • User modeling • Mid-air input • Eye tracking