SlideShare a Scribd company logo
1 of 23
Download to read offline
Optimization ofText Input
Anna Feit, Doctoral student, Aalto University
Dagstuhl Seminar on Computational Interactivity, 06.06.2017
…
Mid-air hand gestures for text input
A
B
C
…
Z
space
?
…
27 letters32 gestures
1033 mappings
Which hand gesture to use for which letter?
Decisions:
For each letter and gesture,
assign the letter to the gesture
or not.
Constraints:
No letter / gesture is assigned
more than once.
Evaluation criteria:
Evaluate each design and pick
the best one.
1033 feasible designs
Mathematical or algorithmic method to find the best design in
the space
• Search very large design spaces
• Efficient and rigorous process
• Quantitative guarantees on the goodness of the outcome
• Explicitly trade-off different criteria and constraints
Optimization
Challenges
Formulation Objectives Optimization
Formulation of the design
problem and space,
identification of design
variables and constraints
argmin
𝐾
෍
𝑘𝜖𝐾
෍
𝑙𝜖𝐾
𝐶 𝐾(𝑘, 𝑙)
Modeling of evaluation criteria
and combination into a fast to
compute objective function
(interaction cost)
Mathematical solver or
approximation algorithm to
efficiently and thoroughly
search the design space
Objective 1
Objective2
The (quadratic) letter assignment problem
Given: n letters – 𝑖, 𝑗 𝜖 𝛴
m gestures – 𝑘, 𝑙 𝜖 𝐺
Let: 𝑥𝑖𝑘 = 1 if letter 𝑖 is assigned to gesture 𝑘, 𝑥𝑖𝑘 = 0 otherwise
𝐗 = {𝑥𝑖𝑘 | ∀ 𝑖 𝜖 𝛴, 𝑘 𝜖 𝐺, 𝑥𝑖𝑘 𝜖 0,1 } characterises the full design space
[Burkhard, 1977]
Formulation
Goal: Find the assignment of letters to gestures that minimize the cost of typing
one character after another
Formulation
[Zhai, Hunter & Smith, 2000]
[Light & Anderson, 1993]
Formulation
[Oulasvirta & Karrenbauer, 2014]
” It is almost impossible to write correctly French
with a keyboard marketed in France”
French Ministry of Culture and Communications
[Feit, Nancel,Weir, John, Bailly,
Karrenbauer, Oulasvirta, upcoming]
Formulation
[Feit, Nancel,Weir, John, Bailly,
Karrenbauer, Oulasvirta, upcoming]
Formulation
é è à ù ê Ê É È À ç Ç æ Æ œ Œ ß ẞ þ Þ ð Ð ŋ Ŋ ij IJ ə Ə ʒ Ʒ & θ ı İ @ ™ ® ©
ſ º ª · ´ ˋ ˆ ¨ ˉ ̲ ˘ ̑ ˇ ˜ ˙ ̣ ̊ ˝ ˵ ¸ ˛ ̦ ̵ ̷ + < > = ± × ÷ ≤ ≥ ≃ % ‰ √ ∞ ¼ ½ ¾
# /  | . , ; : ! ? ¡ ¿ … - - — – _ * † ‡ § ( ) [ ] { } “ ” ‘ ’ « » ‚ „ ‹ › € $ £ ¢ ¤ ¥ ₩
?? ?
> 𝟏𝟎 𝟐𝟏𝟑
𝒂𝒔𝒔𝒊𝒈𝒏𝒎𝒆𝒏𝒕𝒔
[Feit, Nancel,Weir, John, Bailly,
Karrenbauer, Oulasvirta, upcoming]
Formulation
è
Objectives
• Performance – Fitts’ law weighted by letter pair frequency
• QWERTY similarity [Dunlop & Levine 2012]
• Word or gesture clarity [Dunlop & Levine 2012, Smith, Bi & Zhai 2015]
Objectives
Text entry is a complex task involving cognitive and motor processes.
Fast performance involves more than quickly pointing from one key to
another
• Different performance factors
• Different tasks
• Different skill levels
• Different strategies
Objectives
[How we type,
Feit, Weir, Oulasvirta, CHI 2016]
Objectives
…
|C6| = 0.38
Middle vs. Ring, participant 2046
Non-instructed:Ring
Instructed: Middle
Gesture performance models
based on Fitts’ law and theories
of motor control
Anatomical comfort:
Individuation index for each
finger
[Investigating the Dexterity of Multi-Finger Input for Mid-AirText Entry,
Sridhar, Feit,Theobalt, Oulasvirta, CHI 2015]
Objectives
[Feit, Nancel,Weir, John, Bailly,
Karrenbauer, Oulasvirta, upcoming]
Standardization committee:
”The new keyboard should facilitate typing of correct french, should be easy to learn and intuitive to use”
• Performance and ergonomics of typing a
special character before or after a letter
• Intuitive and easy to learn:
• Grouping similar characters
• Position similar to QWERTY
• Language statistics take into account different
typing tasks, e.g. programming, social media
usage, formal writing, etc.
Optimization
Mathematical, exact methods
Linear or Integer Programming, Branch and Bound
methods
Pro:
• Explicit bounds and guarantees on optimality
• Fast solvers available, e.g. Gurobi, CPLEX (IBM)
Con:
• Objective function in closed mathematical form
• Not so flexible (e.g. noisy input data, interactive
optimization, multi-objectives, etc. )
Heuristic approximation algorithms
• Simulated annealing, Genetic algorithms, Biology
inspired algorithms etc.
Pro:
• Straightforward to implement and standard
implementations available
• Flexible, e.g. combine with simulation models
Con:
• No bounds or guarantee to find the global
optimum
• Potentially slow
• Formulation of design space and constraints
Optimization
[Feit, Nancel,Weir, John, Bailly,
Karrenbauer, Oulasvirta, upcoming]
Mathematical solver: Gurobi
• Guarantees to cover the full
design space
• Gives explicit bounds
• Nevertheless: cannot solve to
the global optimum
Challenge: integrate
optimization with stakeholders’
opinions
Challenges
• Multi-objective optimization: weighted sum versus pareto
optimization
• No ”one size fits all”: trade-off different tasks, skills, strategies, etc.
or optimally adapt
• Models and input data: efficient, mathematical models, noisy data
(App) Menus
(Web)
Layouts
UI elements
Gestures
www.annafeit.de @AnnaFeit
Anna Feit
Doctoral student,
finishing end 2017
Research topics:
• Text entry
• UI Optimization
I also know a bit about:
• User modeling
• Mid-air input
• Eye tracking

More Related Content

Similar to Optimization of Text Input

Analytics Boot Camp - Slides
Analytics Boot Camp - SlidesAnalytics Boot Camp - Slides
Analytics Boot Camp - SlidesAditya Joshi
 
PL Lecture 01 - preliminaries
PL Lecture 01 - preliminariesPL Lecture 01 - preliminaries
PL Lecture 01 - preliminariesSchwannden Kuo
 
Lepor: augmented automatic MT evaluation metric
Lepor: augmented automatic MT evaluation metricLepor: augmented automatic MT evaluation metric
Lepor: augmented automatic MT evaluation metricLifeng (Aaron) Han
 
LEPOR: an augmented machine translation evaluation metric - Thesis PPT
LEPOR: an augmented machine translation evaluation metric - Thesis PPT LEPOR: an augmented machine translation evaluation metric - Thesis PPT
LEPOR: an augmented machine translation evaluation metric - Thesis PPT Lifeng (Aaron) Han
 
VOC real world enterprise needs
VOC real world enterprise needsVOC real world enterprise needs
VOC real world enterprise needsIvan Berlocher
 
Applying Mobile Device Soft Keyboards to Collaborative Multitouch Tabletop Di...
Applying Mobile Device Soft Keyboards to Collaborative Multitouch Tabletop Di...Applying Mobile Device Soft Keyboards to Collaborative Multitouch Tabletop Di...
Applying Mobile Device Soft Keyboards to Collaborative Multitouch Tabletop Di...Niklas Elmqvist
 
Training at AI Frontiers 2018 - Lukasz Kaiser: Sequence to Sequence Learning ...
Training at AI Frontiers 2018 - Lukasz Kaiser: Sequence to Sequence Learning ...Training at AI Frontiers 2018 - Lukasz Kaiser: Sequence to Sequence Learning ...
Training at AI Frontiers 2018 - Lukasz Kaiser: Sequence to Sequence Learning ...AI Frontiers
 
Part 1
Part 1Part 1
Part 1butest
 
How good of developer are you?
How good of developer are you?How good of developer are you?
How good of developer are you?Roy Wasse
 
Real-time DirectTranslation System for Sinhala and Tamil Languages.
Real-time DirectTranslation System for Sinhala and Tamil Languages.Real-time DirectTranslation System for Sinhala and Tamil Languages.
Real-time DirectTranslation System for Sinhala and Tamil Languages.Sheeyam Shellvacumar
 
week1-module1-introduction-merged.pdf
week1-module1-introduction-merged.pdfweek1-module1-introduction-merged.pdf
week1-module1-introduction-merged.pdfObedurRahman1
 
CUHK intern PPT. Machine Translation Evaluation: Methods and Tools
CUHK intern PPT. Machine Translation Evaluation: Methods and Tools CUHK intern PPT. Machine Translation Evaluation: Methods and Tools
CUHK intern PPT. Machine Translation Evaluation: Methods and Tools Lifeng (Aaron) Han
 
Statistics Concepts and Descriptive Measures Grading Guide.docx
Statistics Concepts and Descriptive Measures Grading Guide.docxStatistics Concepts and Descriptive Measures Grading Guide.docx
Statistics Concepts and Descriptive Measures Grading Guide.docxwhitneyleman54422
 
Statistics Concepts and Descriptive Measures Grading Guide.docx
Statistics Concepts and Descriptive Measures Grading Guide.docxStatistics Concepts and Descriptive Measures Grading Guide.docx
Statistics Concepts and Descriptive Measures Grading Guide.docxsusanschei
 
Artificial Intelligence for Automated Software Testing
Artificial Intelligence for Automated Software TestingArtificial Intelligence for Automated Software Testing
Artificial Intelligence for Automated Software TestingLionel Briand
 
Cncwebworld c programming,
Cncwebworld c programming,Cncwebworld c programming,
Cncwebworld c programming,CNC WEB WORLD
 
End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF
End-to-end sequence labeling via bi-directional LSTM-CNNs-CRFEnd-to-end sequence labeling via bi-directional LSTM-CNNs-CRF
End-to-end sequence labeling via bi-directional LSTM-CNNs-CRFJayavardhan Reddy Peddamail
 

Similar to Optimization of Text Input (20)

Chounta avouris arv2011
Chounta avouris arv2011Chounta avouris arv2011
Chounta avouris arv2011
 
Analytics Boot Camp - Slides
Analytics Boot Camp - SlidesAnalytics Boot Camp - Slides
Analytics Boot Camp - Slides
 
BDD Primer
BDD PrimerBDD Primer
BDD Primer
 
PL Lecture 01 - preliminaries
PL Lecture 01 - preliminariesPL Lecture 01 - preliminaries
PL Lecture 01 - preliminaries
 
Lepor: augmented automatic MT evaluation metric
Lepor: augmented automatic MT evaluation metricLepor: augmented automatic MT evaluation metric
Lepor: augmented automatic MT evaluation metric
 
LEPOR: an augmented machine translation evaluation metric - Thesis PPT
LEPOR: an augmented machine translation evaluation metric - Thesis PPT LEPOR: an augmented machine translation evaluation metric - Thesis PPT
LEPOR: an augmented machine translation evaluation metric - Thesis PPT
 
VOC real world enterprise needs
VOC real world enterprise needsVOC real world enterprise needs
VOC real world enterprise needs
 
Applying Mobile Device Soft Keyboards to Collaborative Multitouch Tabletop Di...
Applying Mobile Device Soft Keyboards to Collaborative Multitouch Tabletop Di...Applying Mobile Device Soft Keyboards to Collaborative Multitouch Tabletop Di...
Applying Mobile Device Soft Keyboards to Collaborative Multitouch Tabletop Di...
 
Training at AI Frontiers 2018 - Lukasz Kaiser: Sequence to Sequence Learning ...
Training at AI Frontiers 2018 - Lukasz Kaiser: Sequence to Sequence Learning ...Training at AI Frontiers 2018 - Lukasz Kaiser: Sequence to Sequence Learning ...
Training at AI Frontiers 2018 - Lukasz Kaiser: Sequence to Sequence Learning ...
 
Part 1
Part 1Part 1
Part 1
 
How good of developer are you?
How good of developer are you?How good of developer are you?
How good of developer are you?
 
Real-time DirectTranslation System for Sinhala and Tamil Languages.
Real-time DirectTranslation System for Sinhala and Tamil Languages.Real-time DirectTranslation System for Sinhala and Tamil Languages.
Real-time DirectTranslation System for Sinhala and Tamil Languages.
 
week1-module1-introduction-merged.pdf
week1-module1-introduction-merged.pdfweek1-module1-introduction-merged.pdf
week1-module1-introduction-merged.pdf
 
CUHK intern PPT. Machine Translation Evaluation: Methods and Tools
CUHK intern PPT. Machine Translation Evaluation: Methods and Tools CUHK intern PPT. Machine Translation Evaluation: Methods and Tools
CUHK intern PPT. Machine Translation Evaluation: Methods and Tools
 
Statistics Concepts and Descriptive Measures Grading Guide.docx
Statistics Concepts and Descriptive Measures Grading Guide.docxStatistics Concepts and Descriptive Measures Grading Guide.docx
Statistics Concepts and Descriptive Measures Grading Guide.docx
 
Statistics Concepts and Descriptive Measures Grading Guide.docx
Statistics Concepts and Descriptive Measures Grading Guide.docxStatistics Concepts and Descriptive Measures Grading Guide.docx
Statistics Concepts and Descriptive Measures Grading Guide.docx
 
R programmingmilano
R programmingmilanoR programmingmilano
R programmingmilano
 
Artificial Intelligence for Automated Software Testing
Artificial Intelligence for Automated Software TestingArtificial Intelligence for Automated Software Testing
Artificial Intelligence for Automated Software Testing
 
Cncwebworld c programming,
Cncwebworld c programming,Cncwebworld c programming,
Cncwebworld c programming,
 
End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF
End-to-end sequence labeling via bi-directional LSTM-CNNs-CRFEnd-to-end sequence labeling via bi-directional LSTM-CNNs-CRF
End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF
 

Recently uploaded

ESSENTIAL FEATURES REQUIRED FOR ESTABLISHING FOUR TYPES OF BIOSAFETY LABORATO...
ESSENTIAL FEATURES REQUIRED FOR ESTABLISHING FOUR TYPES OF BIOSAFETY LABORATO...ESSENTIAL FEATURES REQUIRED FOR ESTABLISHING FOUR TYPES OF BIOSAFETY LABORATO...
ESSENTIAL FEATURES REQUIRED FOR ESTABLISHING FOUR TYPES OF BIOSAFETY LABORATO...Chayanika Das
 
BACTERIAL SECRETION SYSTEM by Dr. Chayanika Das
BACTERIAL SECRETION SYSTEM by Dr. Chayanika DasBACTERIAL SECRETION SYSTEM by Dr. Chayanika Das
BACTERIAL SECRETION SYSTEM by Dr. Chayanika DasChayanika Das
 
Presentation about adversarial image attacks
Presentation about adversarial image attacksPresentation about adversarial image attacks
Presentation about adversarial image attacksKoshinKhodiyar
 
DETECTION OF MUTATION BY CLB METHOD.pptx
DETECTION OF MUTATION BY CLB METHOD.pptxDETECTION OF MUTATION BY CLB METHOD.pptx
DETECTION OF MUTATION BY CLB METHOD.pptx201bo007
 
Science (Communication) and Wikipedia - Potentials and Pitfalls
Science (Communication) and Wikipedia - Potentials and PitfallsScience (Communication) and Wikipedia - Potentials and Pitfalls
Science (Communication) and Wikipedia - Potentials and PitfallsDobusch Leonhard
 
Introduction of Organ-On-A-Chip - Creative Biolabs
Introduction of Organ-On-A-Chip - Creative BiolabsIntroduction of Organ-On-A-Chip - Creative Biolabs
Introduction of Organ-On-A-Chip - Creative BiolabsCreative-Biolabs
 
Remote patient monitoring :Health care transformation
Remote patient monitoring :Health care transformationRemote patient monitoring :Health care transformation
Remote patient monitoring :Health care transformationfahad Alotaibiu
 
DNA isolation molecular biology practical.pptx
DNA isolation molecular biology practical.pptxDNA isolation molecular biology practical.pptx
DNA isolation molecular biology practical.pptxGiDMOh
 
STELLAR SYSTEM IN PTERIDOPHYTE Seminar 2023- By Karishma
STELLAR SYSTEM IN PTERIDOPHYTE Seminar 2023- By KarishmaSTELLAR SYSTEM IN PTERIDOPHYTE Seminar 2023- By Karishma
STELLAR SYSTEM IN PTERIDOPHYTE Seminar 2023- By KarishmaAMiracle3
 
Observation of Gravitational Waves from the Coalescence of a 2.5–4.5 M⊙ Compa...
Observation of Gravitational Waves from the Coalescence of a 2.5–4.5 M⊙ Compa...Observation of Gravitational Waves from the Coalescence of a 2.5–4.5 M⊙ Compa...
Observation of Gravitational Waves from the Coalescence of a 2.5–4.5 M⊙ Compa...Sérgio Sacani
 
EGYPTIAN IMPRINT IN SPAIN Lecture by Dr Abeer Zahana
EGYPTIAN IMPRINT IN SPAIN Lecture by Dr Abeer ZahanaEGYPTIAN IMPRINT IN SPAIN Lecture by Dr Abeer Zahana
EGYPTIAN IMPRINT IN SPAIN Lecture by Dr Abeer ZahanaDr.Mahmoud Abbas
 
Food_safety_Management_pptx.pptx in microbiology
Food_safety_Management_pptx.pptx in microbiologyFood_safety_Management_pptx.pptx in microbiology
Food_safety_Management_pptx.pptx in microbiologyHemantThakare8
 
Pests of Sunflower_Binomics_Identification_Dr.UPR
Pests of Sunflower_Binomics_Identification_Dr.UPRPests of Sunflower_Binomics_Identification_Dr.UPR
Pests of Sunflower_Binomics_Identification_Dr.UPRPirithiRaju
 
Environmental acoustics- noise criteria.pptx
Environmental acoustics- noise criteria.pptxEnvironmental acoustics- noise criteria.pptx
Environmental acoustics- noise criteria.pptxpriyankatabhane
 
Environmental Acoustics- Speech interference level, acoustics calibrator.pptx
Environmental Acoustics- Speech interference level, acoustics calibrator.pptxEnvironmental Acoustics- Speech interference level, acoustics calibrator.pptx
Environmental Acoustics- Speech interference level, acoustics calibrator.pptxpriyankatabhane
 
Efficient Fourier Pricing of Multi-Asset Options: Quasi-Monte Carlo & Domain ...
Efficient Fourier Pricing of Multi-Asset Options: Quasi-Monte Carlo & Domain ...Efficient Fourier Pricing of Multi-Asset Options: Quasi-Monte Carlo & Domain ...
Efficient Fourier Pricing of Multi-Asset Options: Quasi-Monte Carlo & Domain ...Chiheb Ben Hammouda
 
𝗧𝗖𝗢 (𝙩𝙧𝙖𝙣𝙨-𝗰𝘆𝗰𝗹𝗼𝗼𝗰𝘁𝗲𝗻𝗲) 𝗗𝗲𝗿𝗶𝘃𝗮𝘁𝗶𝘃𝗲𝘀: 𝗧𝗵𝗲 𝗙𝗮𝘀𝘁𝗲𝘀𝘁 𝗖𝗹𝗶𝗰𝗸 𝗥𝗲𝗮𝗰𝘁𝗶𝗼𝗻 𝗥𝗲𝗮𝗴𝗲𝗻𝘁𝘀
𝗧𝗖𝗢 (𝙩𝙧𝙖𝙣𝙨-𝗰𝘆𝗰𝗹𝗼𝗼𝗰𝘁𝗲𝗻𝗲) 𝗗𝗲𝗿𝗶𝘃𝗮𝘁𝗶𝘃𝗲𝘀: 𝗧𝗵𝗲 𝗙𝗮𝘀𝘁𝗲𝘀𝘁 𝗖𝗹𝗶𝗰𝗸 𝗥𝗲𝗮𝗰𝘁𝗶𝗼𝗻 𝗥𝗲𝗮𝗴𝗲𝗻𝘁𝘀𝗧𝗖𝗢 (𝙩𝙧𝙖𝙣𝙨-𝗰𝘆𝗰𝗹𝗼𝗼𝗰𝘁𝗲𝗻𝗲) 𝗗𝗲𝗿𝗶𝘃𝗮𝘁𝗶𝘃𝗲𝘀: 𝗧𝗵𝗲 𝗙𝗮𝘀𝘁𝗲𝘀𝘁 𝗖𝗹𝗶𝗰𝗸 𝗥𝗲𝗮𝗰𝘁𝗶𝗼𝗻 𝗥𝗲𝗮𝗴𝗲𝗻𝘁𝘀
𝗧𝗖𝗢 (𝙩𝙧𝙖𝙣𝙨-𝗰𝘆𝗰𝗹𝗼𝗼𝗰𝘁𝗲𝗻𝗲) 𝗗𝗲𝗿𝗶𝘃𝗮𝘁𝗶𝘃𝗲𝘀: 𝗧𝗵𝗲 𝗙𝗮𝘀𝘁𝗲𝘀𝘁 𝗖𝗹𝗶𝗰𝗸 𝗥𝗲𝗮𝗰𝘁𝗶𝗼𝗻 𝗥𝗲𝗮𝗴𝗲𝗻𝘁𝘀Tokyo Chemicals Industry (TCI)
 
Role of Gibberellins, mode of action and external applications.pptx
Role of Gibberellins, mode of action and external applications.pptxRole of Gibberellins, mode of action and external applications.pptx
Role of Gibberellins, mode of action and external applications.pptxjana861314
 
Rabies ,a deadly viral disease transmitted by the most beautiful beings. by ...
Rabies ,a deadly viral disease transmitted by the most beautiful beings.  by ...Rabies ,a deadly viral disease transmitted by the most beautiful beings.  by ...
Rabies ,a deadly viral disease transmitted by the most beautiful beings. by ...uzmashireenmbe01
 

Recently uploaded (20)

ESSENTIAL FEATURES REQUIRED FOR ESTABLISHING FOUR TYPES OF BIOSAFETY LABORATO...
ESSENTIAL FEATURES REQUIRED FOR ESTABLISHING FOUR TYPES OF BIOSAFETY LABORATO...ESSENTIAL FEATURES REQUIRED FOR ESTABLISHING FOUR TYPES OF BIOSAFETY LABORATO...
ESSENTIAL FEATURES REQUIRED FOR ESTABLISHING FOUR TYPES OF BIOSAFETY LABORATO...
 
BACTERIAL SECRETION SYSTEM by Dr. Chayanika Das
BACTERIAL SECRETION SYSTEM by Dr. Chayanika DasBACTERIAL SECRETION SYSTEM by Dr. Chayanika Das
BACTERIAL SECRETION SYSTEM by Dr. Chayanika Das
 
Presentation about adversarial image attacks
Presentation about adversarial image attacksPresentation about adversarial image attacks
Presentation about adversarial image attacks
 
DETECTION OF MUTATION BY CLB METHOD.pptx
DETECTION OF MUTATION BY CLB METHOD.pptxDETECTION OF MUTATION BY CLB METHOD.pptx
DETECTION OF MUTATION BY CLB METHOD.pptx
 
Science (Communication) and Wikipedia - Potentials and Pitfalls
Science (Communication) and Wikipedia - Potentials and PitfallsScience (Communication) and Wikipedia - Potentials and Pitfalls
Science (Communication) and Wikipedia - Potentials and Pitfalls
 
Introduction of Organ-On-A-Chip - Creative Biolabs
Introduction of Organ-On-A-Chip - Creative BiolabsIntroduction of Organ-On-A-Chip - Creative Biolabs
Introduction of Organ-On-A-Chip - Creative Biolabs
 
Remote patient monitoring :Health care transformation
Remote patient monitoring :Health care transformationRemote patient monitoring :Health care transformation
Remote patient monitoring :Health care transformation
 
DNA isolation molecular biology practical.pptx
DNA isolation molecular biology practical.pptxDNA isolation molecular biology practical.pptx
DNA isolation molecular biology practical.pptx
 
STELLAR SYSTEM IN PTERIDOPHYTE Seminar 2023- By Karishma
STELLAR SYSTEM IN PTERIDOPHYTE Seminar 2023- By KarishmaSTELLAR SYSTEM IN PTERIDOPHYTE Seminar 2023- By Karishma
STELLAR SYSTEM IN PTERIDOPHYTE Seminar 2023- By Karishma
 
Observation of Gravitational Waves from the Coalescence of a 2.5–4.5 M⊙ Compa...
Observation of Gravitational Waves from the Coalescence of a 2.5–4.5 M⊙ Compa...Observation of Gravitational Waves from the Coalescence of a 2.5–4.5 M⊙ Compa...
Observation of Gravitational Waves from the Coalescence of a 2.5–4.5 M⊙ Compa...
 
EGYPTIAN IMPRINT IN SPAIN Lecture by Dr Abeer Zahana
EGYPTIAN IMPRINT IN SPAIN Lecture by Dr Abeer ZahanaEGYPTIAN IMPRINT IN SPAIN Lecture by Dr Abeer Zahana
EGYPTIAN IMPRINT IN SPAIN Lecture by Dr Abeer Zahana
 
Food_safety_Management_pptx.pptx in microbiology
Food_safety_Management_pptx.pptx in microbiologyFood_safety_Management_pptx.pptx in microbiology
Food_safety_Management_pptx.pptx in microbiology
 
Pests of Sunflower_Binomics_Identification_Dr.UPR
Pests of Sunflower_Binomics_Identification_Dr.UPRPests of Sunflower_Binomics_Identification_Dr.UPR
Pests of Sunflower_Binomics_Identification_Dr.UPR
 
Environmental acoustics- noise criteria.pptx
Environmental acoustics- noise criteria.pptxEnvironmental acoustics- noise criteria.pptx
Environmental acoustics- noise criteria.pptx
 
Bioenergetics and the role of ATP to drive the beats of life.
Bioenergetics and the role of ATP to drive the beats of life.Bioenergetics and the role of ATP to drive the beats of life.
Bioenergetics and the role of ATP to drive the beats of life.
 
Environmental Acoustics- Speech interference level, acoustics calibrator.pptx
Environmental Acoustics- Speech interference level, acoustics calibrator.pptxEnvironmental Acoustics- Speech interference level, acoustics calibrator.pptx
Environmental Acoustics- Speech interference level, acoustics calibrator.pptx
 
Efficient Fourier Pricing of Multi-Asset Options: Quasi-Monte Carlo & Domain ...
Efficient Fourier Pricing of Multi-Asset Options: Quasi-Monte Carlo & Domain ...Efficient Fourier Pricing of Multi-Asset Options: Quasi-Monte Carlo & Domain ...
Efficient Fourier Pricing of Multi-Asset Options: Quasi-Monte Carlo & Domain ...
 
𝗧𝗖𝗢 (𝙩𝙧𝙖𝙣𝙨-𝗰𝘆𝗰𝗹𝗼𝗼𝗰𝘁𝗲𝗻𝗲) 𝗗𝗲𝗿𝗶𝘃𝗮𝘁𝗶𝘃𝗲𝘀: 𝗧𝗵𝗲 𝗙𝗮𝘀𝘁𝗲𝘀𝘁 𝗖𝗹𝗶𝗰𝗸 𝗥𝗲𝗮𝗰𝘁𝗶𝗼𝗻 𝗥𝗲𝗮𝗴𝗲𝗻𝘁𝘀
𝗧𝗖𝗢 (𝙩𝙧𝙖𝙣𝙨-𝗰𝘆𝗰𝗹𝗼𝗼𝗰𝘁𝗲𝗻𝗲) 𝗗𝗲𝗿𝗶𝘃𝗮𝘁𝗶𝘃𝗲𝘀: 𝗧𝗵𝗲 𝗙𝗮𝘀𝘁𝗲𝘀𝘁 𝗖𝗹𝗶𝗰𝗸 𝗥𝗲𝗮𝗰𝘁𝗶𝗼𝗻 𝗥𝗲𝗮𝗴𝗲𝗻𝘁𝘀𝗧𝗖𝗢 (𝙩𝙧𝙖𝙣𝙨-𝗰𝘆𝗰𝗹𝗼𝗼𝗰𝘁𝗲𝗻𝗲) 𝗗𝗲𝗿𝗶𝘃𝗮𝘁𝗶𝘃𝗲𝘀: 𝗧𝗵𝗲 𝗙𝗮𝘀𝘁𝗲𝘀𝘁 𝗖𝗹𝗶𝗰𝗸 𝗥𝗲𝗮𝗰𝘁𝗶𝗼𝗻 𝗥𝗲𝗮𝗴𝗲𝗻𝘁𝘀
𝗧𝗖𝗢 (𝙩𝙧𝙖𝙣𝙨-𝗰𝘆𝗰𝗹𝗼𝗼𝗰𝘁𝗲𝗻𝗲) 𝗗𝗲𝗿𝗶𝘃𝗮𝘁𝗶𝘃𝗲𝘀: 𝗧𝗵𝗲 𝗙𝗮𝘀𝘁𝗲𝘀𝘁 𝗖𝗹𝗶𝗰𝗸 𝗥𝗲𝗮𝗰𝘁𝗶𝗼𝗻 𝗥𝗲𝗮𝗴𝗲𝗻𝘁𝘀
 
Role of Gibberellins, mode of action and external applications.pptx
Role of Gibberellins, mode of action and external applications.pptxRole of Gibberellins, mode of action and external applications.pptx
Role of Gibberellins, mode of action and external applications.pptx
 
Rabies ,a deadly viral disease transmitted by the most beautiful beings. by ...
Rabies ,a deadly viral disease transmitted by the most beautiful beings.  by ...Rabies ,a deadly viral disease transmitted by the most beautiful beings.  by ...
Rabies ,a deadly viral disease transmitted by the most beautiful beings. by ...
 

Optimization of Text Input

  • 1. Optimization ofText Input Anna Feit, Doctoral student, Aalto University Dagstuhl Seminar on Computational Interactivity, 06.06.2017
  • 2.
  • 3. … Mid-air hand gestures for text input
  • 4. A B C … Z space ? … 27 letters32 gestures 1033 mappings Which hand gesture to use for which letter?
  • 5. Decisions: For each letter and gesture, assign the letter to the gesture or not. Constraints: No letter / gesture is assigned more than once. Evaluation criteria: Evaluate each design and pick the best one. 1033 feasible designs
  • 6. Mathematical or algorithmic method to find the best design in the space • Search very large design spaces • Efficient and rigorous process • Quantitative guarantees on the goodness of the outcome • Explicitly trade-off different criteria and constraints Optimization
  • 7. Challenges Formulation Objectives Optimization Formulation of the design problem and space, identification of design variables and constraints argmin 𝐾 ෍ 𝑘𝜖𝐾 ෍ 𝑙𝜖𝐾 𝐶 𝐾(𝑘, 𝑙) Modeling of evaluation criteria and combination into a fast to compute objective function (interaction cost) Mathematical solver or approximation algorithm to efficiently and thoroughly search the design space Objective 1 Objective2
  • 8. The (quadratic) letter assignment problem Given: n letters – 𝑖, 𝑗 𝜖 𝛴 m gestures – 𝑘, 𝑙 𝜖 𝐺 Let: 𝑥𝑖𝑘 = 1 if letter 𝑖 is assigned to gesture 𝑘, 𝑥𝑖𝑘 = 0 otherwise 𝐗 = {𝑥𝑖𝑘 | ∀ 𝑖 𝜖 𝛴, 𝑘 𝜖 𝐺, 𝑥𝑖𝑘 𝜖 0,1 } characterises the full design space [Burkhard, 1977] Formulation
  • 9. Goal: Find the assignment of letters to gestures that minimize the cost of typing one character after another Formulation
  • 10. [Zhai, Hunter & Smith, 2000] [Light & Anderson, 1993] Formulation [Oulasvirta & Karrenbauer, 2014]
  • 11. ” It is almost impossible to write correctly French with a keyboard marketed in France” French Ministry of Culture and Communications [Feit, Nancel,Weir, John, Bailly, Karrenbauer, Oulasvirta, upcoming] Formulation
  • 12. [Feit, Nancel,Weir, John, Bailly, Karrenbauer, Oulasvirta, upcoming] Formulation é è à ù ê Ê É È À ç Ç æ Æ œ Œ ß ẞ þ Þ ð Ð ŋ Ŋ ij IJ ə Ə ʒ Ʒ & θ ı İ @ ™ ® © ſ º ª · ´ ˋ ˆ ¨ ˉ ̲ ˘ ̑ ˇ ˜ ˙ ̣ ̊ ˝ ˵ ¸ ˛ ̦ ̵ ̷ + < > = ± × ÷ ≤ ≥ ≃ % ‰ √ ∞ ¼ ½ ¾ # / | . , ; : ! ? ¡ ¿ … - - — – _ * † ‡ § ( ) [ ] { } “ ” ‘ ’ « » ‚ „ ‹ › € $ £ ¢ ¤ ¥ ₩ ?? ? > 𝟏𝟎 𝟐𝟏𝟑 𝒂𝒔𝒔𝒊𝒈𝒏𝒎𝒆𝒏𝒕𝒔
  • 13. [Feit, Nancel,Weir, John, Bailly, Karrenbauer, Oulasvirta, upcoming] Formulation è
  • 14. Objectives • Performance – Fitts’ law weighted by letter pair frequency • QWERTY similarity [Dunlop & Levine 2012] • Word or gesture clarity [Dunlop & Levine 2012, Smith, Bi & Zhai 2015]
  • 15. Objectives Text entry is a complex task involving cognitive and motor processes. Fast performance involves more than quickly pointing from one key to another • Different performance factors • Different tasks • Different skill levels • Different strategies
  • 16. Objectives [How we type, Feit, Weir, Oulasvirta, CHI 2016]
  • 17. Objectives … |C6| = 0.38 Middle vs. Ring, participant 2046 Non-instructed:Ring Instructed: Middle Gesture performance models based on Fitts’ law and theories of motor control Anatomical comfort: Individuation index for each finger [Investigating the Dexterity of Multi-Finger Input for Mid-AirText Entry, Sridhar, Feit,Theobalt, Oulasvirta, CHI 2015]
  • 18. Objectives [Feit, Nancel,Weir, John, Bailly, Karrenbauer, Oulasvirta, upcoming] Standardization committee: ”The new keyboard should facilitate typing of correct french, should be easy to learn and intuitive to use” • Performance and ergonomics of typing a special character before or after a letter • Intuitive and easy to learn: • Grouping similar characters • Position similar to QWERTY • Language statistics take into account different typing tasks, e.g. programming, social media usage, formal writing, etc.
  • 19. Optimization Mathematical, exact methods Linear or Integer Programming, Branch and Bound methods Pro: • Explicit bounds and guarantees on optimality • Fast solvers available, e.g. Gurobi, CPLEX (IBM) Con: • Objective function in closed mathematical form • Not so flexible (e.g. noisy input data, interactive optimization, multi-objectives, etc. ) Heuristic approximation algorithms • Simulated annealing, Genetic algorithms, Biology inspired algorithms etc. Pro: • Straightforward to implement and standard implementations available • Flexible, e.g. combine with simulation models Con: • No bounds or guarantee to find the global optimum • Potentially slow • Formulation of design space and constraints
  • 20. Optimization [Feit, Nancel,Weir, John, Bailly, Karrenbauer, Oulasvirta, upcoming] Mathematical solver: Gurobi • Guarantees to cover the full design space • Gives explicit bounds • Nevertheless: cannot solve to the global optimum Challenge: integrate optimization with stakeholders’ opinions
  • 21. Challenges • Multi-objective optimization: weighted sum versus pareto optimization • No ”one size fits all”: trade-off different tasks, skills, strategies, etc. or optimally adapt • Models and input data: efficient, mathematical models, noisy data
  • 23. www.annafeit.de @AnnaFeit Anna Feit Doctoral student, finishing end 2017 Research topics: • Text entry • UI Optimization I also know a bit about: • User modeling • Mid-air input • Eye tracking