Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
SICSA Summer School
on Computational Interaction
Letter Assignment
Anna Feit
23.06.2015
The Assignment Problem
Keyboard Optimization
Heuristic Approaches
Competition
Integer Programming
Overview
The Assignment
Problem
On secret agents and the choice of the right weapon …
Assignment Problem
Anna Feit – SICSA Summer School on Computational Interaction - 4
[Image: www.classtools.net]
[Image: pa...
Assignment Problem
p11 p12 p13 p14
p21 p22 p23 p24
p31 p32 p33 p34
p41 p42 p43 p44
Assign each
task to exactly
one agent s...
How to model the problem?
One variable xij for each Agent-Task pair:
xij = 1 if Agent i is assigned to Task j.
x11=1
Agent...
How to model the problem?
x11=1
x12=0
x13=0
x14=0
Profit for Agent 1:
Agent 1 is assigned to task 1
Anna Feit – SICSA Summ...
How to model the problem?
Objective function:
Maximize the profit for all agents
Anna Feit – SICSA Summer School on Comput...
How to model the problem?
Objective function:
Maximize the profit for all agents
Constraints:
•  Each task j must be assig...
How to model the problem?
Objective function:
Maximize the profit for all agents
Constraints:
•  Each task j must be assig...
Mathematical formulation
Easy problem
Linear assignment problem: only linear terms in the
objective function.
Anna Feit – ...
Application to real world problems
•  Send taxis to customers:
Profit = route price – cost to reach customer
•  Assign lec...
Generalized Assignment problem
cij
pij
i j
ci1
ci2
ci3
Assigning a task i to an agent j
gives a profit pij but has a cost ...
Mathematical formulation
Assignment problem:
Generalized
Assignment Problem:
Maximize profit
The sum of task costs
assigne...
Mathematical formulation
Assignment problem:
Generalized
Assignment Problem:
NP-hard problem
Anna Feit – SICSA Summer Scho...
Quadratic Assignment Problem
Relations within the two sets
Quadratic Assignment Problem
Flow between Factories
Anna Feit – SICSA Summer School on Computational Interaction - 17
Quadratic Assignment Problem
2
4
3
1
Distance between locations
Anna Feit – SICSA Summer School on Computational Interacti...
How to model this?
One variable xiv for each Factory-Location pair:
xiv = 1 if Factory i is assigned to Location v
1 2
x13...
Mathematical formulation
Linear
assignment problem:
Quadratic
assignment problem:
Assign each Factory to
exactly one Locat...
Mathematical formulation
Linear
assignment problem:
Quadratic
assignment problem:
NP-hard problem
Anna Feit – SICSA Summer...
More Variations
•  Stable Marriage Problem
–  Takes into account preferences
–  Application: Assign students to hospitals
...
Keyboard optimization
Challenging the Qwerty Keyboard
Task: Design a smartphone keyboard
that is optimized for performance
[Image: www.theinq
What is wrong with Qwerty?
The Letter Assignment Problem
ABCDEFGHIJKLMNOPQRSTUVWXYZ	
  
?!
Goal: Optimize performance
26 letters, 26 keys = 26!>  ​10...
How can we model this problem?
ABCDEFGHIJKLMNOPQRSTUVWXYZ	
  
?!
Anna Feit – SICSA Summer School on Computational Interact...
Letter Assignment Problem as QAP
Remember:
1 2
x13 x24
f12
d34
T H
14
7
pt,h
t14,7
Bigram probability
Pointing time
xt,14 ...
Letter Assignment Problem as QAP
T H
14
7
pt,h
t14,7
Bigram probability
Pointing time
xt,14 xh,7pi,j
Bigram frequency base...
Letter Assignment Problem as QAP
T H
14
7
pt,h
t14,7
Bigram probability
Pointing time
xt,14 xh,7
Each key assigned to exac...
Letter Assignment Problem as QAP
T H
14
7
pt,h
t14,7
Bigram probability
Pointing time
xt,14 xh,7
NP-hard problem
​ 𝟏 𝟎↑ 𝟐 ...
Heuristic approaches
Heuristic Approaches
•  Do not rely on mathematical formulation
•  Approximate the global optimum
Good choice to:
•  find ...
Heuristic Approaches
Here: two classes of heuristics:
•  Greedy / Constructive heuristics
•  Local search heuristics
Try t...
General Principle
Greedy approach
1.  Start with an empty
solution S = Ø.
2.  Iteratively add elements
to S that bring the...
Greedy Heuristics: Example
Heuristic: What element should be added next?
Minimize pointing time for frequent bigrams
1.  S...
Greedy Heuristics: Example
t h
th 1.52
he 1.28
in 0.94
er 0.94
an 0.82
re 0.68
nd 0.63
at 0.59
on 0.57
nt 0.56
…
Heuristic...
Greedy Heuristics: Example
t
th 1.52
he 1.28
in 0.94
er 0.94
an 0.82
re 0.68
nd 0.63
at 0.59
on 0.57
nt 0.56
…
h e
Heurist...
Greedy Heuristics: Example
t
th 1.52
he 1.28
in 0.94
er 0.94
an 0.82
re 0.68
nd 0.63
at 0.59
on 0.57
nt 0.56
…
h e i n
Heu...
Greedy Heuristics: Example
t
th 1.52
he 1.28
in 0.94
er 0.94
an 0.82
re 0.68
nd 0.63
at 0.59
on 0.57
nt 0.56
…
h e i n
Heu...
Local Search Heuristics
Heuristic: how to choose a neighbourhood
Anna Feit – SICSA Summer School on Computational Interact...
Local Search Heuristics
Heuristic: how to choose a neighbourhood
S
N(S)
Main components:
1.  Goal function f(S) to
evaluat...
Local Search Heuristics: Example
Keyboard Optimization
1.  Start from a random keyboard S
2.  Neighbourhood N(S):
All keyb...
Problem
[Source: http://bit.ly/1SvePF6]
The optimum of a neighborhood may not be the
global optimum
Neighborhood
Anna Feit...
Metaheuristics
•  High-level heuristic that tells you how to change the
local search heuristic
•  Strategy to overcome loc...
Tabu Search
Basic Principle:
1.  Store a tabu list of the last t solutions.
2.  Perform a local search where the solutions...
Tabu search
Main components:
•  How to define N(S) (Local search heuristic)
•  Length of the Tabu list
•  The stopping rul...
Simulated Annealing
Idea:
•  accept a worse solution from N(S) with a
probabilty 0 < p < 1
•  the worse the solution the s...
Simulated Annealing
Basic principle:
1.  Start with a random solution S
Init Temperature T and cooling ratio 0 < r < 1
2. ...
Simulated Annealing
Main components:
•  How to define N(S) (Local search heuristic)
•  Temperature T, cooling ration r, lo...
Genetic Algorithms
Inspired by the natural evolution of biological systems
Anna Feit – SICSA Summer School on Computationa...
Genetic Algorithms
Inspired by the natural evolution of biological systems
Idea:
•  Start with a set of solutions (= popul...
Genetic Algorithms
Basic principle:
1.  Start with a random set of solutions (population)
2.  Evaluate the fitness f(S) of...
Genetic Algorithms
Main components:
1.  Population size and termination criterium
2.  Fitness evaluation: f(S) similar to ...
A good heuristic ensures…
Communication: Definition of neighborhood should allow to
reach any optimal solution
Anna Feit –...
A good heuristic ensures…
Communication: Definition of neighborhood should allow to
reach any optimal solution
S
Anna Feit...
A good heuristic ensures…
Diversification:
e.g. through high initial temperature, large tabu list
S
Anna Feit – SICSA Summ...
A good heuristic ensures…
Intensification: e.g. use bigger N(S) if S is good
S
Anna Feit – SICSA Summer School on Computat...
Optimized keyboard
The metropolis keyboard [Zhai, Hunter and Smith, 2000]
Variation of simulated annealing
•  Goal functio...
Metropolis Keyboard
Diversification and Communication:
high temperature, keys move to
random directions
Intensification: L...
Optimizing for more than one Finger
The Kalq keyboard [Oulasvirta et al. CHI 2013]
Keyboard optimization for two-thumb tex...
The Kalq Keyboard
Combination of Local Search and Simulated annealing:
1.  Diversification:
Local optimization for 5000 ra...
Optimizing for multiple criteria
Multiobjective Keyboard Optimization
[Dunlop and Levine, 2012]
Multiple goal functions
• ...
Optimizing for multiple criteria
Multiobjective Keyboard Optimization
[Dunlop and Levine, CHI 2012]
Variant of Local searc...
Optimizing for multiple criteria
~37 wpm
(vs. 33 for Qwerty)
Anna Feit – SICSA Summer School on Computational Interaction ...
Summary
•  Heuristics approaches to find a good solution in
reasonable time
Ø Constructive vs. Local search
•  Metaheuris...
COMPETITION
Exact methods
Integer programming
Integer Programming (IP)
Linear Assignment problem
All decision variables are (binary) integers
Anna Feit – SICSA Summer S...
Integer Programming (IP)
Integer variables can be used to model
•  indivisibilities,
e.g. number of to-be-ordered products...
Integer Programming (IP)
Integer programming is about solving IP problems.
In comparison to Heuristic approaches they
•  i...
Linear vs. Integer programming
Example
Anna Feit – SICSA Summer School on Computational Interaction - 72
Linear vs. Integer programming
Linear program (LP)
Easy!
(1.95, 4.92)
Example
Anna Feit – SICSA Summer School on Computati...
Linear vs. Integer programming
Integer program (IP)
Hard!
(1.95, 4.92)
(5,0)
Example
Anna Feit – SICSA Summer School on Co...
How to solve an IP
Basic principle:
•  Find lower and upper bounds for the problem.
•  Iteratively compute tighter and tig...
Linear vs. Integer programming
(1.95, 4.92)Example
Upper bound:
Optimal LP solution
Anna Feit – SICSA Summer School on Com...
Linear vs. Integer programming
Example
Upper bound:
Optimal LP solution
Lower bound:
Any feasible integer solution
(1,1)
O...
Branch and Bound
Divide and Conquer approach
Basic idea:
•  Divide the problem into easier subproblems (Branch)
•  Solve s...
Example: Keyboard Optimization
[Karrenbauer and Oulasvirta, UIST 2014]
S
S1
S2
xa,1 = 1 xa,1 = 0
a a
Anna Feit – SICSA Sum...
Example: Keyboard Optimization
[Karrenbauer and Oulasvirta, UIST 2014]
S
S1
S2
xa,1 = 1 xa,1 = 0
a a
xb,1 = 1 xb,1 = 0
S11...
Example: Keyboard Optimization
[Karrenbauer and Oulasvirta, UIST 2014]
S
S1
S2
xa,1 = 1 xa,1 = 0
xb,1 = 1 xb,1 = 0
S11 S12...
Example: Keyboard Optimization
[Karrenbauer and Oulasvirta, UIST 2014]
S
S1
S2
xa,1 = 1 xa,1 = 0
xb,1 = 1 xb,1 = 0
S11 S12...
Example: Keyboard Optimization
[Karrenbauer and Oulasvirta, UIST 2014]
S
S1
S2
xa,1 = 1 xa,1 = 0
xb,1 = 1 xb,1 = 0
S11 S12...
Example: Keyboard Optimization
[Karrenbauer and Oulasvirta, 2014]
S
S1
S2
xa,1 = 1 xa,1 = 0
xb,1 = 1 xb,1 = 0
S11 S12
ub =...
IP solution to Keyboard Optimization
[Karrenbauer and Oulasvirta, UIST 2014]
42.65 wpm,
within 7% of optimum
Metropolis:
4...
Summary
•  Many Optimization methods applied in Keyboard
Optimization
•  Heuristic approaches: reasonable solution in fast...
Further readings
•  Dunlop M. and Levine J. (2012). Multidimensional pareto optimization of
touchscreen keyboards for spee...
Upcoming SlideShare
Loading in …5
×

Model-Based User Interface Optimization: Part II: LETTER ASSIGNMENT - At SICSA Summer School on Computational Interaction 2015

1,741 views

Published on

Tutorial on Model-Based User Interface Optimization. Part I: LETTER ASSIGNMENT by Anna Feit of Aalto University. Presented at SICSA Summer School on Computational Interaction in 2015 in Glasgow. Note: This one-day lecture is divided into multiple parts.

Published in: Design
  • Be the first to comment

  • Be the first to like this

Model-Based User Interface Optimization: Part II: LETTER ASSIGNMENT - At SICSA Summer School on Computational Interaction 2015

  1. 1. SICSA Summer School on Computational Interaction Letter Assignment Anna Feit 23.06.2015
  2. 2. The Assignment Problem Keyboard Optimization Heuristic Approaches Competition Integer Programming Overview
  3. 3. The Assignment Problem On secret agents and the choice of the right weapon …
  4. 4. Assignment Problem Anna Feit – SICSA Summer School on Computational Interaction - 4 [Image: www.classtools.net] [Image: paxsims.wordpress.com]
  5. 5. Assignment Problem p11 p12 p13 p14 p21 p22 p23 p24 p31 p32 p33 p34 p41 p42 p43 p44 Assign each task to exactly one agent such that we get most profit. Anna Feit – SICSA Summer School on Computational Interaction - 5
  6. 6. How to model the problem? One variable xij for each Agent-Task pair: xij = 1 if Agent i is assigned to Task j. x11=1 Agent 1 is assigned to task 1 x12=0 x13=0 x14=0 Anna Feit – SICSA Summer School on Computational Interaction - 6
  7. 7. How to model the problem? x11=1 x12=0 x13=0 x14=0 Profit for Agent 1: Agent 1 is assigned to task 1 Anna Feit – SICSA Summer School on Computational Interaction - 7
  8. 8. How to model the problem? Objective function: Maximize the profit for all agents Anna Feit – SICSA Summer School on Computational Interaction - 8
  9. 9. How to model the problem? Objective function: Maximize the profit for all agents Constraints: •  Each task j must be assigned to exactly one agent Anna Feit – SICSA Summer School on Computational Interaction - 9
  10. 10. How to model the problem? Objective function: Maximize the profit for all agents Constraints: •  Each task j must be assigned to exactly one agent •  Each agent i must be assigned to exactly one task Anna Feit – SICSA Summer School on Computational Interaction - 10
  11. 11. Mathematical formulation Easy problem Linear assignment problem: only linear terms in the objective function. Anna Feit – SICSA Summer School on Computational Interaction - 11
  12. 12. Application to real world problems •  Send taxis to customers: Profit = route price – cost to reach customer •  Assign lectures to rooms Cost = number of lectures that are assigned to the same room But: very limited problem description •  Same number of agents and tasks •  Assign only one task to each agent •  How about other criteria, e.g. preferences? Anna Feit – SICSA Summer School on Computational Interaction - 12
  13. 13. Generalized Assignment problem cij pij i j ci1 ci2 ci3 Assigning a task i to an agent j gives a profit pij but has a cost cij An agent can work on multiple tasks but has a work capacity tici1 ci2 ci3 ti Anna Feit – SICSA Summer School on Computational Interaction - 13
  14. 14. Mathematical formulation Assignment problem: Generalized Assignment Problem: Maximize profit The sum of task costs assigned to agent i can not be larger than his working capacity ti Assign each task to one agent Anna Feit – SICSA Summer School on Computational Interaction - 14
  15. 15. Mathematical formulation Assignment problem: Generalized Assignment Problem: NP-hard problem Anna Feit – SICSA Summer School on Computational Interaction - 15
  16. 16. Quadratic Assignment Problem Relations within the two sets
  17. 17. Quadratic Assignment Problem Flow between Factories Anna Feit – SICSA Summer School on Computational Interaction - 17
  18. 18. Quadratic Assignment Problem 2 4 3 1 Distance between locations Anna Feit – SICSA Summer School on Computational Interaction - 18
  19. 19. How to model this? One variable xiv for each Factory-Location pair: xiv = 1 if Factory i is assigned to Location v 1 2 x13 x24 f12 d34 Anna Feit – SICSA Summer School on Computational Interaction - 19
  20. 20. Mathematical formulation Linear assignment problem: Quadratic assignment problem: Assign each Factory to exactly one LocationMinimize the costs between locations and factories Anna Feit – SICSA Summer School on Computational Interaction - 20
  21. 21. Mathematical formulation Linear assignment problem: Quadratic assignment problem: NP-hard problem Anna Feit – SICSA Summer School on Computational Interaction - 21
  22. 22. More Variations •  Stable Marriage Problem –  Takes into account preferences –  Application: Assign students to hospitals •  Stable Roommates Problem –  No separation into two sets, assigns one element to any of the other elements •  Weapon-Target Assignment problem –  Model cooperation between agents –  Not all targets must be assigned to a weapon Anna Feit – SICSA Summer School on Computational Interaction - 22
  23. 23. Keyboard optimization Challenging the Qwerty Keyboard
  24. 24. Task: Design a smartphone keyboard that is optimized for performance [Image: www.theinq
  25. 25. What is wrong with Qwerty?
  26. 26. The Letter Assignment Problem ABCDEFGHIJKLMNOPQRSTUVWXYZ   ?! Goal: Optimize performance 26 letters, 26 keys = 26!>  ​10↑25  different mappings More than stars in the observable universe! Anna Feit – SICSA Summer School on Computational Interaction - 26
  27. 27. How can we model this problem? ABCDEFGHIJKLMNOPQRSTUVWXYZ   ?! Anna Feit – SICSA Summer School on Computational Interaction - 27
  28. 28. Letter Assignment Problem as QAP Remember: 1 2 x13 x24 f12 d34 T H 14 7 pt,h t14,7 Bigram probability Pointing time xt,14 xh,7 Anna Feit – SICSA Summer School on Computational Interaction - 28
  29. 29. Letter Assignment Problem as QAP T H 14 7 pt,h t14,7 Bigram probability Pointing time xt,14 xh,7pi,j Bigram frequency based on a language corpus tv,w Pointing time as given by Fitts’ Law. Anna Feit – SICSA Summer School on Computational Interaction - 29
  30. 30. Letter Assignment Problem as QAP T H 14 7 pt,h t14,7 Bigram probability Pointing time xt,14 xh,7 Each key assigned to exactly one letter Each letter assigned to exactly one key Anna Feit – SICSA Summer School on Computational Interaction - 30
  31. 31. Letter Assignment Problem as QAP T H 14 7 pt,h t14,7 Bigram probability Pointing time xt,14 xh,7 NP-hard problem ​ 𝟏 𝟎↑ 𝟐 𝟓  mappings Anna Feit – SICSA Summer School on Computational Interaction - 31
  32. 32. Heuristic approaches
  33. 33. Heuristic Approaches •  Do not rely on mathematical formulation •  Approximate the global optimum Good choice to: •  find some ”good” solution in a short time •  solve problems that are too complex to be solved mathematically Anna Feit – SICSA Summer School on Computational Interaction - 33
  34. 34. Heuristic Approaches Here: two classes of heuristics: •  Greedy / Constructive heuristics •  Local search heuristics Try to find a good solution within a given time, no guarantee on how good it is. Anna Feit – SICSA Summer School on Computational Interaction - 34
  35. 35. General Principle Greedy approach 1.  Start with an empty solution S = Ø. 2.  Iteratively add elements to S that bring the highest value. 3.  Stop when no more elements can be added that would improve S. Local search 1.  Start with a feasible solution S. 2.  Iteratively move to a better ”neighbour” by changing S. 3.  Stop if no better solution can be found in the neighborhood. Heuristics that tell you which element to add next Heuristics on how to construct a neighbour Anna Feit – SICSA Summer School on Computational Interaction - 35
  36. 36. Greedy Heuristics: Example Heuristic: What element should be added next? Minimize pointing time for frequent bigrams 1.  Start with the empty keyboard 2.  Take the next most frequent letter pair 3.  Assign the 2 letters such that the key distance is minimized 4.  Stop when all letters are assigned th 1.52 he 1.28 in 0.94 er 0.94 an 0.82 re 0.68 nd 0.63 at 0.59 on 0.57 nt 0.56 … Anna Feit – SICSA Summer School on Computational Interaction - 36
  37. 37. Greedy Heuristics: Example t h th 1.52 he 1.28 in 0.94 er 0.94 an 0.82 re 0.68 nd 0.63 at 0.59 on 0.57 nt 0.56 … Heuristic: What element should be added next? Minimize pointing time for frequent bigrams 1.  Start with the empty keyboard 2.  Take the next most frequent letter pair 3.  Assign the 2 letters such that the key distance is minimized 4.  Stop when all letters are assigned Anna Feit – SICSA Summer School on Computational Interaction - 37
  38. 38. Greedy Heuristics: Example t th 1.52 he 1.28 in 0.94 er 0.94 an 0.82 re 0.68 nd 0.63 at 0.59 on 0.57 nt 0.56 … h e Heuristic: What element should be added next? Minimize pointing time for frequent bigrams 1.  Start with the empty keyboard 2.  Take the next most frequent letter pair 3.  Assign the 2 letters such that the key distance is minimized 4.  Stop when all letters are assigned Anna Feit – SICSA Summer School on Computational Interaction - 38
  39. 39. Greedy Heuristics: Example t th 1.52 he 1.28 in 0.94 er 0.94 an 0.82 re 0.68 nd 0.63 at 0.59 on 0.57 nt 0.56 … h e i n Heuristic: What element should be added next? Minimize pointing time for frequent bigrams 1.  Start with the empty keyboard 2.  Take the next most frequent letter pair 3.  Assign the 2 letters such that the key distance is minimized 4.  Stop when all letters are assigned Anna Feit – SICSA Summer School on Computational Interaction - 39
  40. 40. Greedy Heuristics: Example t th 1.52 he 1.28 in 0.94 er 0.94 an 0.82 re 0.68 nd 0.63 at 0.59 on 0.57 nt 0.56 … h e i n Heuristic: What element should be added next? Minimize pointing time for frequent bigrams 1.  Start with the empty keyboard 2.  Take the next most frequent letter pair 3.  Assign the 2 letters such that the key distance is minimized 4.  Stop when all letters are assigned Anna Feit – SICSA Summer School on Computational Interaction - 40
  41. 41. Local Search Heuristics Heuristic: how to choose a neighbourhood Anna Feit – SICSA Summer School on Computational Interaction - 41
  42. 42. Local Search Heuristics Heuristic: how to choose a neighbourhood S N(S) Main components: 1.  Goal function f(S) to evaluate the solutions 2.  Rules how to modify S to obtain the neighbourhood N(S) Currently best: Note: f(S) should penalize infeasibility Anna Feit – SICSA Summer School on Computational Interaction - 42
  43. 43. Local Search Heuristics: Example Keyboard Optimization 1.  Start from a random keyboard S 2.  Neighbourhood N(S): All keyboards obtained from S by swapping 2 keys 3.  Evaluation of neighborhood: Compute the average pointing time of S, sum of Fitts’ pointing time weighted by bigram frequency 4.  Choose the best neighbour S* of S. If f(S*) < f(S): go to 2. Else: Stop, S is the local optimum. Anna Feit – SICSA Summer School on Computational Interaction - 43
  44. 44. Problem [Source: http://bit.ly/1SvePF6] The optimum of a neighborhood may not be the global optimum Neighborhood Anna Feit – SICSA Summer School on Computational Interaction - 44
  45. 45. Metaheuristics •  High-level heuristic that tells you how to change the local search heuristic •  Strategy to overcome local optima •  Naive Example: Multistart local search –  Local search heuristic: Obtain neighbordhood by swapping 2 keys of a keyboard –  Metaheuristic: If a local optimum is found, start over with a completely new randomized layout Anna Feit – SICSA Summer School on Computational Interaction - 45
  46. 46. Tabu Search Basic Principle: 1.  Store a tabu list of the last t solutions. 2.  Perform a local search where the solutions in the tabu list cannot be chosen. 3.  If a local minimum S is found proceed with the next best S* ∈ N(S) not in the tabu list and add S to the tabu list. 4.  At termination return the best solution found so far. N(S) S Tabu: N(S*) S Tabu: S* N(S)
  47. 47. Tabu search Main components: •  How to define N(S) (Local search heuristic) •  Length of the Tabu list •  The stopping rule. For example: –  Total number of iterations –  Number of iterations without improvement Anna Feit – SICSA Summer School on Computational Interaction - 47
  48. 48. Simulated Annealing Idea: •  accept a worse solution from N(S) with a probabilty 0 < p < 1 •  the worse the solution the smaller p •  p decreases at each iteration Inspired by metallurgy: Heat a metal to a temperature T and let it cool down slowly at rate r  to soften it.
  49. 49. Simulated Annealing Basic principle: 1.  Start with a random solution S Init Temperature T and cooling ratio 0 < r < 1 2.  For L iterations: a)  Choose a random solution S* from N(S) b)  Let d = f(S*) – f(S) •  If d<0: proceed with S* (S* is better) •  Else: proceed with S* with probability p=​ 𝑒↑− 𝑑/ 𝑇  3.  Reduce the temperature: T = rT and go to 2. 4.  At termination return the best solution found so far. Anna Feit – SICSA Summer School on Computational Interaction - 49
  50. 50. Simulated Annealing Main components: •  How to define N(S) (Local search heuristic) •  Temperature T, cooling ration r, loop length L •  The stopping rule. For example: –  Total number of iterations –  Number of iterations without improvement –  Small Temperature T < ε Anna Feit – SICSA Summer School on Computational Interaction - 50
  51. 51. Genetic Algorithms Inspired by the natural evolution of biological systems Anna Feit – SICSA Summer School on Computational Interaction - 51
  52. 52. Genetic Algorithms Inspired by the natural evolution of biological systems Idea: •  Start with a set of solutions (= population) •  At each iteration (= generation) create a new population by combining the best solutions Anna Feit – SICSA Summer School on Computational Interaction - 52
  53. 53. Genetic Algorithms Basic principle: 1.  Start with a random set of solutions (population) 2.  Evaluate the fitness f(S) of each solution 3.  Repeat until a new population is created: a)  Select a pair of parent solutions based on their fitness b)  With probability pc crossover the parents to form two offspring solutions. Otherwise create two offsprings as copies of the parents c)  With probability pm mutate the offsprings 4.  Repeat from step 2. At termination report the best solution found so far. Anna Feit – SICSA Summer School on Computational Interaction - 53
  54. 54. Genetic Algorithms Main components: 1.  Population size and termination criterium 2.  Fitness evaluation: f(S) similar to the goal function in local search (incl. penalty for infeasibility) 3.  Parent selection: probability proportional to its fitness. Variations: e.g. choose one parent randomly 4.  Crossover: implemented as string combinations. E.g. combine parents at random point xk in string: 5.  Mutation: random modifications, e.g. swap elements (also occurs in nature) Anna Feit – SICSA Summer School on Computational Interaction - 54
  55. 55. A good heuristic ensures… Communication: Definition of neighborhood should allow to reach any optimal solution Anna Feit – SICSA Summer School on Computational Interaction - 55
  56. 56. A good heuristic ensures… Communication: Definition of neighborhood should allow to reach any optimal solution S Anna Feit – SICSA Summer School on Computational Interaction - 56
  57. 57. A good heuristic ensures… Diversification: e.g. through high initial temperature, large tabu list S Anna Feit – SICSA Summer School on Computational Interaction - 57
  58. 58. A good heuristic ensures… Intensification: e.g. use bigger N(S) if S is good S Anna Feit – SICSA Summer School on Computational Interaction - 58
  59. 59. Optimized keyboard The metropolis keyboard [Zhai, Hunter and Smith, 2000] Variation of simulated annealing •  Goal function: Fitts’ pointing time •  Neighbour construction: No fixed layout, move a random key in a random direction by a random amount •  Several annealing and cooling iterations until ”a sufficiently efficient keyboard was found” e b c h t Anna Feit – SICSA Summer School on Computational Interaction - 59
  60. 60. Metropolis Keyboard Diversification and Communication: high temperature, keys move to random directions Intensification: Low temperature, movement toward optimal keyboard 43.1 wpm (vs. 36 for Qwerty) Anna Feit – SICSA Summer School on Computational Interaction - 60
  61. 61. Optimizing for more than one Finger The Kalq keyboard [Oulasvirta et al. CHI 2013] Keyboard optimization for two-thumb text entry Goal function: •  Adapted Fitts’ Law modeling performance with two thumbs •  Thumb alternation: + Preparation gain - Recover cost after long time Optimization procedure is the same, only goal function exchanged Anna Feit – SICSA Summer School on Computational Interaction - 61
  62. 62. The Kalq Keyboard Combination of Local Search and Simulated annealing: 1.  Diversification: Local optimization for 5000 random layouts 500 iterations 2.  Communication: Simulated annealing starting from the 100 best solutions, 3000 iterations 3.  Intensification: Local optimization for the 10 best layouts, 10.000 iterations 49 wpm (vs. 47 for split Qwerty) Anna Feit – SICSA Summer School on Computational Interaction - 62
  63. 63. Optimizing for multiple criteria Multiobjective Keyboard Optimization [Dunlop and Levine, 2012] Multiple goal functions •  Fitts’ pointing time •  Spell correction •  Qwerty similarity in vs. on if vs. of … Anna Feit – SICSA Summer School on Computational Interaction - 63
  64. 64. Optimizing for multiple criteria Multiobjective Keyboard Optimization [Dunlop and Levine, CHI 2012] Variant of Local search: Goal: find a set of optimal solutions that dominate each other in one of the 3 goal functions •  Start with locally optimized set of solutions •  Neighborhood: –  randomly swap two keys –  randomly swap extra keys with 25% probability •  Termination: 2000 iterations Anna Feit – SICSA Summer School on Computational Interaction - 64
  65. 65. Optimizing for multiple criteria ~37 wpm (vs. 33 for Qwerty) Anna Feit – SICSA Summer School on Computational Interaction - 65
  66. 66. Summary •  Heuristics approaches to find a good solution in reasonable time Ø Constructive vs. Local search •  Metaheuristics to overcome local minima Ø Tabu search Ø Simulated Annealing Ø Genetic algorithms •  Define main components: Ø Local search: Goal function, Neighborhood, Termination criterium, Ø Meta heuristics: length of tabu list, temperature, crossover Anna Feit – SICSA Summer School on Computational Interaction - 66
  67. 67. COMPETITION
  68. 68. Exact methods Integer programming
  69. 69. Integer Programming (IP) Linear Assignment problem All decision variables are (binary) integers Anna Feit – SICSA Summer School on Computational Interaction - 69
  70. 70. Integer Programming (IP) Integer variables can be used to model •  indivisibilities, e.g. number of to-be-ordered products •  on/off decisions (binary variables), e.g. hiring or assigning Anna Feit – SICSA Summer School on Computational Interaction - 70
  71. 71. Integer Programming (IP) Integer programming is about solving IP problems. In comparison to Heuristic approaches they •  increase the chance to find the global optimum •  are mathematically exact: •  Proven to find the global optimum or •  Give a guarantee on how good the solution is in comparison to the global optimum •  But: need a lot of time and computing power, efficient solution only for few ”easy” problems. Anna Feit – SICSA Summer School on Computational Interaction - 71
  72. 72. Linear vs. Integer programming Example Anna Feit – SICSA Summer School on Computational Interaction - 72
  73. 73. Linear vs. Integer programming Linear program (LP) Easy! (1.95, 4.92) Example Anna Feit – SICSA Summer School on Computational Interaction - 73
  74. 74. Linear vs. Integer programming Integer program (IP) Hard! (1.95, 4.92) (5,0) Example Anna Feit – SICSA Summer School on Computational Interaction - 74
  75. 75. How to solve an IP Basic principle: •  Find lower and upper bounds for the problem. •  Iteratively compute tighter and tigher bounds by –  Finding good but approximate solutions –  Solving easier problem –  Refining the problems Anna Feit – SICSA Summer School on Computational Interaction - 75
  76. 76. Linear vs. Integer programming (1.95, 4.92)Example Upper bound: Optimal LP solution Anna Feit – SICSA Summer School on Computational Interaction - 76
  77. 77. Linear vs. Integer programming Example Upper bound: Optimal LP solution Lower bound: Any feasible integer solution (1,1) Optimal value: Anna Feit – SICSA Summer School on Computational Interaction - 77
  78. 78. Branch and Bound Divide and Conquer approach Basic idea: •  Divide the problem into easier subproblems (Branch) •  Solve subproblems to obtain upper and lower bounds •  Use bounds to exclude subproblems that cannot lead to an optimal solution (Bound) Typically applied via an enumerative tree Anna Feit – SICSA Summer School on Computational Interaction - 78
  79. 79. Example: Keyboard Optimization [Karrenbauer and Oulasvirta, UIST 2014] S S1 S2 xa,1 = 1 xa,1 = 0 a a Anna Feit – SICSA Summer School on Computational Interaction - 79
  80. 80. Example: Keyboard Optimization [Karrenbauer and Oulasvirta, UIST 2014] S S1 S2 xa,1 = 1 xa,1 = 0 a a xb,1 = 1 xb,1 = 0 S11 S12 bb Anna Feit – SICSA Summer School on Computational Interaction - 80
  81. 81. Example: Keyboard Optimization [Karrenbauer and Oulasvirta, UIST 2014] S S1 S2 xa,1 = 1 xa,1 = 0 xb,1 = 1 xb,1 = 0 S11 S12 ub = 60 wpm lb = 30 wpm … L 30 wpm Anna Feit – SICSA Summer School on Computational Interaction - 81
  82. 82. Example: Keyboard Optimization [Karrenbauer and Oulasvirta, UIST 2014] S S1 S2 xa,1 = 1 xa,1 = 0 xb,1 = 1 xb,1 = 0 S11 S12 ub = 60 wpm lb = 30 wpm Solving the LP: upper bound = 28 wpm … L 30 wpm Best solution: 30 ≤ S ≤60 Anna Feit – SICSA Summer School on Computational Interaction - 82
  83. 83. Example: Keyboard Optimization [Karrenbauer and Oulasvirta, UIST 2014] S S1 S2 xa,1 = 1 xa,1 = 0 xb,1 = 1 xb,1 = 0 S11 S12 ub = 60 wpm lb = 30 wpm xb,1 = 1 S22 Solving the LP: ub= 58 wpm … L 30 wpm Anna Feit – SICSA Summer School on Computational Interaction - 83
  84. 84. Example: Keyboard Optimization [Karrenbauer and Oulasvirta, 2014] S S1 S2 xa,1 = 1 xa,1 = 0 xb,1 = 1 xb,1 = 0 S11 S12 ub = 58 wpm lb = 30 wpm xb,1 = 1 S22 Solving the LP: ub= 58 wpm … L 30 wpm Anna Feit – SICSA Summer School on Computational Interaction - 84
  85. 85. IP solution to Keyboard Optimization [Karrenbauer and Oulasvirta, UIST 2014] 42.65 wpm, within 7% of optimum Metropolis: 43.1 wpm Qwerty: 36 wpm Exact but slow: no optimal solution after 15h But guranteed to find the optimal solution after exponential time Anna Feit – SICSA Summer School on Computational Interaction - 85
  86. 86. Summary •  Many Optimization methods applied in Keyboard Optimization •  Heuristic approaches: reasonable solution in fast time •  Metaheuristics: non-zero probability to find the global optimum in infinite time •  Integer Programming: •  Slow and complex •  Guaranteed optimal solution after exponential time •  Bounds on how good the current best solution is
  87. 87. Further readings •  Dunlop M. and Levine J. (2012). Multidimensional pareto optimization of touchscreen keyboards for speed, familiarity and improved spell checking. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12). ACM •  Karrenbauer, A., and Oulasvirta, A. Improvements to keyboard optimization with integer programming. Proceedings of the 27th annual ACM symposium on User interface software and technology. ACM, 2014. •  Oulasvirta A. et al. (2013). Improving two-thumb text entry on touchscreen devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM •  Wolsey, L. A. (1998). Integer programming •  Zhai, S. , Hunter, M. and Smith B. A. (2000). The metropolis keyboard - an exploration of quantitative techniques for virtual keyboard design. In Proceedings of the 13th annual ACM symposium on User interface software and technology (UIST '00). ACM Anna Feit – SICSA Summer School on Computational Interaction - 87

×