[TITLE] Breaking the Boundaries of Human-in-the-Loop Optimization [ABSTRACT] Human-in-the-loop optimization (HILO) has emerged as a principled solution for parametric optimization, leveraging artificial intelligence techniques to generate designs while incorporating human evaluation of design qualities. While HILO has demonstrated success across various fields such as robotics, engineering, machine learning, and human-computer interaction, it still faces two significant limitations. First, it primarily focuses on single-objective tasks, leading to narrow application scope. Second, HILO usually starts with no prior information, resulting in low efficiency. This talk introduces a range of computational augmentations aimed at addressing these limitations. We explore advanced artificial intelligence methods to tackle multi-objective tasks and investigate various strategies for constructing prior models to enhance the efficiency of HILO. By showcasing HILO's capability in addressing real-world problems and discussing its role in the future, this talk hopes to encourage the audience to apply HILO to their own challenges and domains. [BIO] Yi-Chi Liao is a Ph.D. candidate researching in computational interaction at Aalto University, under the supervision of Prof. Antti Oulasvirta. His research primarily revolves around optimization-based interface design and user modeling using reinforcement learning. Yi-Chi's previous works were published at top-tier conferences in the human-computer interaction field, such as CHI and UIST. For more information about Yi-Chi, please visit his website at http://yichiliao.com.

1. Saarland University, July 6th, 2023
Breaking the Boundaries of
Human-in-the-Loop Optimization
Yi-Chi Liao. Ph.D. Candidate, UI Group @ Aalto University
Website: http://yichiliao.com | Twitter: @yichiliao | Email: yi-chi.liao@aalto.
f
1

2. • Ph.D. dissertation:
• Human-in-the-Loop Design Optimization
• Current research group:
• User Interfaces Group at Aalto University, led by Prof Antti Oulasvirta
• Research interests:
• Optimization (inference-free); human-in-the-loop optimization, simulation-based optimization
• User modelling via reinforcement learning
• My tools:
• Bayesian optimization, reinforcement learning, meta-learning, and physics simulation.
• Am joining in November!
Yi-Chi Liao
2

3. Three goals
• Introduce human-in-the-loop optimization (HILO)
• Introduce the recent advancements in HILO
• Turn you into a Bayesian optimization user
3

4. Let's play a game! Here's how it works:
You can choose any floating-point number between 0 and 1.2.
Each number corresponds to a specific amount of money in return.
The challenge is to find the number that yields the highest amount of money!
4

5. Design optimization
0 1
0
3
What is the best location to place the optical sensor?
x ~ [0,1]
y ~ [0,3]
[Optimal Sensor Position for a Computer Mouse, CHI ’20]
5

6. Design optimization
0 1
0
3
6

7. Design optimization
0 1
0
3
X := design space
x := a design instance
7
x ~ [0,1]
y ~ [0,3]

8. Design optimization
0 1
0
3
X := design space
x := a design instance
Optimal design
8

9. Design optimization
0 1
0
3
9
f is unknown function!
f is expensive to evaluate!

10. [Caeses (https://www.caeses.com/)]
Design optimization
[Human-in-the-loop optimization of exoskeleton assistance during walking, 2017]
10

11. Design space (X) Objective-function space (Y)
x1
x2
x3
x4
x5
y1
y2
y3
y4
y5
Black-box function,
y = f(x)
Design instance, x Objective function value, y
Design optimization
11

12. Manual
optimization
Human-in-the-loop optimization
(HILO)
Beyond current
HILO
12

13. Manual
optimization
Human-in-the-loop optimization
(HILO)
Beyond current
HILO
13

14. Manual optimization
Design and
prototype
Analysis User testing
14

15. Manual optimization
[ThirdHand: Wearing a Robotic Arm to Experience Rich Force Feedback, Liao et al., 2015]
What is the optimal torque value(s) that maximizes the gaming experience?
15

16. Manual optimization
What is the optimal way to split the strokes?
[EdgeVib: E
ff
ective Alphanumeric Character Output Using a Wrist-Worn Tactile Display, Liao et al., 2015]
16

17. Manual optimization
[Outside-In: Visualizing Out-of-Sight Regions-of-Interest in a 360 Video Using Spatial Picture-in-Picture Preview, Lin et al., 2017]
What is the optimal position, size, and tilt angle of the window?
17

18. Manual optimization
Design and
prototype
Analysis User testing
• Pros:
• It is easy and feasible
• Cons:
• It does not guarantee the result
• The developers may introduce bias
• Can not adapt instantly
18

19. “Don’t ever make the mistake [of thinking] that you can
design something better than what you get from ruthless
massively parallel trial-and-error with a feedback cycle.”
— Linus Torvalds
19

20. Manual
optimization
Human-in-the-loop optimization
(HILO)
Beyond current
HILO
20

21. Human-in-the-loop optimization (HILO)
Interactive system
paremeterized by x
Computational
optimizer
User
Design candidate,
x ∈ X
Objective-function
value, y ∈ Y
Interaction
Designer
Design space, X
Objective space, Y
Optimal design, x
21

22. But which optimizer?
22

23. Bayesian Optimization (BO)!
23

24. Will BO solve this?
24

25. Bayesian optimization —
1.Surrogate model of the target
problem: a Gaussian Process
regression
2.Acquisition function: a
function estimates the utility
of selecting a design choice
25

26. 26
1.Able to (roughly) predict the
outcome of a design choice
2.Able to identify the design
choice that may yield the most
improvements within a huge
design space
Bayesian optimization —
1.Surrogate model of the target
problem: a Gaussian Process
regression
2.Acquisition function: a
function estimates the utility
of selecting a design choice

27. 27
True function
Surrogate model

28. 28
Observations
(Evaluations)
Surrogate model

29. 29
Predicted mean value
Surrogate model
Predicted con
fi
dence interval
i.e., uncertainty

30. 30
Acquisition function
Acquisition value

31. 31

32. 32

33. 33

34. 34

35. 35

36. 36

37. Surrogate model: Gaussian Process
• GP is a generalized form of multivariate Gaussian (normal)
distribution.
• Covariance function (kernel):
• White noise kernel
• Exponentiated quadratic kernel
• Periodic kernel
37

38. Selection of kernels
Exponentiated quadratic kernel Periodic kernel
38

39. Acquisition function
• AF quantifies the utility, potential, or worth of different x points based
on the GP surrogate model.
• Computing AF is way cheaper than actual evaluation!
39

40. Acquisition function
• AF quantifies the utility, potential, or worth of different x points based
on the GP surrogate model.
• Computing AF is way cheaper than actual evaluation!
• Common acquisition functions:
• Expected Improvements (EI)
• Probability of Improvements (PI)
• Upper Confidence Bound (UCB)
40

41. Why is BO so good (mathematically)?
41
• BO is a general tool with the least assumption of the problem.
• BO is clearly explainable and transparent.
• BO is a computational framework allows for easy customization & extension.

42. Applications of Bayesian Optimization
42

43. 43
arxiv.org/abs/2303.14291
BO in engineering
BO in natural science
BO in social/
economical/political
decision-making

44. [Bayesian optimization explains human active search, Borji and Itti, 2013]
[Generalization guides human exploration in vast decision spaces, Wu et al., 2017]
44
BO is the best model/framework to explain human’s
parametric decision-making!

45. When Bayesian optimization performs poorly?
45
• GP will not work well when dealing with
non-continuous function
• More sample points will be needed in high-
dimensional (input/output) function
• Gets too expensive when handling with huge
amount of data: GP
fi
tting time ~ O(m^3)
where m is number of observations

46. Bayesian optimization for human-involved
interactive systems
[Human-in-the-loop optimization of exoskeleton assistance during walking, 2017]
46

47. [A Bayesian Interactive Optimization Approach to Procedural Animation Design, Siggraph SCA ’10]
47
Bayesian optimization for human-involved
interactive systems

48. 48
[Designing Engaging Games Using Bayesian Optimization, Khajah et al., 2016]
Bayesian optimization for human-involved
interactive systems

49. Interested in BO for HCI? Also check out Yuki’s works!!!
49
https://koyama.xyz/

50. Is HILO / BO perfect?
There are boundaries!
• HILO only addresses single-objective problems
• HILO reduces the designer’s agency and ownership in the process
• HILO only optimizes for one user/task at a time
• HILO only works for prototype-free interactions (software interfaces)
• HILO is still not efficient enough!
50

51. Manual
optimization
Human-in-the-loop optimization
(HILO)
Beyond current
HILO
51

52. From a single objective to multiple objectives
52

53. From a single objective to multiple objectives
53
Can we come up with a new
acquisition function?

54. From a single objective to multiple objectives
54
[Single- and Multiobjective Evolutionary Design Optimization Using
Gaussian Random Field Metamodels, Emmerich, 2005]
[Pareto frontier learning with expensive correlated objectives,
Shah & Ghahramani, 2016]

55. Short duration,
high amplitude
Long duration,
high amplitude
Long duration,
low amplitude
Short duration,
low amplitude
Short duration,
high amplitude
Long duration,
high amplitude
Long duration,
low amplitude
Short duration,
low amplitude
What is the optimal set of tacton (tactile icon)?
55
[Interaction Design With Multi-Objective Bayesian Optimization, Liao et al., 2023]

56. Short duration,
high amplitude
Long duration,
high amplitude
Long duration,
low amplitude
Short duration,
low amplitude
Short duration,
high amplitude
Long duration,
high amplitude
Long duration,
low amplitude
Short duration,
low amplitude
What is the optimal set of tacton (tactile icon)?
Length indicates duration
Set 1
56
[Interaction Design With Multi-Objective Bayesian Optimization, Liao et al., 2023]

57. Short duration,
high amplitude
Long duration,
high amplitude
Long duration,
low amplitude
Short duration,
low amplitude
Short duration,
high amplitude
Long duration,
high amplitude
Long duration,
low amplitude
Short duration,
low amplitude
What is the optimal set of tacton (tactile icon)?
Length indicates duration
Color indicates vibration amplitude
Set 1
Set 2
57
[Interaction Design With Multi-Objective Bayesian Optimization, Liao et al., 2023]

58. Short duration,
high amplitude
Long duration,
high amplitude
Long duration,
low amplitude
Short duration,
low amplitude
Short duration,
high amplitude
Long duration,
high amplitude
Long duration,
low amplitude
Short duration,
low amplitude
What is the optimal set of tacton (tactile icon)?
Length indicates duration
Color indicates vibration amplitude
Set 1
Set 2
58
Set 3
[Interaction Design With Multi-Objective Bayesian Optimization, Liao et al., 2023]

59. Short duration,
high amplitude
Long duration,
high amplitude
Long duration,
low amplitude
Short duration,
low amplitude
Short duration,
high amplitude
Long duration,
high amplitude
Long duration,
low amplitude
Short duration,
low amplitude
[Interaction Design With Multi-Objective Bayesian Optimization, Liao et al., 2023]
What are the design objectives?
59

60. a b
60
[Interaction Design With Multi-Objective Bayesian Optimization, Liao et al., 2023]

61. [Investigating Positive and Negative Qualities of Human-in-the-Loop Optimization for Designing Interaction Techniques, Chan et al., 2022]
61

62. [Investigating Positive and Negative Qualities of Human-in-the-Loop Optimization for Designing Interaction Techniques, Chan et al., 2022]
What is the optimal design?
Imagine there are just 10 discrete levels in each parameter,
the number of possible combinations is already 10^4!
62

63. [Investigating Positive and Negative Qualities of Human-in-the-Loop Optimization for Designing Interaction Techniques, Chan et al., 2022]
What are the design objectives?
63

64. [Investigating Positive and Negative Qualities of Human-in-the-Loop Optimization for Designing Interaction Techniques, Chan et al., 2022]
What are the design objectives?
E
ffi
ciency & accuracy
64

65. [Investigating Positive and Negative Qualities of Human-in-the-Loop Optimization for Designing Interaction Techniques, Chan et al., 2022]
65

66. [Investigating Positive and Negative Qualities of Human-in-the-Loop Optimization for Designing Interaction Techniques, Chan et al., 2022]
Does multi-objective BO really perform better
than human designers?
66

67. How do designers perceive BO as a tool?
[Interaction Design With Multi-Objective Bayesian Optimization, Liao et al., 2023]
67

68. [Investigating Positive and Negative Qualities of Human-in-the-Loop Optimization for Designing Interaction Techniques, Chan et al., 2022]
Shortcomings of HILO?
68

69. Collaborative Bayesian optimization
69
Human’s strength
Quickly determine whether a
design is good or bad
AI (BO)’s strength
More principled exploration
+

70. A cooperative system where designers can steer the
design when needed
70
[Cooperative Multi-Objective Bayesian Design Optimization, Mo et al., submitted to TiiS 2023]

71. A cooperative system where designers can steer the
design when needed
71
[Cooperative Multi-Objective Bayesian Design Optimization, Mo et al., submitted to TiiS 2023]
Results:
The Cooperative BO reaches
comparable user performances
with less full evaluations (more
e
ffi
ciently).
The designers felt more engaging
in the process than fully relying on
BO.

72. Boundaries of current HILO
• HILO only addresses single-objective problems
• HILO reduces the designer’s agency and ownership in the process
• HILO only optimizes for one user/task at a time
• HILO only works for prototype-free interactions (software interfaces)
• HILO is still not efficient enough!
72

73. Boundaries of current HILO
• HILO only addresses single-objective problems
• HILO reduces the designer’s agency and ownership in the process
• HILO only optimizes for one user/task at a time
• HILO only works for prototype-free interactions (software interfaces)
• HILO is still not efficient enough!
73

74. From a single user to a population of users
74
[Practical Approaches to Group-Level Multi-Objective Bayesian Optimization in Interaction Technique Design, Liao et al., submitted 2023]
Aggregate
Sample over
design space
Global Pareto-optimal
performances
y2
y1
Mean prediction of y1
Mean prediction of y2
x
y
P1
y
x
x
y
x
y
Global GP
y
x
P2
P3 P4

75. Boundaries of current HILO
• HILO only addresses single-objective problems
• HILO reduces the designer’s agency and ownership in the process
• HILO only optimizes for one user/task at a time
• HILO only works for prototype-free interactions (software interfaces)
• HILO is still not efficient enough!
75

76. From prototyping to physical emulation
76
[Button Simulation and Design via FDVV Models, Liao et al., 2020]

77. From prototyping to physical emulation
77
[Button Simulation and Design via FDVV Models, Liao et al., 2020]

78. Boundaries of current HILO
• HILO only addresses single-objective problems
• HILO reduces the designer’s agency and ownership in the process
• HILO only optimizes for one user/task at a time
• HILO only works for prototype-free interactions (software interfaces)
• HILO is still not efficient enough!
78

79. Time-efficiency issue
• Bayesian optimization is mainly seen as a design tool. That is, it is
used to derive the optimal design parameter prior to deploying the
interface/interaction on end-users.
79

80. Time-efficiency issue
• Bayesian optimization is mainly seen as a design tool. That is, it is
used to derive the optimal design parameter prior to deploying the
interface/interaction on end-users.
• Why?
• It takes ~ 10 x “NO of design parameters” iterations to converge.
This takes 40 - 60 mins
80

81. But why? Isn’t BO the best optimization tool?
• Yes! It is the upper bound of optimization methods!
• But it suffers from the cold-start problem
81

82. Can we boost the efficiency of BO?
82

83. Can we boost the efficiency of BO?
Yes! By augmenting BO with prior information!
83

84. 84
[Practical Approaches to Group-Level Multi-Objective Bayesian Optimization in Interaction Technique Design, Liao et al., submitted 2023]
Extract
representative
points
x
y
All observations across users
Initialize
BO
Mean prediction of y1
Mean prediction of y2
Observed y1
Observed y2
x
y
Warm-Start GP BO with a new user
x
y
New samples
Warm-start Bayesian optimization

85. 85
Current HILO

86. 86
Current HILO
Single-
objective
a b
Pareto frontier
learning

87. 87
Current HILO
Single-
objective Lack agency
Cooperative
AI
Pareto frontier
learning

88. 88
Current HILO
Single-
objective Lack agency
Cooperative
AI
Only for
1 user/task
Population
model
Aggregate
Sam
desig
Mean prediction of y1
Mean prediction of y2
x
y
P1
y
x
x
y
x
y
Global GP
y
x
P2
P3 P4
Pareto frontier
learning

89. 89
Current HILO
Single-
objective Lack agency
Cooperative
AI
Only for
1 user/task
Population
model
Only GUI
Physical
emulation
Pareto frontier
learning

90. 90
Current HILO
Single-
objective
Pareto frontier
learning
Lack agency
Cooperative
AI
Only for
1 user/task
Population
model
Only GUI
Physical
emulation
Low e
ffi
ciency
Constructing
prior
Extract
representative
points
x
y
All observations across users
Initialize
BO
Mean prediction of y1
Mean prediction of y2
Observed y1
Observed y2
x
y
Warm-Start GP BO with a new user
x
y
New samples

91. My advisor -
Prof. Antti Oulasvirta
Prof. Sunjun Kim
Dr. Kashyap Todi Dr. John Dudley Prof. Per Ola Kristensson Prof. Byungjoo Lee
Prof. Liwei Chan Dr. Aditya Acharya Antti Keurulainen Prof. Andrew Howes
Dr. Aakar Gupta Dr. Ruta Desai
George B. Mo
Dr. Tanya Jonker Dr. Hrvoje Benko Alec Pierce
Aini Putkonen Dr. Hee-Seung Moon Aleksi Ikkala
Amazing people who
made my Ph.D. studies
a wonderful journey!
91

92. 92
How to get Yi-Chi
http://yichiliao.com
Twitter: @yichiliao
Breaking the Boundaries of Human-in-the-Loop Optimization