Antti Oulasvirta, Teemu Roos,
Arttu Modig, Laura Leppänen
Information Capacity of Full-body Movements

Information Capacity of Full-body Movements
Aimed movements are common
motor responses in HCI

Information capacity is measured in
repeated aimed movements
W W
D
[Fitts 54 JEP, Soukoreff & MacKenzie 2004 HCI]
ID
Information Capacity of Full-body Movements

W W
i ii iii iv v vi vii viii ix x
D
Throughput (TP, bits/s) is the rate
with which a user could have sent messages
D
We We
i ii iii iv v vi
Effective width WeInformation Capacity of Full-body Movements [Fitts 54 JEP, Soukoreff & MacKenzie 2004 HCI]
TP = ID / MT = log2(1 + D/W) / MT

[Soukoreff & MacKenzie 2004 HCI]
TP is used for comparing input devices
Information Capacity of Full-body Movements

Fitts-TP
3-10 bps

Information Capacity of Full-body Movements
Limitations of the Fitts-TP
Single movement point
Only end point matters
Target areas fixed in the
environment

Information Capacity of Full-body Movements
Multiple movement points
Continuous movement
Shape of movement

Information Capacity of Full-body Movements
Information capacity is the ability to
produce complex movement at will
“ Since the measurable aspects of motor responses [...]
are continuous variables, their information capacity is
limited only by the amount of statistical variability, or
noise, that is characteristic of repeated efforts to produce
the same response. ”
Paul Fitts (1954)

Information Capacity of Full-body Movements
Challenges
What is complexity?
How to compute information capacity?
Match between two sequences?
How to decorrelate mutual dependencies?
How to capture full-body movement?

X
Movement sequence
Information Capacity of Full-body Movements

X Y
Movement sequence Repetition
Information Capacity of Full-body Movements

X Y
h(X) entropy of X
Information Capacity of Full-body Movements

X Y
h(Y) entropy ofY
Information Capacity of Full-body Movements

X Y
I(X;Y) Mutual information between X andY
I(X;Y) = h(X) – h(X|Y) = h(Y) – h(Y|X)
Information Capacity of Full-body Movements

Information Capacity of Full-body Movements
Computational pipeline
x"
y"
Autoregression+ rx"
ry"
Gaussian+process+
r’x"
r’y"
II Complexity estimation
rxp1 rxp2 rxp3 rxp4 rxp5 rxp6
ryp1 ryp2 ryp3 ryp4 ryp5 ryp6
TP
V Mutual informationIII Dimension reduction
ρyx"
Correla2ons+
I Capture
Canonical+2me+warping+
ix,y"
IV Temporal alignment
xt xt+1 xt+2 xt+3 xt+4 xt+5
εt
(x) εt
(x) εt
(x) εt
(x)
εt
(x) εt
(x)
εt
(y)
yt yt+1 yt+2 yt+3 yt+4 yt+5
εt
(y) εt
(y) εt
(y)
εt
(y) εt
(y)

Step 1: Performance in intended
repetitions is captured
[CMU Mocap DB]
X Y

Information Capacity of Full-body Movements
Step 2: Complexity estimation is done
with 2nd order autoregression
εt-1
y)
xt-1 xt xt+1 xt+2 xt+3 xt+4
yt-1 yt yt+1 yt+2 yt+3 yt+4
εt-1
(x) εt
(x) εt+1
(x) εt+2
(x)
εt+3
(x) εt+4
(x)
εt
(y) εt+1
(y) εt+2
(y)
εt+3
(y) εt+4
(y)
Residuals
X
Y

Information Capacity of Full-body Movements
Step 3: Dimensionality reduction is done
with PCA or GP-LVM
[Lawrence 05 JLMR]
GP-LVM manifold for two dances in the ballet
data (3 latent dimensions)
X Y

Information Capacity of Full-body Movements
Selection of dimensions
0.00
0.05
0.10
0.15
0.20
AverageRMSE ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
0
50
100
150
200
250
AverageThroughput(bps)
2 4 6 8 12 16 20
Latent Dimensions
●
RMSE
Throughput

Information Capacity of Full-body Movements
Step 4:Temporal alignment (optional)
X
Y
CanonicalTimeWarping CTW

Information Capacity of Full-body Movements
X
Y
Step 4:Temporal alignment (optional)
CanonicalTimeWarping CTW

Information Capacity of Full-body Movements CanonicalTimeWarping CTW
X
Y
Step 4:Temporal alignment (optional)

Information Capacity of Full-body Movements [Zhou & De La Torre 2009 NIPS]
Example results

Information Capacity of Full-body Movements
Step 5: Mutual information is calculated
from estimated correlation of residuals
[Kendall & Stuart 68]
Mutual information is determined by the correlation of residuals:
We estimate this and add a bias correction:
Throughput is now mutual information per second

Information Capacity of Full-body Movements
First feasibility tests
Standing still
0 bps
Balancing with one leg
0 bps
Rapid caging of the palm
289 bps
43 bps
without CTW
PhaseSpace full-body suit and glove

Information Capacity of Full-body Movements
Sensitivity to noise in recording
instrument
●
●
●
●
●
●
●
●
● ● ● ● ● ● ● ●
0 0.0005 0.0015 0.0025
02004006008001200
Noise Factor
Throughput(bps)
●
TP(1|2)
TP(2|1)
PCA-TP

Information Capacity of Full-body Movements
Study 1: Ballerina
21-33
12-15
17-18

Information Capacity of Full-body Movements
Unencumbered 4 kg additional weight
Study 2: Mouse
4 Fitts-bps 2 Fitts-bps

Information Capacity of Full-body Movements
0 kg
4 kg
Low ID High ID
38 bps
24 bps 37 bps
37 bps
Unencumbered 4 kg additional weight
umbered 4 kg additional weight

Information Capacity of Full-body Movements
Unencumbered 4 kg additional weight
High-ID
TPs decreased when an ISI of 1,000 ms
was imposed
Slow motion

Information Capacity of Full-body Movements
Study 3: Minority Report

Information Capacity of Full-body Movements
PCA-TP 78
PCA-TP 440

Information Capacity of Full-body Movements
Results replicate a known perceptual
distraction in bimanual motor control
313 bps
353 bps
289 bps
[Meschner et al. 01 Nature]
Sweet spot at ~60 cm

Information Capacity of Full-body Movements
Bonus study: Expert gamer
SpaceFortress
[Boot et al. 10 Acta Psychologica]
First trials
2 bps
21 bps
After 20 hours trials

Information Capacity of Full-body Movements
Fitts-TP
Aimed movements
This paper
Full-body movements
Information Distance Changes in motion direction
Noise Effective width Variability between repetitions
W W
D

Information Capacity of Full-body Movements
Solutions
☐✓
☐
☐
☐
✓
✓
✓
Step 4:Time warping
Step 2:Autoregression
Step 3: Dimension reduction
Step 5: Mutual information
☐✓ Step 1: Optical capture
What is movement complexity?
How to compute information capacity?
Match between two sequences?
How to decorrelate mutual dependencies?
Capturing full-body movement?

Information Capacity of Full-body Movements
• Analyze information capacity allowed by your design
• Compare designs
• Expose human factors
• Explore best potentials for UIs

Information Capacity of Full-body Movements
infocapacity.hiit.fi
antti.oulasvirta@mpii.de
teemu.roos@cs.helsinki.fi
Implementation for Kinect
Interactivity i401