Antti Oulasvirta, Teemu Roos,Arttu Modig, Laura LeppänenInformation Capacity of Full-body Movements
Information Capacity of Full-body MovementsAimed movements are commonmotor responses in HCI
Information capacity is measured inrepeated aimed movementsW WD[Fitts 54 JEP, Soukoreff & MacKenzie 2004 HCI]IDInformation...
W Wi ii iii iv v vi vii viii ix xDThroughput (TP, bits/s) is the ratewith which a user could have sent messagesDWe Wei ii ...
[Soukoreff & MacKenzie 2004 HCI]TP is used for comparing input devicesInformation Capacity of Full-body Movements
Fitts-TP3-10 bps
Information Capacity of Full-body MovementsLimitations of the Fitts-TPSingle movement pointOnly end point mattersTarget ar...
Information Capacity of Full-body MovementsMultiple movement pointsContinuous movementShape of movement
Information Capacity of Full-body MovementsInformation capacity is the ability toproduce complex movement at will“ Since t...
Information Capacity of Full-body MovementsChallengesWhat is complexity?How to compute information capacity?Match between ...
XMovement sequenceInformation Capacity of Full-body Movements
X YMovement sequence RepetitionInformation Capacity of Full-body Movements
X Yh(X) entropy of XInformation Capacity of Full-body Movements
X Yh(Y) entropy ofYInformation Capacity of Full-body Movements
X YI(X;Y) Mutual information between X andYI(X;Y) = h(X) – h(X|Y) = h(Y) – h(Y|X)Information Capacity of Full-body Movements
Information Capacity of Full-body MovementsComputational pipelinex"y"Autoregression+ rx"ry"Gaussian+process+r’x"r’y"II Com...
Step 1: Performance in intendedrepetitions is captured[CMU Mocap DB]X Y
Information Capacity of Full-body MovementsStep 2: Complexity estimation is donewith 2nd order autoregressionεt-1y)xt-1 xt...
Information Capacity of Full-body MovementsStep 3: Dimensionality reduction is donewith PCA or GP-LVM[Lawrence 05 JLMR]GP-...
Information Capacity of Full-body MovementsSelection of dimensions0.000.050.100.150.20AverageRMSE ●●●●●●●●●●●●●●●●●●●05010...
Information Capacity of Full-body MovementsStep 4:Temporal alignment (optional)XYCanonicalTimeWarping CTW
Information Capacity of Full-body MovementsXYStep 4:Temporal alignment (optional)CanonicalTimeWarping CTW
Information Capacity of Full-body Movements CanonicalTimeWarping CTWXYStep 4:Temporal alignment (optional)
Information Capacity of Full-body Movements [Zhou & De La Torre 2009 NIPS]Example results
Information Capacity of Full-body MovementsStep 5: Mutual information is calculatedfrom estimated correlation of residuals...
Information Capacity of Full-body MovementsFirst feasibility testsStanding still0 bpsBalancing with one leg0 bpsRapid cagi...
Information Capacity of Full-body MovementsSensitivity to noise in recordinginstrument●●●●●●●●● ● ● ● ● ● ● ●0 0.0005 0.00...
Information Capacity of Full-body MovementsStudy 1: Ballerina21-3312-1517-18
Information Capacity of Full-body MovementsUnencumbered 4 kg additional weightStudy 2: Mouse4 Fitts-bps 2 Fitts-bps
Information Capacity of Full-body Movements0 kg4 kgLow ID High ID38 bps24 bps 37 bps37 bpsUnencumbered 4 kg additional wei...
Information Capacity of Full-body MovementsUnencumbered 4 kg additional weightHigh-IDTPs decreased when an ISI of 1,000 ms...
Information Capacity of Full-body MovementsStudy 3: Minority Report
Information Capacity of Full-body MovementsPCA-TP 78PCA-TP 440
Information Capacity of Full-body MovementsResults replicate a known perceptualdistraction in bimanual motor control313 bp...
Information Capacity of Full-body MovementsBonus study: Expert gamerSpaceFortress[Boot et al. 10 Acta Psychologica]First t...
Information Capacity of Full-body MovementsFitts-TPAimed movementsThis paperFull-body movementsInformation Distance Change...
Information Capacity of Full-body MovementsSolutions☐✓☐☐☐✓✓✓Step 4:Time warpingStep 2:AutoregressionStep 3: Dimension redu...
Information Capacity of Full-body Movements• Analyze information capacity allowed by your design• Compare designs• Expose ...
Information Capacity of Full-body Movementsinfocapacity.hiit.fiantti.oulasvirta@mpii.deteemu.roos@cs.helsinki.fiImplementati...
Upcoming SlideShare
Loading in...5
×

Information Capacity of Full-body Movements (CHI'13)

373

Published on

Presentation at ACM CHI'13 by Antti Oulasvirta (Max Planck Institute for Informatics) and Teemu Roos(University of Helsinki).

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
373
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Information Capacity of Full-body Movements (CHI'13)

  1. 1. Antti Oulasvirta, Teemu Roos,Arttu Modig, Laura LeppänenInformation Capacity of Full-body Movements
  2. 2. Information Capacity of Full-body MovementsAimed movements are commonmotor responses in HCI
  3. 3. Information capacity is measured inrepeated aimed movementsW WD[Fitts 54 JEP, Soukoreff & MacKenzie 2004 HCI]IDInformation Capacity of Full-body Movements
  4. 4. W Wi ii iii iv v vi vii viii ix xDThroughput (TP, bits/s) is the ratewith which a user could have sent messagesDWe Wei ii iii iv v viEffective width WeInformation Capacity of Full-body Movements [Fitts 54 JEP, Soukoreff & MacKenzie 2004 HCI]TP = ID / MT = log2(1 + D/W) / MT
  5. 5. [Soukoreff & MacKenzie 2004 HCI]TP is used for comparing input devicesInformation Capacity of Full-body Movements
  6. 6. Fitts-TP3-10 bps
  7. 7. Information Capacity of Full-body MovementsLimitations of the Fitts-TPSingle movement pointOnly end point mattersTarget areas fixed in theenvironment
  8. 8. Information Capacity of Full-body MovementsMultiple movement pointsContinuous movementShape of movement
  9. 9. Information Capacity of Full-body MovementsInformation capacity is the ability toproduce complex movement at will“ Since the measurable aspects of motor responses [...]are continuous variables, their information capacity islimited only by the amount of statistical variability, ornoise, that is characteristic of repeated efforts to producethe same response. ”Paul Fitts (1954)
  10. 10. Information Capacity of Full-body MovementsChallengesWhat is complexity?How to compute information capacity?Match between two sequences?How to decorrelate mutual dependencies?How to capture full-body movement?
  11. 11. XMovement sequenceInformation Capacity of Full-body Movements
  12. 12. X YMovement sequence RepetitionInformation Capacity of Full-body Movements
  13. 13. X Yh(X) entropy of XInformation Capacity of Full-body Movements
  14. 14. X Yh(Y) entropy ofYInformation Capacity of Full-body Movements
  15. 15. X YI(X;Y) Mutual information between X andYI(X;Y) = h(X) – h(X|Y) = h(Y) – h(Y|X)Information Capacity of Full-body Movements
  16. 16. Information Capacity of Full-body MovementsComputational pipelinex"y"Autoregression+ rx"ry"Gaussian+process+r’x"r’y"II Complexity estimationrxp1 rxp2 rxp3 rxp4 rxp5 rxp6ryp1 ryp2 ryp3 ryp4 ryp5 ryp6TPV Mutual informationIII Dimension reductionρyx"Correla2ons+I CaptureCanonical+2me+warping+ix,y"IV Temporal alignmentxt xt+1 xt+2 xt+3 xt+4 xt+5εt(x) εt(x) εt(x) εt(x)εt(x) εt(x)εt(y)yt yt+1 yt+2 yt+3 yt+4 yt+5εt(y) εt(y) εt(y)εt(y) εt(y)
  17. 17. Step 1: Performance in intendedrepetitions is captured[CMU Mocap DB]X Y
  18. 18. Information Capacity of Full-body MovementsStep 2: Complexity estimation is donewith 2nd order autoregressionεt-1y)xt-1 xt xt+1 xt+2 xt+3 xt+4yt-1 yt yt+1 yt+2 yt+3 yt+4εt-1(x) εt(x) εt+1(x) εt+2(x)εt+3(x) εt+4(x)εt(y) εt+1(y) εt+2(y)εt+3(y) εt+4(y)ResidualsXY
  19. 19. Information Capacity of Full-body MovementsStep 3: Dimensionality reduction is donewith PCA or GP-LVM[Lawrence 05 JLMR]GP-LVM manifold for two dances in the balletdata (3 latent dimensions)X Y
  20. 20. Information Capacity of Full-body MovementsSelection of dimensions0.000.050.100.150.20AverageRMSE ●●●●●●●●●●●●●●●●●●●050100150200250AverageThroughput(bps)2 4 6 8 12 16 20Latent Dimensions●RMSEThroughput
  21. 21. Information Capacity of Full-body MovementsStep 4:Temporal alignment (optional)XYCanonicalTimeWarping CTW
  22. 22. Information Capacity of Full-body MovementsXYStep 4:Temporal alignment (optional)CanonicalTimeWarping CTW
  23. 23. Information Capacity of Full-body Movements CanonicalTimeWarping CTWXYStep 4:Temporal alignment (optional)
  24. 24. Information Capacity of Full-body Movements [Zhou & De La Torre 2009 NIPS]Example results
  25. 25. Information Capacity of Full-body MovementsStep 5: Mutual information is calculatedfrom estimated correlation of residuals[Kendall & Stuart 68]Mutual information is determined by the correlation of residuals:We estimate this and add a bias correction:Throughput is now mutual information per second
  26. 26. Information Capacity of Full-body MovementsFirst feasibility testsStanding still0 bpsBalancing with one leg0 bpsRapid caging of the palm289 bps43 bpswithout CTWPhaseSpace full-body suit and glove
  27. 27. Information Capacity of Full-body MovementsSensitivity to noise in recordinginstrument●●●●●●●●● ● ● ● ● ● ● ●0 0.0005 0.0015 0.002502004006008001200Noise FactorThroughput(bps)●TP(1|2)TP(2|1)PCA-TP
  28. 28. Information Capacity of Full-body MovementsStudy 1: Ballerina21-3312-1517-18
  29. 29. Information Capacity of Full-body MovementsUnencumbered 4 kg additional weightStudy 2: Mouse4 Fitts-bps 2 Fitts-bps
  30. 30. Information Capacity of Full-body Movements0 kg4 kgLow ID High ID38 bps24 bps 37 bps37 bpsUnencumbered 4 kg additional weightumbered 4 kg additional weight
  31. 31. Information Capacity of Full-body MovementsUnencumbered 4 kg additional weightHigh-IDTPs decreased when an ISI of 1,000 mswas imposedSlow motion
  32. 32. Information Capacity of Full-body MovementsStudy 3: Minority Report
  33. 33. Information Capacity of Full-body MovementsPCA-TP 78PCA-TP 440
  34. 34. Information Capacity of Full-body MovementsResults replicate a known perceptualdistraction in bimanual motor control313 bps353 bps289 bps[Meschner et al. 01 Nature]Sweet spot at ~60 cm
  35. 35. Information Capacity of Full-body MovementsBonus study: Expert gamerSpaceFortress[Boot et al. 10 Acta Psychologica]First trials2 bps21 bpsAfter 20 hours trials
  36. 36. Information Capacity of Full-body MovementsFitts-TPAimed movementsThis paperFull-body movementsInformation Distance Changes in motion directionNoise Effective width Variability between repetitionsW WD
  37. 37. Information Capacity of Full-body MovementsSolutions☐✓☐☐☐✓✓✓Step 4:Time warpingStep 2:AutoregressionStep 3: Dimension reductionStep 5: Mutual information☐✓ Step 1: Optical captureWhat is movement complexity?How to compute information capacity?Match between two sequences?How to decorrelate mutual dependencies?Capturing full-body movement?
  38. 38. Information Capacity of Full-body Movements• Analyze information capacity allowed by your design• Compare designs• Expose human factors• Explore best potentials for UIs
  39. 39. Information Capacity of Full-body Movementsinfocapacity.hiit.fiantti.oulasvirta@mpii.deteemu.roos@cs.helsinki.fiImplementation for KinectInteractivity i401

×