SlideShare a Scribd company logo

Neuromechanics of a Button Press: A talk at CHI 2018, April 2018

To press a button, a finger must push down and pull up with the right force and timing. How the motor system succeeds in button-pressing, in spite of neural noise and lacking direct access to the mechanism of the button, is poorly understood. This paper investigates a unifying account based on neurome- chanics. Mechanics is used to model muscles controlling the finger that contacts the button. Neurocognitive principles are used to model how the motor system learns appropriate muscle activations over repeated strokes though relying on degraded sensory feedback. Neuromechanical simulations yield a rich set of predictions for kinematics, dynamics, and user performance and may aid in understanding and improving input devices. We present a computational implementation and evaluate predictions for common button types.

1 of 31
Download to read offline
Neuromechanics
Antti Oulasvirta, Sunjun Kim, and Byungjoo Lee bit.do/neuromechanics
of a Button Press
Related papers at CHI 2018:
1. Control-theoretic Models of Pointing Tue 9-11.30 517C
2. Impact Activation Improves Rapid Key Pressing Mon 16.30-17.50 514AB
3. Moving Target Selection: A Cue Integration Model This session
Neuromechanics of a Button Press
Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics10x slowdown
Long-term goal:
A theory of input
Design
Feedback
Goals
Anatomy
Skill
Movement
Efficiency
Efficacy
Effort
Neuromechanics of a Button Press
Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics
Existing theories
Information theory
Human performance
Control theory
Cognitive models
Characterize
Intrapolate
Poor transfer
Change in device or task
insists on data collection
or manual task modeling
Neuromechanics of a Button Press
Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics
The brain’s point-of-view
Neuromechanics of a Button Press
Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics
Design
Anatomy
Goals
Feedback
Kinematics
Dynamics
Precision
Effort
Human-like
responses
Adapt
A generative
approach
Neuromechanics of a Button Press
Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics
The
physical
The
neural
What is neuromechanics?
Neural principles of motor control in biomechanical systems
The
physiological
[Enoka 2009]

Recommended

[CHI 2018] Impact Activation Improves Rapid Button Pressing
[CHI 2018] Impact Activation Improves Rapid Button Pressing[CHI 2018] Impact Activation Improves Rapid Button Pressing
[CHI 2018] Impact Activation Improves Rapid Button PressingSunjun Kim
 
Neurobionics and robotic neurorehabilitations
Neurobionics and robotic neurorehabilitationsNeurobionics and robotic neurorehabilitations
Neurobionics and robotic neurorehabilitationsNeurologyKota
 
Stress detection and relief using wearable physiological sensors
Stress detection and relief using wearable physiological sensorsStress detection and relief using wearable physiological sensors
Stress detection and relief using wearable physiological sensorsTELKOMNIKA JOURNAL
 
Brain plasticity and rehabilitation robotic therapies
Brain plasticity and rehabilitation  robotic therapiesBrain plasticity and rehabilitation  robotic therapies
Brain plasticity and rehabilitation robotic therapiesDavid Karchem
 
PEShare.co.uk Shared Resource
PEShare.co.uk Shared ResourcePEShare.co.uk Shared Resource
PEShare.co.uk Shared Resourcepeshare.co.uk
 
Automated Quantitative Measures of Forelimb Function in Rats and Mice
Automated Quantitative Measures of Forelimb Function in Rats and MiceAutomated Quantitative Measures of Forelimb Function in Rats and Mice
Automated Quantitative Measures of Forelimb Function in Rats and MiceInsideScientific
 
Advanced technology in Rehabilitation
Advanced technology in RehabilitationAdvanced technology in Rehabilitation
Advanced technology in RehabilitationVaikunthan Rajaratnam
 
IRJET- Brain Comuter Interface-A Survey
IRJET- Brain Comuter Interface-A SurveyIRJET- Brain Comuter Interface-A Survey
IRJET- Brain Comuter Interface-A SurveyIRJET Journal
 

More Related Content

Similar to Neuromechanics of a Button Press: A talk at CHI 2018, April 2018

IRJET- Wireless Healthcare Monitoring using Android Phones
IRJET- Wireless Healthcare Monitoring using Android PhonesIRJET- Wireless Healthcare Monitoring using Android Phones
IRJET- Wireless Healthcare Monitoring using Android PhonesIRJET Journal
 
45891026 brain-computer-interface-seminar-report
45891026 brain-computer-interface-seminar-report45891026 brain-computer-interface-seminar-report
45891026 brain-computer-interface-seminar-reportkapilpanwariet
 
IRJET -Stress Detection using Arduino
IRJET -Stress Detection using ArduinoIRJET -Stress Detection using Arduino
IRJET -Stress Detection using ArduinoIRJET Journal
 
Improved feature exctraction process to detect seizure using CHBMIT-dataset ...
Improved feature exctraction process to detect seizure  using CHBMIT-dataset ...Improved feature exctraction process to detect seizure  using CHBMIT-dataset ...
Improved feature exctraction process to detect seizure using CHBMIT-dataset ...IJECEIAES
 
Non-Contact Health Monitoring System Using Image and Signal Processing
Non-Contact Health Monitoring System Using Image and Signal ProcessingNon-Contact Health Monitoring System Using Image and Signal Processing
Non-Contact Health Monitoring System Using Image and Signal ProcessingAtul Kumar Sharma
 
IRJET- Automation of a Prosthetic Limb using Shared Control of Brain Mach...
IRJET-  	  Automation of a Prosthetic Limb using Shared Control of Brain Mach...IRJET-  	  Automation of a Prosthetic Limb using Shared Control of Brain Mach...
IRJET- Automation of a Prosthetic Limb using Shared Control of Brain Mach...IRJET Journal
 
Neural interfacing
Neural interfacingNeural interfacing
Neural interfacingKirtan Shah
 
Full Body Spatial Vibrotactile Brain Computer Interface Paradigm
Full Body Spatial Vibrotactile Brain Computer Interface ParadigmFull Body Spatial Vibrotactile Brain Computer Interface Paradigm
Full Body Spatial Vibrotactile Brain Computer Interface ParadigmTakumi Kodama
 
Biorobotics Abstract
Biorobotics AbstractBiorobotics Abstract
Biorobotics AbstractAbida Zama
 
IRJET-Estimation of Meditation Effect on Attention Level using EEG
IRJET-Estimation of Meditation Effect on Attention Level using EEGIRJET-Estimation of Meditation Effect on Attention Level using EEG
IRJET-Estimation of Meditation Effect on Attention Level using EEGIRJET Journal
 
Recognition of new gestures using myo armband for myoelectric prosthetic appl...
Recognition of new gestures using myo armband for myoelectric prosthetic appl...Recognition of new gestures using myo armband for myoelectric prosthetic appl...
Recognition of new gestures using myo armband for myoelectric prosthetic appl...IJECEIAES
 
Spinesense: Advanced Pain Detection System for Spinal Cord
Spinesense: Advanced Pain Detection System for Spinal CordSpinesense: Advanced Pain Detection System for Spinal Cord
Spinesense: Advanced Pain Detection System for Spinal CordIRJET Journal
 
Leap - Learning Adaptive Prosthetic Limb
Leap - Learning Adaptive Prosthetic LimbLeap - Learning Adaptive Prosthetic Limb
Leap - Learning Adaptive Prosthetic LimbPrasanna Venkataramanan
 
Tactile Brain-Computer Interface Using Classification of P300 Responses Evoke...
Tactile Brain-Computer Interface Using Classification of P300 Responses Evoke...Tactile Brain-Computer Interface Using Classification of P300 Responses Evoke...
Tactile Brain-Computer Interface Using Classification of P300 Responses Evoke...Takumi Kodama
 
IRJET- Emotion Analysis for Personality Inference from EEG Signals
IRJET- Emotion Analysis for Personality Inference from EEG SignalsIRJET- Emotion Analysis for Personality Inference from EEG Signals
IRJET- Emotion Analysis for Personality Inference from EEG SignalsIRJET Journal
 
A method for detection and reduction of stress using EEG
A method for detection and reduction of stress using EEGA method for detection and reduction of stress using EEG
A method for detection and reduction of stress using EEGIRJET Journal
 
ApplyingElectroencephalography inEstablishing SafetyCriterion byMeasuring Men...
ApplyingElectroencephalography inEstablishing SafetyCriterion byMeasuring Men...ApplyingElectroencephalography inEstablishing SafetyCriterion byMeasuring Men...
ApplyingElectroencephalography inEstablishing SafetyCriterion byMeasuring Men...IJRTEMJOURNAL
 
ApplyingElectroencephalography inEstablishing SafetyCriterion byMeasuring Men...
ApplyingElectroencephalography inEstablishing SafetyCriterion byMeasuring Men...ApplyingElectroencephalography inEstablishing SafetyCriterion byMeasuring Men...
ApplyingElectroencephalography inEstablishing SafetyCriterion byMeasuring Men...journal ijrtem
 
IRJET- Survey on EEG Based Brainwave Controlled Home Automation
IRJET-  	  Survey on EEG Based Brainwave Controlled Home AutomationIRJET-  	  Survey on EEG Based Brainwave Controlled Home Automation
IRJET- Survey on EEG Based Brainwave Controlled Home AutomationIRJET Journal
 

Similar to Neuromechanics of a Button Press: A talk at CHI 2018, April 2018 (20)

IRJET- Wireless Healthcare Monitoring using Android Phones
IRJET- Wireless Healthcare Monitoring using Android PhonesIRJET- Wireless Healthcare Monitoring using Android Phones
IRJET- Wireless Healthcare Monitoring using Android Phones
 
45891026 brain-computer-interface-seminar-report
45891026 brain-computer-interface-seminar-report45891026 brain-computer-interface-seminar-report
45891026 brain-computer-interface-seminar-report
 
IRJET -Stress Detection using Arduino
IRJET -Stress Detection using ArduinoIRJET -Stress Detection using Arduino
IRJET -Stress Detection using Arduino
 
Improved feature exctraction process to detect seizure using CHBMIT-dataset ...
Improved feature exctraction process to detect seizure  using CHBMIT-dataset ...Improved feature exctraction process to detect seizure  using CHBMIT-dataset ...
Improved feature exctraction process to detect seizure using CHBMIT-dataset ...
 
Non-Contact Health Monitoring System Using Image and Signal Processing
Non-Contact Health Monitoring System Using Image and Signal ProcessingNon-Contact Health Monitoring System Using Image and Signal Processing
Non-Contact Health Monitoring System Using Image and Signal Processing
 
IRJET- Automation of a Prosthetic Limb using Shared Control of Brain Mach...
IRJET-  	  Automation of a Prosthetic Limb using Shared Control of Brain Mach...IRJET-  	  Automation of a Prosthetic Limb using Shared Control of Brain Mach...
IRJET- Automation of a Prosthetic Limb using Shared Control of Brain Mach...
 
Pain in the neck - Bernard Martin
Pain in the neck - Bernard MartinPain in the neck - Bernard Martin
Pain in the neck - Bernard Martin
 
Neural interfacing
Neural interfacingNeural interfacing
Neural interfacing
 
Full Body Spatial Vibrotactile Brain Computer Interface Paradigm
Full Body Spatial Vibrotactile Brain Computer Interface ParadigmFull Body Spatial Vibrotactile Brain Computer Interface Paradigm
Full Body Spatial Vibrotactile Brain Computer Interface Paradigm
 
Biorobotics Abstract
Biorobotics AbstractBiorobotics Abstract
Biorobotics Abstract
 
IRJET-Estimation of Meditation Effect on Attention Level using EEG
IRJET-Estimation of Meditation Effect on Attention Level using EEGIRJET-Estimation of Meditation Effect on Attention Level using EEG
IRJET-Estimation of Meditation Effect on Attention Level using EEG
 
Recognition of new gestures using myo armband for myoelectric prosthetic appl...
Recognition of new gestures using myo armband for myoelectric prosthetic appl...Recognition of new gestures using myo armband for myoelectric prosthetic appl...
Recognition of new gestures using myo armband for myoelectric prosthetic appl...
 
Spinesense: Advanced Pain Detection System for Spinal Cord
Spinesense: Advanced Pain Detection System for Spinal CordSpinesense: Advanced Pain Detection System for Spinal Cord
Spinesense: Advanced Pain Detection System for Spinal Cord
 
Leap - Learning Adaptive Prosthetic Limb
Leap - Learning Adaptive Prosthetic LimbLeap - Learning Adaptive Prosthetic Limb
Leap - Learning Adaptive Prosthetic Limb
 
Tactile Brain-Computer Interface Using Classification of P300 Responses Evoke...
Tactile Brain-Computer Interface Using Classification of P300 Responses Evoke...Tactile Brain-Computer Interface Using Classification of P300 Responses Evoke...
Tactile Brain-Computer Interface Using Classification of P300 Responses Evoke...
 
IRJET- Emotion Analysis for Personality Inference from EEG Signals
IRJET- Emotion Analysis for Personality Inference from EEG SignalsIRJET- Emotion Analysis for Personality Inference from EEG Signals
IRJET- Emotion Analysis for Personality Inference from EEG Signals
 
A method for detection and reduction of stress using EEG
A method for detection and reduction of stress using EEGA method for detection and reduction of stress using EEG
A method for detection and reduction of stress using EEG
 
ApplyingElectroencephalography inEstablishing SafetyCriterion byMeasuring Men...
ApplyingElectroencephalography inEstablishing SafetyCriterion byMeasuring Men...ApplyingElectroencephalography inEstablishing SafetyCriterion byMeasuring Men...
ApplyingElectroencephalography inEstablishing SafetyCriterion byMeasuring Men...
 
ApplyingElectroencephalography inEstablishing SafetyCriterion byMeasuring Men...
ApplyingElectroencephalography inEstablishing SafetyCriterion byMeasuring Men...ApplyingElectroencephalography inEstablishing SafetyCriterion byMeasuring Men...
ApplyingElectroencephalography inEstablishing SafetyCriterion byMeasuring Men...
 
IRJET- Survey on EEG Based Brainwave Controlled Home Automation
IRJET-  	  Survey on EEG Based Brainwave Controlled Home AutomationIRJET-  	  Survey on EEG Based Brainwave Controlled Home Automation
IRJET- Survey on EEG Based Brainwave Controlled Home Automation
 

More from Aalto University

"Computational rationality as a theory of interaction" - CHI'22 paper
"Computational rationality as a theory of interaction" - CHI'22 paper"Computational rationality as a theory of interaction" - CHI'22 paper
"Computational rationality as a theory of interaction" - CHI'22 paperAalto University
 
Observations on typing from 136 million keystrokes - Presentation by Antti Ou...
Observations on typing from 136 million keystrokes - Presentation by Antti Ou...Observations on typing from 136 million keystrokes - Presentation by Antti Ou...
Observations on typing from 136 million keystrokes - Presentation by Antti Ou...Aalto University
 
"Computational Support for Functionality Selection in Interaction Design" CHI...
"Computational Support for Functionality Selection in Interaction Design" CHI..."Computational Support for Functionality Selection in Interaction Design" CHI...
"Computational Support for Functionality Selection in Interaction Design" CHI...Aalto University
 
User Interfaces that Design Themselves: Talk given at Data-Driven Design Day ...
User Interfaces that Design Themselves: Talk given at Data-Driven Design Day ...User Interfaces that Design Themselves: Talk given at Data-Driven Design Day ...
User Interfaces that Design Themselves: Talk given at Data-Driven Design Day ...Aalto University
 
Inverse Modeling for Cognitive Science "in the Wild"
Inverse Modeling for Cognitive Science "in the Wild"Inverse Modeling for Cognitive Science "in the Wild"
Inverse Modeling for Cognitive Science "in the Wild"Aalto University
 
Computational Rationality I - a Lecture at Aalto University by Antti Oulasvirta
Computational Rationality I - a Lecture at Aalto University by Antti OulasvirtaComputational Rationality I - a Lecture at Aalto University by Antti Oulasvirta
Computational Rationality I - a Lecture at Aalto University by Antti OulasvirtaAalto University
 
HCI Research as Problem-Solving [CHI'16, presentation slides]
HCI Research as Problem-Solving [CHI'16, presentation slides] HCI Research as Problem-Solving [CHI'16, presentation slides]
HCI Research as Problem-Solving [CHI'16, presentation slides] Aalto University
 
Can Computers Design? Presented at interaction16, March 2, 2016, Helsinki by ...
Can Computers Design? Presented at interaction16, March 2, 2016, Helsinki by ...Can Computers Design? Presented at interaction16, March 2, 2016, Helsinki by ...
Can Computers Design? Presented at interaction16, March 2, 2016, Helsinki by ...Aalto University
 
Model-Based User Interface Optimization: Part V: DISCUSSION - At SICSA Summer...
Model-Based User Interface Optimization: Part V: DISCUSSION - At SICSA Summer...Model-Based User Interface Optimization: Part V: DISCUSSION - At SICSA Summer...
Model-Based User Interface Optimization: Part V: DISCUSSION - At SICSA Summer...Aalto University
 
Model-Based User Interface Optimization: Part IV: ADVANCED TOPICS - At SICSA ...
Model-Based User Interface Optimization: Part IV: ADVANCED TOPICS - At SICSA ...Model-Based User Interface Optimization: Part IV: ADVANCED TOPICS - At SICSA ...
Model-Based User Interface Optimization: Part IV: ADVANCED TOPICS - At SICSA ...Aalto University
 
Model-Based User Interface Optimization: Part III: SOLVING REAL PROBLEMS - At...
Model-Based User Interface Optimization: Part III: SOLVING REAL PROBLEMS - At...Model-Based User Interface Optimization: Part III: SOLVING REAL PROBLEMS - At...
Model-Based User Interface Optimization: Part III: SOLVING REAL PROBLEMS - At...Aalto University
 
Model-Based User Interface Optimization: Part II: LETTER ASSIGNMENT - At SICS...
Model-Based User Interface Optimization: Part II: LETTER ASSIGNMENT - At SICS...Model-Based User Interface Optimization: Part II: LETTER ASSIGNMENT - At SICS...
Model-Based User Interface Optimization: Part II: LETTER ASSIGNMENT - At SICS...Aalto University
 
Model-Based User Interface Optimization: Part I INTRODUCTION - At SICSA Summe...
Model-Based User Interface Optimization: Part I INTRODUCTION - At SICSA Summe...Model-Based User Interface Optimization: Part I INTRODUCTION - At SICSA Summe...
Model-Based User Interface Optimization: Part I INTRODUCTION - At SICSA Summe...Aalto University
 
CHI 2014 talk by Antti Oulasvirta: Automated Nonlinear Regression Modeling fo...
CHI 2014 talk by Antti Oulasvirta: Automated Nonlinear Regression Modeling fo...CHI 2014 talk by Antti Oulasvirta: Automated Nonlinear Regression Modeling fo...
CHI 2014 talk by Antti Oulasvirta: Automated Nonlinear Regression Modeling fo...Aalto University
 
Information Capacity of Full-body Movements (CHI'13)
Information Capacity of Full-body Movements (CHI'13)Information Capacity of Full-body Movements (CHI'13)
Information Capacity of Full-body Movements (CHI'13)Aalto University
 
Improving Two-Thumb Text Entry on Touchscreen Devices
Improving Two-Thumb Text Entry on Touchscreen DevicesImproving Two-Thumb Text Entry on Touchscreen Devices
Improving Two-Thumb Text Entry on Touchscreen DevicesAalto University
 
Studying interaction with 3D mobile maps
Studying interaction with 3D mobile mapsStudying interaction with 3D mobile maps
Studying interaction with 3D mobile mapsAalto University
 

More from Aalto University (17)

"Computational rationality as a theory of interaction" - CHI'22 paper
"Computational rationality as a theory of interaction" - CHI'22 paper"Computational rationality as a theory of interaction" - CHI'22 paper
"Computational rationality as a theory of interaction" - CHI'22 paper
 
Observations on typing from 136 million keystrokes - Presentation by Antti Ou...
Observations on typing from 136 million keystrokes - Presentation by Antti Ou...Observations on typing from 136 million keystrokes - Presentation by Antti Ou...
Observations on typing from 136 million keystrokes - Presentation by Antti Ou...
 
"Computational Support for Functionality Selection in Interaction Design" CHI...
"Computational Support for Functionality Selection in Interaction Design" CHI..."Computational Support for Functionality Selection in Interaction Design" CHI...
"Computational Support for Functionality Selection in Interaction Design" CHI...
 
User Interfaces that Design Themselves: Talk given at Data-Driven Design Day ...
User Interfaces that Design Themselves: Talk given at Data-Driven Design Day ...User Interfaces that Design Themselves: Talk given at Data-Driven Design Day ...
User Interfaces that Design Themselves: Talk given at Data-Driven Design Day ...
 
Inverse Modeling for Cognitive Science "in the Wild"
Inverse Modeling for Cognitive Science "in the Wild"Inverse Modeling for Cognitive Science "in the Wild"
Inverse Modeling for Cognitive Science "in the Wild"
 
Computational Rationality I - a Lecture at Aalto University by Antti Oulasvirta
Computational Rationality I - a Lecture at Aalto University by Antti OulasvirtaComputational Rationality I - a Lecture at Aalto University by Antti Oulasvirta
Computational Rationality I - a Lecture at Aalto University by Antti Oulasvirta
 
HCI Research as Problem-Solving [CHI'16, presentation slides]
HCI Research as Problem-Solving [CHI'16, presentation slides] HCI Research as Problem-Solving [CHI'16, presentation slides]
HCI Research as Problem-Solving [CHI'16, presentation slides]
 
Can Computers Design? Presented at interaction16, March 2, 2016, Helsinki by ...
Can Computers Design? Presented at interaction16, March 2, 2016, Helsinki by ...Can Computers Design? Presented at interaction16, March 2, 2016, Helsinki by ...
Can Computers Design? Presented at interaction16, March 2, 2016, Helsinki by ...
 
Model-Based User Interface Optimization: Part V: DISCUSSION - At SICSA Summer...
Model-Based User Interface Optimization: Part V: DISCUSSION - At SICSA Summer...Model-Based User Interface Optimization: Part V: DISCUSSION - At SICSA Summer...
Model-Based User Interface Optimization: Part V: DISCUSSION - At SICSA Summer...
 
Model-Based User Interface Optimization: Part IV: ADVANCED TOPICS - At SICSA ...
Model-Based User Interface Optimization: Part IV: ADVANCED TOPICS - At SICSA ...Model-Based User Interface Optimization: Part IV: ADVANCED TOPICS - At SICSA ...
Model-Based User Interface Optimization: Part IV: ADVANCED TOPICS - At SICSA ...
 
Model-Based User Interface Optimization: Part III: SOLVING REAL PROBLEMS - At...
Model-Based User Interface Optimization: Part III: SOLVING REAL PROBLEMS - At...Model-Based User Interface Optimization: Part III: SOLVING REAL PROBLEMS - At...
Model-Based User Interface Optimization: Part III: SOLVING REAL PROBLEMS - At...
 
Model-Based User Interface Optimization: Part II: LETTER ASSIGNMENT - At SICS...
Model-Based User Interface Optimization: Part II: LETTER ASSIGNMENT - At SICS...Model-Based User Interface Optimization: Part II: LETTER ASSIGNMENT - At SICS...
Model-Based User Interface Optimization: Part II: LETTER ASSIGNMENT - At SICS...
 
Model-Based User Interface Optimization: Part I INTRODUCTION - At SICSA Summe...
Model-Based User Interface Optimization: Part I INTRODUCTION - At SICSA Summe...Model-Based User Interface Optimization: Part I INTRODUCTION - At SICSA Summe...
Model-Based User Interface Optimization: Part I INTRODUCTION - At SICSA Summe...
 
CHI 2014 talk by Antti Oulasvirta: Automated Nonlinear Regression Modeling fo...
CHI 2014 talk by Antti Oulasvirta: Automated Nonlinear Regression Modeling fo...CHI 2014 talk by Antti Oulasvirta: Automated Nonlinear Regression Modeling fo...
CHI 2014 talk by Antti Oulasvirta: Automated Nonlinear Regression Modeling fo...
 
Information Capacity of Full-body Movements (CHI'13)
Information Capacity of Full-body Movements (CHI'13)Information Capacity of Full-body Movements (CHI'13)
Information Capacity of Full-body Movements (CHI'13)
 
Improving Two-Thumb Text Entry on Touchscreen Devices
Improving Two-Thumb Text Entry on Touchscreen DevicesImproving Two-Thumb Text Entry on Touchscreen Devices
Improving Two-Thumb Text Entry on Touchscreen Devices
 
Studying interaction with 3D mobile maps
Studying interaction with 3D mobile mapsStudying interaction with 3D mobile maps
Studying interaction with 3D mobile maps
 

Recently uploaded

CHEMICAL TESTS FOR GLYCOSIDES AND ALKALOIDS.pptx
CHEMICAL TESTS FOR GLYCOSIDES AND ALKALOIDS.pptxCHEMICAL TESTS FOR GLYCOSIDES AND ALKALOIDS.pptx
CHEMICAL TESTS FOR GLYCOSIDES AND ALKALOIDS.pptxASWIN ANANDH
 
Differences between syrups and elixirs .pptx
Differences between  syrups and elixirs .pptxDifferences between  syrups and elixirs .pptx
Differences between syrups and elixirs .pptxushakiranmai4
 
A tale of two Lucys - Delft lecture - March 4, 2024
A tale of two Lucys - Delft lecture - March 4, 2024A tale of two Lucys - Delft lecture - March 4, 2024
A tale of two Lucys - Delft lecture - March 4, 2024Richard Gill
 
dkNET Webinar: The Collaborative Microbial Metabolite Center – Democratizing ...
dkNET Webinar: The Collaborative Microbial Metabolite Center – Democratizing ...dkNET Webinar: The Collaborative Microbial Metabolite Center – Democratizing ...
dkNET Webinar: The Collaborative Microbial Metabolite Center – Democratizing ...dkNET
 
Chemical Bonding and it's Types 001.pptx
Chemical Bonding and it's Types 001.pptxChemical Bonding and it's Types 001.pptx
Chemical Bonding and it's Types 001.pptxperiyar arts college
 
commercial production of cellulase enzyme and its uses
commercial production of cellulase enzyme and its usescommercial production of cellulase enzyme and its uses
commercial production of cellulase enzyme and its usesSilpa Selvaraj
 
Lung imaging Using 3D Dual-Echo FID Ultra-short Echo Time MRI with novel Rose...
Lung imaging Using 3D Dual-Echo FID Ultra-short Echo Time MRI with novel Rose...Lung imaging Using 3D Dual-Echo FID Ultra-short Echo Time MRI with novel Rose...
Lung imaging Using 3D Dual-Echo FID Ultra-short Echo Time MRI with novel Rose...Uzay Emir
 
electrophoresis: types, advantages, disadvantages and applications.
electrophoresis: types, advantages, disadvantages and applications.electrophoresis: types, advantages, disadvantages and applications.
electrophoresis: types, advantages, disadvantages and applications.Silpa Selvaraj
 
American Eclipse A Nation’s Epic Race to Catch the_240225_095603
American Eclipse A Nation’s Epic Race to Catch the_240225_095603American Eclipse A Nation’s Epic Race to Catch the_240225_095603
American Eclipse A Nation’s Epic Race to Catch the_240225_095603SOCIEDAD JULIO GARAVITO
 
Seminario biología molecular Lina Charris
Seminario biología molecular Lina CharrisSeminario biología molecular Lina Charris
Seminario biología molecular Lina CharrisLinaMarcelaCharrisRa
 
the menstrual cycle in female reproductive system
the menstrual cycle in female reproductive systemthe menstrual cycle in female reproductive system
the menstrual cycle in female reproductive systemGilmeTripole1
 
Presentacion Mariana Arango- biología molecular
Presentacion Mariana Arango- biología molecularPresentacion Mariana Arango- biología molecular
Presentacion Mariana Arango- biología molecularmarianaarangop
 
Introduction to Chromatography (Column chromatography)
Introduction to Chromatography (Column chromatography)Introduction to Chromatography (Column chromatography)
Introduction to Chromatography (Column chromatography)Ahmed Metwaly
 
Open Access Publishing in Astrophysics and the Open Journal of Astrophysics
Open Access Publishing in Astrophysics and the Open Journal of AstrophysicsOpen Access Publishing in Astrophysics and the Open Journal of Astrophysics
Open Access Publishing in Astrophysics and the Open Journal of AstrophysicsPeter Coles
 
Genetic Code. A comprehensive overview..pdf
Genetic Code. A comprehensive overview..pdfGenetic Code. A comprehensive overview..pdf
Genetic Code. A comprehensive overview..pdfmughalgumar440
 
Cytotoxic Activity of Linum usitatissimum L. Essential oil against Lung Adeno...
Cytotoxic Activity of Linum usitatissimum L. Essential oil against Lung Adeno...Cytotoxic Activity of Linum usitatissimum L. Essential oil against Lung Adeno...
Cytotoxic Activity of Linum usitatissimum L. Essential oil against Lung Adeno...AmalDhivaharS
 
Ento-322, Agrochemicals for agriculture usee
Ento-322, Agrochemicals for agriculture useeEnto-322, Agrochemicals for agriculture usee
Ento-322, Agrochemicals for agriculture useeDrAnita Sharma
 
Anatomy-Reproductive-System for grade 10
Anatomy-Reproductive-System for grade 10Anatomy-Reproductive-System for grade 10
Anatomy-Reproductive-System for grade 10rolanaribato30
 
Seminario biología molecular Lina Charris
Seminario biología molecular Lina CharrisSeminario biología molecular Lina Charris
Seminario biología molecular Lina CharrisLinaMarcelaCharrisRa
 

Recently uploaded (20)

REGULATION OF METABOLISM IN PLANTS AND THE DIFFERENT MECHANISMS
REGULATION OF METABOLISM IN PLANTS  AND THE DIFFERENT MECHANISMSREGULATION OF METABOLISM IN PLANTS  AND THE DIFFERENT MECHANISMS
REGULATION OF METABOLISM IN PLANTS AND THE DIFFERENT MECHANISMS
 
CHEMICAL TESTS FOR GLYCOSIDES AND ALKALOIDS.pptx
CHEMICAL TESTS FOR GLYCOSIDES AND ALKALOIDS.pptxCHEMICAL TESTS FOR GLYCOSIDES AND ALKALOIDS.pptx
CHEMICAL TESTS FOR GLYCOSIDES AND ALKALOIDS.pptx
 
Differences between syrups and elixirs .pptx
Differences between  syrups and elixirs .pptxDifferences between  syrups and elixirs .pptx
Differences between syrups and elixirs .pptx
 
A tale of two Lucys - Delft lecture - March 4, 2024
A tale of two Lucys - Delft lecture - March 4, 2024A tale of two Lucys - Delft lecture - March 4, 2024
A tale of two Lucys - Delft lecture - March 4, 2024
 
dkNET Webinar: The Collaborative Microbial Metabolite Center – Democratizing ...
dkNET Webinar: The Collaborative Microbial Metabolite Center – Democratizing ...dkNET Webinar: The Collaborative Microbial Metabolite Center – Democratizing ...
dkNET Webinar: The Collaborative Microbial Metabolite Center – Democratizing ...
 
Chemical Bonding and it's Types 001.pptx
Chemical Bonding and it's Types 001.pptxChemical Bonding and it's Types 001.pptx
Chemical Bonding and it's Types 001.pptx
 
commercial production of cellulase enzyme and its uses
commercial production of cellulase enzyme and its usescommercial production of cellulase enzyme and its uses
commercial production of cellulase enzyme and its uses
 
Lung imaging Using 3D Dual-Echo FID Ultra-short Echo Time MRI with novel Rose...
Lung imaging Using 3D Dual-Echo FID Ultra-short Echo Time MRI with novel Rose...Lung imaging Using 3D Dual-Echo FID Ultra-short Echo Time MRI with novel Rose...
Lung imaging Using 3D Dual-Echo FID Ultra-short Echo Time MRI with novel Rose...
 
electrophoresis: types, advantages, disadvantages and applications.
electrophoresis: types, advantages, disadvantages and applications.electrophoresis: types, advantages, disadvantages and applications.
electrophoresis: types, advantages, disadvantages and applications.
 
American Eclipse A Nation’s Epic Race to Catch the_240225_095603
American Eclipse A Nation’s Epic Race to Catch the_240225_095603American Eclipse A Nation’s Epic Race to Catch the_240225_095603
American Eclipse A Nation’s Epic Race to Catch the_240225_095603
 
Seminario biología molecular Lina Charris
Seminario biología molecular Lina CharrisSeminario biología molecular Lina Charris
Seminario biología molecular Lina Charris
 
the menstrual cycle in female reproductive system
the menstrual cycle in female reproductive systemthe menstrual cycle in female reproductive system
the menstrual cycle in female reproductive system
 
Presentacion Mariana Arango- biología molecular
Presentacion Mariana Arango- biología molecularPresentacion Mariana Arango- biología molecular
Presentacion Mariana Arango- biología molecular
 
Introduction to Chromatography (Column chromatography)
Introduction to Chromatography (Column chromatography)Introduction to Chromatography (Column chromatography)
Introduction to Chromatography (Column chromatography)
 
Open Access Publishing in Astrophysics and the Open Journal of Astrophysics
Open Access Publishing in Astrophysics and the Open Journal of AstrophysicsOpen Access Publishing in Astrophysics and the Open Journal of Astrophysics
Open Access Publishing in Astrophysics and the Open Journal of Astrophysics
 
Genetic Code. A comprehensive overview..pdf
Genetic Code. A comprehensive overview..pdfGenetic Code. A comprehensive overview..pdf
Genetic Code. A comprehensive overview..pdf
 
Cytotoxic Activity of Linum usitatissimum L. Essential oil against Lung Adeno...
Cytotoxic Activity of Linum usitatissimum L. Essential oil against Lung Adeno...Cytotoxic Activity of Linum usitatissimum L. Essential oil against Lung Adeno...
Cytotoxic Activity of Linum usitatissimum L. Essential oil against Lung Adeno...
 
Ento-322, Agrochemicals for agriculture usee
Ento-322, Agrochemicals for agriculture useeEnto-322, Agrochemicals for agriculture usee
Ento-322, Agrochemicals for agriculture usee
 
Anatomy-Reproductive-System for grade 10
Anatomy-Reproductive-System for grade 10Anatomy-Reproductive-System for grade 10
Anatomy-Reproductive-System for grade 10
 
Seminario biología molecular Lina Charris
Seminario biología molecular Lina CharrisSeminario biología molecular Lina Charris
Seminario biología molecular Lina Charris
 

Neuromechanics of a Button Press: A talk at CHI 2018, April 2018

  • 1. Neuromechanics Antti Oulasvirta, Sunjun Kim, and Byungjoo Lee bit.do/neuromechanics of a Button Press Related papers at CHI 2018: 1. Control-theoretic Models of Pointing Tue 9-11.30 517C 2. Impact Activation Improves Rapid Key Pressing Mon 16.30-17.50 514AB 3. Moving Target Selection: A Cue Integration Model This session
  • 2. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics10x slowdown Long-term goal: A theory of input Design Feedback Goals Anatomy Skill Movement Efficiency Efficacy Effort
  • 3. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Existing theories Information theory Human performance Control theory Cognitive models Characterize Intrapolate Poor transfer Change in device or task insists on data collection or manual task modeling
  • 4. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics The brain’s point-of-view
  • 5. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Design Anatomy Goals Feedback Kinematics Dynamics Precision Effort Human-like responses Adapt A generative approach
  • 6. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics The physical The neural What is neuromechanics? Neural principles of motor control in biomechanical systems The physiological [Enoka 2009]
  • 7. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Input is control of mediated sensations The brain Peripheral nervous system Sensory nervous system Action potential Physical stimulation Feedforward Feedback Limbs Sensory organs Device
  • 8. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Probabilistic internal model Sensory signals Activation signal Cue integration Integrated percept Perceptual control task Prediction error Prediction Muscle, bone, tissue, device, sensory organs Overview of the theory
  • 9. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Theory: Elements Biomechanical simulation Cue integration Probabilistic internal model Perceptual control [Enoka 2009] [Sung-Hee Lee 2009] [Ernst 2004, 2006] [Powers 1973, 2009] [Clark 2013] [Hohwy 2016]
  • 10. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Too brief for closed-loop control Button mechanism not perceived Ephemeral sensations Noisy neuromuscular system 10x slowdown Precise time and force Effective skill transfer Ability to adapt and recover Button- pressing: A miracle
  • 11. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Random movements are unsuccessful
  • 12. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Biomechanics simulation Noisy muscle activation signal Mechanoreceptive sensor Proprioceptive sensor
  • 13. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Silicone finger tip Pressure sensor Hill type muscle Joint angle sensor Real button Robotic implementation + Noise + Noise+ Noise
  • 14. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics No motor noise à super-human performance
  • 15. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Sensory signals Cue integration Perceptual center eeds a threshold value. This ls: visual and auditory (beep). o-beep delays are assumed to utation of p-Centers m is connected to four extero- ion, proprioception, audition, ity i produces a p-center pci. sfer of a neural signal evoked hanoreceptors. We are espe- noreceptors on the finger pad ming of a button press. Slowly ensitive to coarse spatial struc- flat top surface of the button), ers respond to motion. Kim in signals from the fingertip ion, and jerk from the finger t force and indentation have d force correlates highly with use maximum likelihood estimation (MLE) to obtain estimate of pco. For another implementation of cue inte see [35]. In MLE, assuming that a single-cue estim unbiased but corrupted by Gaussian noise, the optimal s for estimating pco is a weighted average [16, 17]: pco = Â i wi pci where wi = 1/s2 i Âi 1/s2 i with wi being the weight given to the ith single-cue es and s2 i being that estimate’s variance. Figure 6 sho emplary p-center calculations: signal-specific (pci) an grated p-centers (pco) from 100 simulated runs of NEU CHANIC pressing a tactile button. Note that absolute ences among pci do not affect pco, only signal varian The integrated timing estimate is robust to long delays auditory or visual feedback. This assumption is base study showing that physiological events that take place q within a few hundred milliseconds, do not tend to be over- nor underestimations of event durations [14]. Maximum likelihood estimator “When was the button activated?” Modality-specific noise variances Proprio- ceptive Tactile Integration is sensitive to how reliable the cues are
  • 16. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics No mechanoreceptive feedback signal
  • 17. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Perceptual control task Choose (1) a motor command and (2) expected perception that minimizes this objective: The oned ntin- prio- ion), Gaus- form error tiva- cuss esult ptual enter com- f cue (GP) and e the with signal offset µ, signal amplitude t, and duration s of the agonist (A+) muscle. We have set physiologically plausible extrema (min and max) for the activation parameters. Note that this formulation assumes that the antagonist muscle re- sists motion passively. More determinate pull-up motion can be achieved by adding similar parameters for the antagonist muscle (A-). The objective is to determine motor command (q) and as- sociated estimate of perceived button activation (pce) that minimize error: min q,pce EP(q, pce)+EA(q)+EC(q)+wFM(q) (2) where EP is perceptual error, EA is error in activating the button, and EC is error in making contact (button cap not touched). FM is muscle force expenditure computed from the Hill muscle model (see below), and w is a tuning factor. We assume that activation and contact errors are trivial to perceive. Therefore, EA and EC are binary: 1 in the case of error and 0 otherwise. Perceptual error EP is defined as distance (in time) between expected p-center pce and observed p-center pco: Objective function Perceptual error Activation failure (binary) Contact failure (binary) Muscle force User goal: Lightness vs. precision
  • 18. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Perceptual error Minimize error between expected and perceived activation time
  • 19. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics No button- activation term
  • 20. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics No muscle effort term
  • 21. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics A probabilistic internal model learns motor outputs that minimize the perceptual objective A Bayesian optimizer with a Gaussian Process prior Approximate Bayesian Computation (ABC)Approximate Bayesian Computation (ABC) Approximate Bayesian Computat ... Finds a good button press after 10-20 trials
  • 22. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Model parameters Table 1. Model parameters. Button parameters here given for physical buttons. Task parameters (e.g., finger starting height) are given in text. f denotes function Variable Description Value, Unit Ref. fr Radius of finger cone 7.0 mm fw Length of finger 60 mm rf Density of finger 985 kg/m3 cf Damping of finger pulp 1.5 N·s/m [64] kf Stiffness of finger pulp f, N/m [65] wb Width of key cap 14 mm db Depth of key cap 10 mm rb Density of key cap 700 kg/m3 cb Damping of button 0.1 N·s/m ks Elasticity of muscle 0.8·PCSA [38] kd Elasticity of muscle 0.1·ks [38] kc Damping of muscle 6 N·s/m [38] PCSA Phys. cross-sectional area 4 cm2 L0ag, L0an Initial muscle length 300 mm sn Neuromuscular noise 5·10 2 sm Mechanoreception noise 1·10 8 sp Proprioception noise 8·10 7 sa Sound and audition noise 5·10 4 sv Display and vision noise 2·10 2 system. We have set these parameters manually in order to reproduce certain basic effects: Neuromuscular noise, which reflects the joint additive contribution of neural and muscular Physically measurable Tuned based on literature
  • 23. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Simulation workbench (MATLAB) bit.do/neuromechanics But are the outputs realistic?
  • 24. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Precision & success: Predictions Figure 7. Data collection on press kinematics: A single-subject study. High-fidelity optical motion tracking was used to track a marker on the finger nail. A custom-made single-button setup was created using switches and key caps from commercial keyboards. SIMULATIONS: COMPARING BUTTON DESIGNS Most precise Less precise Least precise High success High success Low success Push- button Touch Mid-air Order supported by literature
  • 25. Data collection: A single-subject study “Press rhythmically in a manner natural for you”
  • 26. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Mid-air button: Kinematics [Torre & Balasubramaniam 2009] Human Model Similar to our data and literature
  • 27. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Dynamics Force-displacement curves Predicted peak forces 1.5-2.9N Force ranges similar to literature cle force–displacement behavior for a tactile with an effort-minimizing term in the objective task performance (performance in button activation). We co clude, that although much work remains to be done, the resul support the ’optimal black box’ assumption. And many mor analyses could done, such as looking at the effect of unreliab feedback, oscillation of the finger tip, such as when walkin or the effects that impairments like essential tremor have. FUTURE WORK Modeling latent neural and cognitive constructs, such as nois poses a scientific challenge for future research. Change i noise parameters has a large and poorly understood effect o dynamics downstream. However, without noise, a button ca be activated with arbitrary precision. For example, cuttin sensory noise parameters to 10 9 reduces perceptual error t the order of 1.5·10 6 s. Our noise model was tuned manuall to reproduce some standard findings on sensory modalities. T “Light touch” Force(N)
  • 28. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Touch buttons: Kinematics Displacement-velocity curves e 8. Displacement–velocity curves for four button types from single-subject recordings (top) and simulations (bottom Shape similar except in release Human Model Similar result for push-buttons
  • 29. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Signal-dependent noise Muscle and joint models Force perception Noise parameter identification Limitations
  • 30. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Neuro- science PhysicsPhysiology Machine learning Biomech. simulation EE and signal processing A unifying account A generative simulation Can it be made to work beyond buttons?
  • 31. bit.do/neuromechanics Antti Oulasvirta, Sunjun Kim, Byungjoo Lee Acknowledgements: Jong-In Lee, Aleksi Pesonen, Yunfei Xiu, and Crista Kaukinen Related papers at CHI 2018: 1. Control-theoretic models of Pointing Tue 9.00 517C 2. Impact Activation Improves Rapid Key Pressing Mon 16.30 514AB 3. Moving Target Selection: A Cue Integration Model This session Figure 8. Displacement–velocity curves for four button types from single-subject recordings (top) and simulations (bottom). Data Matlab codeRobot