Muscle sensing can be used to provide complementary information about finger movement and posture for multi-touch table top surfaces.

1. Enhancing Input On and Above the
Interactive Surface with Muscle
Sensing
Hrvoje Benko, T. Scott Saponas, Dan Morris, and Desney Tan
Summarized & Presented by:
Reem Alattas

2. Combining Muscle and Touch Sensing
• Touch-sensitive surfaces and EMG provide
complementary streams of information.
– Touch-sensitive surfaces provide precise location
and tracking information.
– EMG can detect which muscle groups, and
consequently which fingers, are engaged in the
current interaction.

3. Hardware and Setup
• Microsoft Surface
• BioSemi Active Two EMG device
– Samples eight sensor channels at 2048 Hz
– 6 sensors and two ground electrodes around the
upper forearm of the dominant hand
– 2 sensors on the forearm of the non-dominant
hand

4. Interpretation of Muscle Signals
• Level of pressure
• Contact finger identification
• “Pinch” and “Throw” gestures
• “Flick” gesture

5. Hybrid EMG-Surface Interactions
• Pressure-sensitive painting
• Finger-aware painting
• Finger-dependent pick and throw
• Undo flick

6. Exploratory System Evaluation
• Participants: 6 (3 females)
• 90 minutes
• $10 compensation

7. Goals
• Feasibility Validation
• Reliability Assessment

8. Tasks
• Task 1
– Copy an image from a given paper template using the pressure-
sensitive painting technique
• Task 2
– Copy an image from a given paper template using the finger-aware
painting technique
• Task 3
– Make a series of vertical lines across the surface, changing color with
each vertical line
• Task 4
– Write the numbers from 1 to 10 on the surface, executing the “undo
flick” gesture after each even number, but not after odd numbers.
• Task 5
– Presented with a pile of six images on a canvas, either copy or move
each image to another canvas, depending on the image category.

9. Tasks

10. Results

11. Results
• Task 1 mean accuracy = 93.9%
– All participants were able to effectively manipulate pressure to
control brush darkness.
• Task 2:
– All six participants completed the target drawing.
– One had some difficulty reliably selecting the finger color.
• Task 3 mean accuracy = 90.9%
• Task 4:
– Five out of six participants were able to reliably execute and
control the “undo flick” gesture without any false positives.
• Task 5:
– Three perfect executions

12. Conclusion
• The proposed approach enhances the existing
tabletop paradigm and enables new
interaction techniques not typically possible
with standard interactive surfaces.

13. Questions