SlideShare a Scribd company logo

TouchLog: Finger Micro Gesture Recognition Using Photo-Reflective Sensors

ISWC2023

TouchLog: Finger Micro Gesture Recognition Using Photo-Reflective Sensors

1 of 15
Download to read offline
Riku Kitamura1) , Takumi Yamamoto1) , Yuta Sugiura1)
TouchLog
Finger Micro Gesture Recognition
Using Photo-Reflective Sensors
1)Keio University
• Human fingertips are dexterous and capable of precise movements
• Intuitive and diverse input capabilities
• Many wearable devices are equipped with fingertip gesture input
• Finger micro gestures (FMG) are gaining attention
2
Background
TipText: Capacitance sensing
[Xu, 2019]
Efring: Electric field sensing
[Chen, 2023]
Pyro: PIR sensors
[Gong, 2017]
• FMG using the thumb and index finger
• Social acceptability[1]
• Haptic Feedback
• Reduced fatigue
• Confidential input
• No environmental sound restrictions
• Input method that is one-handed and does not cover the fingertip
3
Motivation
[1] Radu-Daniel Vatavu, IFAD Gestures: Understanding Users’ Gesture Input Performance with Index-Finger Augmentation Devices, CHI’23, 2023
4
Our Approach
• Use fingernail-type device
• Fingertip skin deformation information is acquired
using 7 photo-reflective sensors
Gesture Input Method Utilizing Fingertip Skin Deformation
• Fingernail-type device
• 7 photo reflective sensors are placed on fingernail-shaped output by a 3D printer
• Sensors face the finger side
5
Implementation: Hardware
Sensor direction
• Record for 2.4 sec on average (160 frames) per gesture
• Feature extractions
• Time-series data was divided into 5 parts(with 32-frame intervals)
• Statistical features were extracted for each divided part
• Feature value of 175 dimensions(for 7 sensor values × 5 divisions × 5 statistical features)
• Random forest
6
Implementation: Software
1
2
3
4
5
6
7
Time-series data (160 frames)
Feature Extraction
・・・
Divide into 5 parts
s
Mean
Variance
Max
Min
Median

Recommended

Converting Tatamis into Touch Sensors by Measuring Capacitance
Converting Tatamis into Touch Sensors by Measuring CapacitanceConverting Tatamis into Touch Sensors by Measuring Capacitance
Converting Tatamis into Touch Sensors by Measuring Capacitancesugiuralab
 
Human Movement Recognition Using Internal Sensors of a Smartphone-based HMD (...
Human Movement Recognition Using Internal Sensors of a Smartphone-based HMD (...Human Movement Recognition Using Internal Sensors of a Smartphone-based HMD (...
Human Movement Recognition Using Internal Sensors of a Smartphone-based HMD (...sugiuralab
 
ReflecTouch - CHI 2022
ReflecTouch - CHI 2022ReflecTouch - CHI 2022
ReflecTouch - CHI 2022sugiuralab
 
Estimation of Fingertip Contact Force by Measuring Skin Deformation and Postu...
Estimation of Fingertip Contact Force by Measuring Skin Deformation and Postu...Estimation of Fingertip Contact Force by Measuring Skin Deformation and Postu...
Estimation of Fingertip Contact Force by Measuring Skin Deformation and Postu...sugiuralab
 
Skinput technology
Skinput technologySkinput technology
Skinput technologyneha pevekar
 
Skinput
SkinputSkinput
Skinputaxme86
 
Video-Based Hand Tracking for Screening Cervical Myelopathy (ISVC2021)
Video-Based Hand Tracking for Screening Cervical Myelopathy (ISVC2021)Video-Based Hand Tracking for Screening Cervical Myelopathy (ISVC2021)
Video-Based Hand Tracking for Screening Cervical Myelopathy (ISVC2021)sugiuralab
 
Hand geometry recognition: an approach for closed and separated fingers
Hand geometry recognition: an approach for closed and  separated fingersHand geometry recognition: an approach for closed and  separated fingers
Hand geometry recognition: an approach for closed and separated fingersIJECEIAES
 

More Related Content

Similar to TouchLog: Finger Micro Gesture Recognition Using Photo-Reflective Sensors

Identification and Authentication Using Clavicles
Identification and Authentication Using ClaviclesIdentification and Authentication Using Clavicles
Identification and Authentication Using Claviclessugiuralab
 
Wearable technologies: what's brewing in the lab?
Wearable technologies: what's brewing in the lab?Wearable technologies: what's brewing in the lab?
Wearable technologies: what's brewing in the lab?Daniel Roggen
 
Multimodal Multi-sensory Interaction for Mixed Reality
Multimodal Multi-sensory Interaction for Mixed RealityMultimodal Multi-sensory Interaction for Mixed Reality
Multimodal Multi-sensory Interaction for Mixed RealityMark Billinghurst
 
Personal Identification using Gait Data on Slipper-device with Accelerometer ...
Personal Identification using Gait Data on Slipper-device with Accelerometer ...Personal Identification using Gait Data on Slipper-device with Accelerometer ...
Personal Identification using Gait Data on Slipper-device with Accelerometer ...sugiuralab
 
An ontology for reasoning on body-based gestures
 An ontology for reasoning on body-based gestures An ontology for reasoning on body-based gestures
An ontology for reasoning on body-based gesturesJean Vanderdonckt
 
Advanced Interaction Techniques for Medical Intervention and Diagnosis
Advanced Interaction Techniques for Medical Intervention and DiagnosisAdvanced Interaction Techniques for Medical Intervention and Diagnosis
Advanced Interaction Techniques for Medical Intervention and DiagnosisLars Jacobs
 
Masktrap: Designing and Identifying Gestures to Transform Mask Strap into an ...
Masktrap: Designing and Identifying Gestures to Transform Mask Strap into an ...Masktrap: Designing and Identifying Gestures to Transform Mask Strap into an ...
Masktrap: Designing and Identifying Gestures to Transform Mask Strap into an ...sugiuralab
 
Pattern recognition Hand Geometry
Pattern recognition Hand GeometryPattern recognition Hand Geometry
Pattern recognition Hand GeometryMazin Alwaaly
 
A systematic approach towards designing low-cost motor and cognitive rehabili...
A systematic approach towards designing low-cost motor and cognitive rehabili...A systematic approach towards designing low-cost motor and cognitive rehabili...
A systematic approach towards designing low-cost motor and cognitive rehabili...Sergi Bermudez i Badia
 
COMP 4010 Lecture12 - Research Directions in AR and VR
COMP 4010 Lecture12 - Research Directions in AR and VRCOMP 4010 Lecture12 - Research Directions in AR and VR
COMP 4010 Lecture12 - Research Directions in AR and VRMark Billinghurst
 
Knee Osteoarthritis Classification System Examination on Wearable Daily-Use I...
Knee Osteoarthritis Classification System Examination on Wearable Daily-Use I...Knee Osteoarthritis Classification System Examination on Wearable Daily-Use I...
Knee Osteoarthritis Classification System Examination on Wearable Daily-Use I...sugiuralab
 
Sensor Based Ambient Assisted Living
Sensor Based Ambient Assisted LivingSensor Based Ambient Assisted Living
Sensor Based Ambient Assisted LivingYiannis Kompatsiaris
 
Performance and Ergonomics of Touch Surfaces: A Comparative Study using Bio...
Performance and Ergonomics of Touch Surfaces: A Comparative Study using Bio...Performance and Ergonomics of Touch Surfaces: A Comparative Study using Bio...
Performance and Ergonomics of Touch Surfaces: A Comparative Study using Bio...Myroslav Bachynskyi
 
Sign language recognition for deaf and dumb people
Sign language recognition for deaf and dumb peopleSign language recognition for deaf and dumb people
Sign language recognition for deaf and dumb peopleeSAT Publishing House
 
COMP 4010 Lecture10 AR/VR Research Directions
COMP 4010 Lecture10 AR/VR Research DirectionsCOMP 4010 Lecture10 AR/VR Research Directions
COMP 4010 Lecture10 AR/VR Research DirectionsMark Billinghurst
 
Work completion seminar defence
Work completion seminar defenceWork completion seminar defence
Work completion seminar defenceMahdi Babaei
 
The-Digitalization-of-the-Walking-Stick-for-the-Blind
The-Digitalization-of-the-Walking-Stick-for-the-BlindThe-Digitalization-of-the-Walking-Stick-for-the-Blind
The-Digitalization-of-the-Walking-Stick-for-the-BlindVandan Patel
 
Elderly activity recognition and classification for application in assisted l...
Elderly activity recognition and classification for application in assisted l...Elderly activity recognition and classification for application in assisted l...
Elderly activity recognition and classification for application in assisted l...Sara Egidi
 
Case Study - TapAssist
Case Study - TapAssistCase Study - TapAssist
Case Study - TapAssistGrace Chien
 

Similar to TouchLog: Finger Micro Gesture Recognition Using Photo-Reflective Sensors (20)

Identification and Authentication Using Clavicles
Identification and Authentication Using ClaviclesIdentification and Authentication Using Clavicles
Identification and Authentication Using Clavicles
 
Wearable technologies: what's brewing in the lab?
Wearable technologies: what's brewing in the lab?Wearable technologies: what's brewing in the lab?
Wearable technologies: what's brewing in the lab?
 
Multimodal Multi-sensory Interaction for Mixed Reality
Multimodal Multi-sensory Interaction for Mixed RealityMultimodal Multi-sensory Interaction for Mixed Reality
Multimodal Multi-sensory Interaction for Mixed Reality
 
Personal Identification using Gait Data on Slipper-device with Accelerometer ...
Personal Identification using Gait Data on Slipper-device with Accelerometer ...Personal Identification using Gait Data on Slipper-device with Accelerometer ...
Personal Identification using Gait Data on Slipper-device with Accelerometer ...
 
An ontology for reasoning on body-based gestures
 An ontology for reasoning on body-based gestures An ontology for reasoning on body-based gestures
An ontology for reasoning on body-based gestures
 
Advanced Interaction Techniques for Medical Intervention and Diagnosis
Advanced Interaction Techniques for Medical Intervention and DiagnosisAdvanced Interaction Techniques for Medical Intervention and Diagnosis
Advanced Interaction Techniques for Medical Intervention and Diagnosis
 
Masktrap: Designing and Identifying Gestures to Transform Mask Strap into an ...
Masktrap: Designing and Identifying Gestures to Transform Mask Strap into an ...Masktrap: Designing and Identifying Gestures to Transform Mask Strap into an ...
Masktrap: Designing and Identifying Gestures to Transform Mask Strap into an ...
 
Pattern recognition Hand Geometry
Pattern recognition Hand GeometryPattern recognition Hand Geometry
Pattern recognition Hand Geometry
 
A systematic approach towards designing low-cost motor and cognitive rehabili...
A systematic approach towards designing low-cost motor and cognitive rehabili...A systematic approach towards designing low-cost motor and cognitive rehabili...
A systematic approach towards designing low-cost motor and cognitive rehabili...
 
COMP 4010 Lecture12 - Research Directions in AR and VR
COMP 4010 Lecture12 - Research Directions in AR and VRCOMP 4010 Lecture12 - Research Directions in AR and VR
COMP 4010 Lecture12 - Research Directions in AR and VR
 
Skinput
SkinputSkinput
Skinput
 
Knee Osteoarthritis Classification System Examination on Wearable Daily-Use I...
Knee Osteoarthritis Classification System Examination on Wearable Daily-Use I...Knee Osteoarthritis Classification System Examination on Wearable Daily-Use I...
Knee Osteoarthritis Classification System Examination on Wearable Daily-Use I...
 
Sensor Based Ambient Assisted Living
Sensor Based Ambient Assisted LivingSensor Based Ambient Assisted Living
Sensor Based Ambient Assisted Living
 
Performance and Ergonomics of Touch Surfaces: A Comparative Study using Bio...
Performance and Ergonomics of Touch Surfaces: A Comparative Study using Bio...Performance and Ergonomics of Touch Surfaces: A Comparative Study using Bio...
Performance and Ergonomics of Touch Surfaces: A Comparative Study using Bio...
 
Sign language recognition for deaf and dumb people
Sign language recognition for deaf and dumb peopleSign language recognition for deaf and dumb people
Sign language recognition for deaf and dumb people
 
COMP 4010 Lecture10 AR/VR Research Directions
COMP 4010 Lecture10 AR/VR Research DirectionsCOMP 4010 Lecture10 AR/VR Research Directions
COMP 4010 Lecture10 AR/VR Research Directions
 
Work completion seminar defence
Work completion seminar defenceWork completion seminar defence
Work completion seminar defence
 
The-Digitalization-of-the-Walking-Stick-for-the-Blind
The-Digitalization-of-the-Walking-Stick-for-the-BlindThe-Digitalization-of-the-Walking-Stick-for-the-Blind
The-Digitalization-of-the-Walking-Stick-for-the-Blind
 
Elderly activity recognition and classification for application in assisted l...
Elderly activity recognition and classification for application in assisted l...Elderly activity recognition and classification for application in assisted l...
Elderly activity recognition and classification for application in assisted l...
 
Case Study - TapAssist
Case Study - TapAssistCase Study - TapAssist
Case Study - TapAssist
 

More from sugiuralab

Pinch Force Measurement Using a Geomagnetic Sensor
Pinch Force Measurement Using a Geomagnetic SensorPinch Force Measurement Using a Geomagnetic Sensor
Pinch Force Measurement Using a Geomagnetic Sensorsugiuralab
 
Smartphone-Based Teaching System for Neonate Soothing Motions
Smartphone-Based Teaching System for Neonate Soothing MotionsSmartphone-Based Teaching System for Neonate Soothing Motions
Smartphone-Based Teaching System for Neonate Soothing Motionssugiuralab
 
Tactile Presentation of Orchestral Conductor's Motion Trajectory
Tactile Presentation of Orchestral Conductor's Motion TrajectoryTactile Presentation of Orchestral Conductor's Motion Trajectory
Tactile Presentation of Orchestral Conductor's Motion Trajectorysugiuralab
 
Seeing the Wind: An Interactive Mist Interface for Airflow Input
Seeing the Wind: An Interactive Mist Interface for Airflow InputSeeing the Wind: An Interactive Mist Interface for Airflow Input
Seeing the Wind: An Interactive Mist Interface for Airflow Inputsugiuralab
 
Estimation of Violin Bow Pressure Using Photo-Reflective Sensors
Estimation of Violin Bow Pressure Using Photo-Reflective SensorsEstimation of Violin Bow Pressure Using Photo-Reflective Sensors
Estimation of Violin Bow Pressure Using Photo-Reflective Sensorssugiuralab
 
バウンサーを動かす外付けデバイス
バウンサーを動かす外付けデバイスバウンサーを動かす外付けデバイス
バウンサーを動かす外付けデバイスsugiuralab
 
A Virtual Window Using Curtains and Image Projection
A Virtual Window Using Curtains and Image ProjectionA Virtual Window Using Curtains and Image Projection
A Virtual Window Using Curtains and Image Projectionsugiuralab
 
スマートフォンを用いた新生児あやし動作の教示システム
スマートフォンを用いた新生児あやし動作の教示システムスマートフォンを用いた新生児あやし動作の教示システム
スマートフォンを用いた新生児あやし動作の教示システムsugiuralab
 
指側面を入力インタフェース化するリング型デバイス
指側面を入力インタフェース化するリング型デバイス指側面を入力インタフェース化するリング型デバイス
指側面を入力インタフェース化するリング型デバイスsugiuralab
 
ヒアラブルデバイスにおける音漏れ信号を用いた空中ジェスチャ認識
ヒアラブルデバイスにおける音漏れ信号を用いた空中ジェスチャ認識ヒアラブルデバイスにおける音漏れ信号を用いた空中ジェスチャ認識
ヒアラブルデバイスにおける音漏れ信号を用いた空中ジェスチャ認識sugiuralab
 
磁石内臓イヤリングによる磁力変化を利用したジェスチャ識別
磁石内臓イヤリングによる磁力変化を利用したジェスチャ識別磁石内臓イヤリングによる磁力変化を利用したジェスチャ識別
磁石内臓イヤリングによる磁力変化を利用したジェスチャ識別sugiuralab
 
Maintenance-Free Smart Hand Dynamometer.pdf
Maintenance-Free Smart Hand Dynamometer.pdfMaintenance-Free Smart Hand Dynamometer.pdf
Maintenance-Free Smart Hand Dynamometer.pdfsugiuralab
 
Augmented Sports of Badminton by Changing Opening Status of Shuttle’s Feathers
Augmented Sports of Badminton by Changing Opening Status of Shuttle’s FeathersAugmented Sports of Badminton by Changing Opening Status of Shuttle’s Feathers
Augmented Sports of Badminton by Changing Opening Status of Shuttle’s Featherssugiuralab
 
指腹開放型入力デバイス における文字ジェスチャ識別
指腹開放型入力デバイス における文字ジェスチャ識別指腹開放型入力デバイス における文字ジェスチャ識別
指腹開放型入力デバイス における文字ジェスチャ識別sugiuralab
 
Carpal Tunnel Syndrome Estimation through Median Nerve Segmentation in Ultr...
Carpal Tunnel Syndrome Estimation  through Median Nerve Segmentation  in Ultr...Carpal Tunnel Syndrome Estimation  through Median Nerve Segmentation  in Ultr...
Carpal Tunnel Syndrome Estimation through Median Nerve Segmentation in Ultr...sugiuralab
 
静電容量の測定による畳のタッチセンサ化手法の提案
静電容量の測定による畳のタッチセンサ化手法の提案静電容量の測定による畳のタッチセンサ化手法の提案
静電容量の測定による畳のタッチセンサ化手法の提案sugiuralab
 
Virtual IMU Data Augmentation by Spring-Joint Model for Motion Exercises Reco...
Virtual IMU Data Augmentation by Spring-Joint Model for Motion Exercises Reco...Virtual IMU Data Augmentation by Spring-Joint Model for Motion Exercises Reco...
Virtual IMU Data Augmentation by Spring-Joint Model for Motion Exercises Reco...sugiuralab
 
反射型光センサを用いた指先のジェスチャ識別
反射型光センサを用いた指先のジェスチャ識別反射型光センサを用いた指先のジェスチャ識別
反射型光センサを用いた指先のジェスチャ識別sugiuralab
 
スマートフォンの座標変化による握力測定手法の提案.pdf
スマートフォンの座標変化による握力測定手法の提案.pdfスマートフォンの座標変化による握力測定手法の提案.pdf
スマートフォンの座標変化による握力測定手法の提案.pdfsugiuralab
 
Video Analysis of Hand Gestures for Distinguishing Patients with Carpal Tunne...
Video Analysis of Hand Gestures for Distinguishing Patients with Carpal Tunne...Video Analysis of Hand Gestures for Distinguishing Patients with Carpal Tunne...
Video Analysis of Hand Gestures for Distinguishing Patients with Carpal Tunne...sugiuralab
 

More from sugiuralab (20)

Pinch Force Measurement Using a Geomagnetic Sensor
Pinch Force Measurement Using a Geomagnetic SensorPinch Force Measurement Using a Geomagnetic Sensor
Pinch Force Measurement Using a Geomagnetic Sensor
 
Smartphone-Based Teaching System for Neonate Soothing Motions
Smartphone-Based Teaching System for Neonate Soothing MotionsSmartphone-Based Teaching System for Neonate Soothing Motions
Smartphone-Based Teaching System for Neonate Soothing Motions
 
Tactile Presentation of Orchestral Conductor's Motion Trajectory
Tactile Presentation of Orchestral Conductor's Motion TrajectoryTactile Presentation of Orchestral Conductor's Motion Trajectory
Tactile Presentation of Orchestral Conductor's Motion Trajectory
 
Seeing the Wind: An Interactive Mist Interface for Airflow Input
Seeing the Wind: An Interactive Mist Interface for Airflow InputSeeing the Wind: An Interactive Mist Interface for Airflow Input
Seeing the Wind: An Interactive Mist Interface for Airflow Input
 
Estimation of Violin Bow Pressure Using Photo-Reflective Sensors
Estimation of Violin Bow Pressure Using Photo-Reflective SensorsEstimation of Violin Bow Pressure Using Photo-Reflective Sensors
Estimation of Violin Bow Pressure Using Photo-Reflective Sensors
 
バウンサーを動かす外付けデバイス
バウンサーを動かす外付けデバイスバウンサーを動かす外付けデバイス
バウンサーを動かす外付けデバイス
 
A Virtual Window Using Curtains and Image Projection
A Virtual Window Using Curtains and Image ProjectionA Virtual Window Using Curtains and Image Projection
A Virtual Window Using Curtains and Image Projection
 
スマートフォンを用いた新生児あやし動作の教示システム
スマートフォンを用いた新生児あやし動作の教示システムスマートフォンを用いた新生児あやし動作の教示システム
スマートフォンを用いた新生児あやし動作の教示システム
 
指側面を入力インタフェース化するリング型デバイス
指側面を入力インタフェース化するリング型デバイス指側面を入力インタフェース化するリング型デバイス
指側面を入力インタフェース化するリング型デバイス
 
ヒアラブルデバイスにおける音漏れ信号を用いた空中ジェスチャ認識
ヒアラブルデバイスにおける音漏れ信号を用いた空中ジェスチャ認識ヒアラブルデバイスにおける音漏れ信号を用いた空中ジェスチャ認識
ヒアラブルデバイスにおける音漏れ信号を用いた空中ジェスチャ認識
 
磁石内臓イヤリングによる磁力変化を利用したジェスチャ識別
磁石内臓イヤリングによる磁力変化を利用したジェスチャ識別磁石内臓イヤリングによる磁力変化を利用したジェスチャ識別
磁石内臓イヤリングによる磁力変化を利用したジェスチャ識別
 
Maintenance-Free Smart Hand Dynamometer.pdf
Maintenance-Free Smart Hand Dynamometer.pdfMaintenance-Free Smart Hand Dynamometer.pdf
Maintenance-Free Smart Hand Dynamometer.pdf
 
Augmented Sports of Badminton by Changing Opening Status of Shuttle’s Feathers
Augmented Sports of Badminton by Changing Opening Status of Shuttle’s FeathersAugmented Sports of Badminton by Changing Opening Status of Shuttle’s Feathers
Augmented Sports of Badminton by Changing Opening Status of Shuttle’s Feathers
 
指腹開放型入力デバイス における文字ジェスチャ識別
指腹開放型入力デバイス における文字ジェスチャ識別指腹開放型入力デバイス における文字ジェスチャ識別
指腹開放型入力デバイス における文字ジェスチャ識別
 
Carpal Tunnel Syndrome Estimation through Median Nerve Segmentation in Ultr...
Carpal Tunnel Syndrome Estimation  through Median Nerve Segmentation  in Ultr...Carpal Tunnel Syndrome Estimation  through Median Nerve Segmentation  in Ultr...
Carpal Tunnel Syndrome Estimation through Median Nerve Segmentation in Ultr...
 
静電容量の測定による畳のタッチセンサ化手法の提案
静電容量の測定による畳のタッチセンサ化手法の提案静電容量の測定による畳のタッチセンサ化手法の提案
静電容量の測定による畳のタッチセンサ化手法の提案
 
Virtual IMU Data Augmentation by Spring-Joint Model for Motion Exercises Reco...
Virtual IMU Data Augmentation by Spring-Joint Model for Motion Exercises Reco...Virtual IMU Data Augmentation by Spring-Joint Model for Motion Exercises Reco...
Virtual IMU Data Augmentation by Spring-Joint Model for Motion Exercises Reco...
 
反射型光センサを用いた指先のジェスチャ識別
反射型光センサを用いた指先のジェスチャ識別反射型光センサを用いた指先のジェスチャ識別
反射型光センサを用いた指先のジェスチャ識別
 
スマートフォンの座標変化による握力測定手法の提案.pdf
スマートフォンの座標変化による握力測定手法の提案.pdfスマートフォンの座標変化による握力測定手法の提案.pdf
スマートフォンの座標変化による握力測定手法の提案.pdf
 
Video Analysis of Hand Gestures for Distinguishing Patients with Carpal Tunne...
Video Analysis of Hand Gestures for Distinguishing Patients with Carpal Tunne...Video Analysis of Hand Gestures for Distinguishing Patients with Carpal Tunne...
Video Analysis of Hand Gestures for Distinguishing Patients with Carpal Tunne...
 

Recently uploaded

LLMs, LMMs, their Improvement Suggestions and the Path towards AGI.pdf
LLMs, LMMs, their Improvement Suggestions and the Path towards AGI.pdfLLMs, LMMs, their Improvement Suggestions and the Path towards AGI.pdf
LLMs, LMMs, their Improvement Suggestions and the Path towards AGI.pdfThomas Poetter
 
From eSIMs to iSIMs: It’s Inside the Manufacturing
From eSIMs to iSIMs: It’s Inside the ManufacturingFrom eSIMs to iSIMs: It’s Inside the Manufacturing
From eSIMs to iSIMs: It’s Inside the ManufacturingSoracom Global, Inc.
 
My self introduction to know others abut me
My self  introduction to know others abut meMy self  introduction to know others abut me
My self introduction to know others abut meManoj Prabakar B
 
Importance of magazines in education ppt
Importance of magazines in education pptImportance of magazines in education ppt
Importance of magazines in education pptsafnarafeek2002
 
Q1 Memory Fabric Forum: SMART CXL Product Lineup
Q1 Memory Fabric Forum: SMART CXL Product LineupQ1 Memory Fabric Forum: SMART CXL Product Lineup
Q1 Memory Fabric Forum: SMART CXL Product LineupMemory Fabric Forum
 
Breaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI TechnologyBreaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI TechnologySafe Software
 
Bit N Build Poland
Bit N Build PolandBit N Build Poland
Bit N Build PolandGDSC PJATK
 
Heltun_HE-RS01_User_Manual_B9AH.pdf
Heltun_HE-RS01_User_Manual_B9AH.pdfHeltun_HE-RS01_User_Manual_B9AH.pdf
Heltun_HE-RS01_User_Manual_B9AH.pdfMarielaL5
 
Journey of Television in World & in India
Journey of Television in World & in IndiaJourney of Television in World & in India
Journey of Television in World & in IndiaAdarshAgarwal66
 
Bluetooth Low Energy(BLE) and beacons working
Bluetooth Low Energy(BLE) and beacons workingBluetooth Low Energy(BLE) and beacons working
Bluetooth Low Energy(BLE) and beacons workingshrey Ansh
 
21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN
21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN
21ST CENTURY LITERACY FROM TRADITIONAL TO MODERNRonnelBaroc
 
Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...
Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...
Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...Adrian Sanabria
 
Unlocking the Cloud's True Potential: Why Multitenancy Is The Key?
Unlocking the Cloud's True Potential: Why Multitenancy Is The Key?Unlocking the Cloud's True Potential: Why Multitenancy Is The Key?
Unlocking the Cloud's True Potential: Why Multitenancy Is The Key?GleecusTechlabs1
 
OTel Orientation_ How to Train Teams (OTel in Practice).pdf
OTel Orientation_ How to Train Teams (OTel in Practice).pdfOTel Orientation_ How to Train Teams (OTel in Practice).pdf
OTel Orientation_ How to Train Teams (OTel in Practice).pdfPaige Cruz
 
AWS reInvent 2023 recaps from Chicago AWS user group
AWS reInvent 2023 recaps from Chicago AWS user groupAWS reInvent 2023 recaps from Chicago AWS user group
AWS reInvent 2023 recaps from Chicago AWS user groupAWS Chicago
 
Navigating the Never Normal Strategies for Portfolio Leaders
Navigating the Never Normal Strategies for Portfolio LeadersNavigating the Never Normal Strategies for Portfolio Leaders
Navigating the Never Normal Strategies for Portfolio LeadersOnePlan Solutions
 
Curtain Module Manual Zigbee Neo CS01-1C.pdf
Curtain Module Manual Zigbee Neo CS01-1C.pdfCurtain Module Manual Zigbee Neo CS01-1C.pdf
Curtain Module Manual Zigbee Neo CS01-1C.pdfDomotica daVinci
 
Z-Wave Fan coil Thermostat Heltun_HE-HT01_User_Manual.pdf
Z-Wave Fan coil Thermostat Heltun_HE-HT01_User_Manual.pdfZ-Wave Fan coil Thermostat Heltun_HE-HT01_User_Manual.pdf
Z-Wave Fan coil Thermostat Heltun_HE-HT01_User_Manual.pdfDomotica daVinci
 
Introduction to Serverless with AWS Lambda in C#.pptx
Introduction to Serverless with AWS Lambda in C#.pptxIntroduction to Serverless with AWS Lambda in C#.pptx
Introduction to Serverless with AWS Lambda in C#.pptxBrandon Minnick, MBA
 

Recently uploaded (20)

LLMs, LMMs, their Improvement Suggestions and the Path towards AGI.pdf
LLMs, LMMs, their Improvement Suggestions and the Path towards AGI.pdfLLMs, LMMs, their Improvement Suggestions and the Path towards AGI.pdf
LLMs, LMMs, their Improvement Suggestions and the Path towards AGI.pdf
 
From eSIMs to iSIMs: It’s Inside the Manufacturing
From eSIMs to iSIMs: It’s Inside the ManufacturingFrom eSIMs to iSIMs: It’s Inside the Manufacturing
From eSIMs to iSIMs: It’s Inside the Manufacturing
 
My self introduction to know others abut me
My self  introduction to know others abut meMy self  introduction to know others abut me
My self introduction to know others abut me
 
Importance of magazines in education ppt
Importance of magazines in education pptImportance of magazines in education ppt
Importance of magazines in education ppt
 
Q1 Memory Fabric Forum: SMART CXL Product Lineup
Q1 Memory Fabric Forum: SMART CXL Product LineupQ1 Memory Fabric Forum: SMART CXL Product Lineup
Q1 Memory Fabric Forum: SMART CXL Product Lineup
 
Breaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI TechnologyBreaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI Technology
 
Bit N Build Poland
Bit N Build PolandBit N Build Poland
Bit N Build Poland
 
Heltun_HE-RS01_User_Manual_B9AH.pdf
Heltun_HE-RS01_User_Manual_B9AH.pdfHeltun_HE-RS01_User_Manual_B9AH.pdf
Heltun_HE-RS01_User_Manual_B9AH.pdf
 
Journey of Television in World & in India
Journey of Television in World & in IndiaJourney of Television in World & in India
Journey of Television in World & in India
 
Bluetooth Low Energy(BLE) and beacons working
Bluetooth Low Energy(BLE) and beacons workingBluetooth Low Energy(BLE) and beacons working
Bluetooth Low Energy(BLE) and beacons working
 
COE AI Lab Universities
COE AI Lab UniversitiesCOE AI Lab Universities
COE AI Lab Universities
 
21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN
21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN
21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN
 
Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...
Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...
Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...
 
Unlocking the Cloud's True Potential: Why Multitenancy Is The Key?
Unlocking the Cloud's True Potential: Why Multitenancy Is The Key?Unlocking the Cloud's True Potential: Why Multitenancy Is The Key?
Unlocking the Cloud's True Potential: Why Multitenancy Is The Key?
 
OTel Orientation_ How to Train Teams (OTel in Practice).pdf
OTel Orientation_ How to Train Teams (OTel in Practice).pdfOTel Orientation_ How to Train Teams (OTel in Practice).pdf
OTel Orientation_ How to Train Teams (OTel in Practice).pdf
 
AWS reInvent 2023 recaps from Chicago AWS user group
AWS reInvent 2023 recaps from Chicago AWS user groupAWS reInvent 2023 recaps from Chicago AWS user group
AWS reInvent 2023 recaps from Chicago AWS user group
 
Navigating the Never Normal Strategies for Portfolio Leaders
Navigating the Never Normal Strategies for Portfolio LeadersNavigating the Never Normal Strategies for Portfolio Leaders
Navigating the Never Normal Strategies for Portfolio Leaders
 
Curtain Module Manual Zigbee Neo CS01-1C.pdf
Curtain Module Manual Zigbee Neo CS01-1C.pdfCurtain Module Manual Zigbee Neo CS01-1C.pdf
Curtain Module Manual Zigbee Neo CS01-1C.pdf
 
Z-Wave Fan coil Thermostat Heltun_HE-HT01_User_Manual.pdf
Z-Wave Fan coil Thermostat Heltun_HE-HT01_User_Manual.pdfZ-Wave Fan coil Thermostat Heltun_HE-HT01_User_Manual.pdf
Z-Wave Fan coil Thermostat Heltun_HE-HT01_User_Manual.pdf
 
Introduction to Serverless with AWS Lambda in C#.pptx
Introduction to Serverless with AWS Lambda in C#.pptxIntroduction to Serverless with AWS Lambda in C#.pptx
Introduction to Serverless with AWS Lambda in C#.pptx
 

TouchLog: Finger Micro Gesture Recognition Using Photo-Reflective Sensors

  • 1. Riku Kitamura1) , Takumi Yamamoto1) , Yuta Sugiura1) TouchLog Finger Micro Gesture Recognition Using Photo-Reflective Sensors 1)Keio University
  • 2. • Human fingertips are dexterous and capable of precise movements • Intuitive and diverse input capabilities • Many wearable devices are equipped with fingertip gesture input • Finger micro gestures (FMG) are gaining attention 2 Background TipText: Capacitance sensing [Xu, 2019] Efring: Electric field sensing [Chen, 2023] Pyro: PIR sensors [Gong, 2017]
  • 3. • FMG using the thumb and index finger • Social acceptability[1] • Haptic Feedback • Reduced fatigue • Confidential input • No environmental sound restrictions • Input method that is one-handed and does not cover the fingertip 3 Motivation [1] Radu-Daniel Vatavu, IFAD Gestures: Understanding Users’ Gesture Input Performance with Index-Finger Augmentation Devices, CHI’23, 2023
  • 4. 4 Our Approach • Use fingernail-type device • Fingertip skin deformation information is acquired using 7 photo-reflective sensors Gesture Input Method Utilizing Fingertip Skin Deformation
  • 5. • Fingernail-type device • 7 photo reflective sensors are placed on fingernail-shaped output by a 3D printer • Sensors face the finger side 5 Implementation: Hardware Sensor direction
  • 6. • Record for 2.4 sec on average (160 frames) per gesture • Feature extractions • Time-series data was divided into 5 parts(with 32-frame intervals) • Statistical features were extracted for each divided part • Feature value of 175 dimensions(for 7 sensor values × 5 divisions × 5 statistical features) • Random forest 6 Implementation: Software 1 2 3 4 5 6 7 Time-series data (160 frames) Feature Extraction ・・・ Divide into 5 parts s Mean Variance Max Min Median
  • 7. • Selected from FMG related work[2,3] • Ensure diversity of gestures 7 Gesture Set Gesture sets [2] Taizhou Chen, EFRing: Enabling Thumb-to-Index-Finger Microgesture Interaction through Electric Field Sensing Using Single Smart Ring, IMWUT, 2023 [3] Jung Gong, Pyro: Thumb-Tip Gesture Recognition Using Pyroelectric Infrared Sensing, UIST’17, 2017
  • 8. • General Model • Individual Model • LOOCV (Leave One Out Cross Validation) Model 8 Three Models for Identification Accuracy 9 user’s data 1 user’s data × 10 participants One user’s data Training data Test data 90% 10% × 10 participants Training data Test data All user’s data Training data Test data 90% 10%
  • 9. • 10 right-handed participants wore the device on their right index fingers • 10 participants × 11 gestures × 20 times = 2200 data • Before the experiment, each gesture was practiced • No device removal during experiment 9 Experiment Count of participants (Male/Female) 10 (5/5) Average age 22.4(SD = 1.57) Dominant hand Right Information of Participants
  • 10. • General Model : 91.1% • Individual Model: 91.5% (SD = 3.1%) • LOOCV Model : 67.6% (SD = 14.6%) 10 Results
  • 11. • General Model : 91.1% • Individual Model: 91.5% (SD = 3.1%) • LOOCV Model : 67.6% (SD = 14.6%) 11 Results
  • 12. • Continuous input 12 Discussion: Other Input Possibilities
  • 13. • Contact pressure input 13 Discussion: Other Input Possibilities
  • 14. • User dependence • Finger size, dexterity, fingertip skin condition • Investigate multiple device sizes, fingertip humidity and softness • Use in a real environment • Effects of sunlight • Use the device while walking or running • Conduct verification under conditions similar to real-world environments 14 Limitations and Future Work
  • 15. 15 Summary Background Human fingertips are dexterous and capable of fine movements Motivation Input method that is one-handed and does not cover the belly of the fingers Related work Loss of finger belly perception Photo reflective sensors are used for skin deformation acquisition Our approach Gesture input method utilizing fingertip skin deformation Implementation Device using photo-reflective sensors Classify 11 gestures using Random forest Evaluation General Model: 91.1% Individual Model: 91.5% LOOCV Model: 67.6% Future work Device that can be worn comfortably in daily life