2016 IEEE 5th Global Conference on Consumer Electronics(GCCE2016) Yuya Koyama, Michiko Nishiyama, Kazuhiro Watanabe Department of science and engineering, Soka University

1. Gait Monitoring for Human Activity
Recognition Using Perceptive Shoe Based
on Hetero-core Fiber Optics
Yuya Koyama, Michiko Nishiyama, Kazuhiro Watanabe
Department of science and engineering, Soka University

2. Contents
1. Introduction
2. Perceptive shoe using hetero-core fiber optics
3. Weight monitor during various activities
4. Features extraction and test
5. Conclusion

3. Background
Monitoring technologies for
human activities
Personal belongings constantly
support users instead of human
Position, motion
Message, alarm
Ubiquitous computing
Human should be not aware of the sensors
Human activity
・Walk
・Stand
・Run
・Sit
etc.
Human activities need to be
monitored without obtrusive
sensors for comfortable systems

4. Conventional techniques
Extraction from body images obtained by
the installed cameras in life spaces
Vision methods
OptiTrack
 occlusion problems
 expensive setting in infrastructures
Inertial sensors on the body segments
Wearable inertial sensors
L. Bao and S. S. Intille, "Activity recognition from user-annotated
acceleration data," in Proceedings of PERVASIVE 2004, vol. LNCS
3001, A. Ferscha and F. Mattern, Eds. Berlin Heidelberg:
Springer-Verlag, 2004, pp. 1-17.
Restriction to the human with
awareness of body worn sensors
The human activities can be recognized
from the motion information without
occlusion

5. Perceptive shoes
Human activity
・Walk
・Stand
・Run
・Sit
etc.
Monitoring
Purpose
The shoes with perception to human activity
 The sensing functions are fused with normal shoes
 No constraint to human body

6. Purpose
 Foot weight monitoring with perceptive shoes during
human activity in actual field
 Features extraction and determination of activity
Perceptive shoes：
combination shoes with hetero-core fiber optics
→Monitoring human activity in real time

7. Hetero-core optical fiber
Hetero-core portion
Cladding
125μm
Core 9μmCore5μm
Transmission lineTransmission line
Core 9μm
Features and advantages
Soft and light weight element
High sensitivity to soft bending on sensor portion
No need for temperature compensation
Real-time measurement based on optical intensity
Combination of shoes and hetero-core optical fiber
Shoes embedded
optical fiber
Soft bending detection by measuring optical intensity change with
LED/PD device

8. Plastic
sheet
Fiber line
10 [mm]
10 [mm]
2 [mm]
Shoes for monitoring weight changes
Insole
Weight sensor element
Hetero-core optical fiber
The sensors are
embedded in left insole
Perceptive shoes combined with
hetero-core fiber optics
Pressure
Cross-sectional view

9. 9
Optical fiber transmission line
Sampling rate: 50Hz
Data
Perceptive shoes
with hetero-core fiber optics
Weight
element-2
Weight
element-1
Communication
with Bluetooth
135L mm
76W mm
27T mm
Multi-channel optical data
acquisition device
(λ=1.31[mm])
Mobile device
with Android OS
134L mm
71W mm
12T mm
Weight element-1
Weight element-2
Insole
Optical fiber lines
Weight monitoring system

10. Activity monitoring test
Walking Walking on the spot
Running Standing
Sitting
Perceptive shoes
with hetero-core
fiber optics
Insole
Multi-channel optical data
acquisition device
Smart phone
Activities

11. Time [s]
Weight[N/cm2]
Time [s]
Weight[N/cm2]
Walking Walking on the spot
Walking:
Each weight increased in order from back to front
→the gravity center moves from heel to toe position
Walking on the spot:
Two weights changed at the same time
→No shift of the gravity center
Result for activity monitoring test

12. Running:
The time interval of foot flat was short compared to that of walking
Standing:
The weight responses did not change
Time [s]
Weight[N/cm2]
Running
Time [s]
Weight[N/cm2]
Standing
Result for activity monitoring test

13. Time [s]
Weight[N/cm2]
Sitting
Sitting:
 The weight responses did not change
 Small fluctuation
 The sum of responses smaller than that of standing
Different features in foot weight were successfully
monitored by the perceptive shoe in real time
Result for activity monitoring test

14. Features extraction
0
2
4
6
8
10
0 1 2 3 4
Time interval
↓
Standard deviation
-100
-50
0
50
100
-10
-5
0
5
10
Time
differentiation
Difference of
front and back weight
Variation
Max

15. Start
20 < dmax
5 < dmax0.5 < t
2 < sdf-b
Walking
Walking
on the spot
StandingRunning Sitting
Y N
Y N
Y
N
Y
N
Decision table and Flow chart
Activity dmax t sdf-b
Walking 20<dmax 0.5<t 2<sdf-b
Walking on the spot 20<dmax 0.5<t sdf-b<2
Standing 5<dmax<20 not applicable not applicable
Running 20<dmax t<0.5 not applicable
Sitting on a chair dmax<5 not applicable not applicable
Decision table
Flow chart
dmax: Maximum of time differentiation
t: Time interval
sdf-b: Standard deviation of front and back weight

16. 0
5
10
15
20
25
0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80
Weight[N/cm2]
Time [s]
Result of determination
Decision Running
on the spot
Walking
Sitting
Standing
Error for “Sitting”:
Judgement condition for sitting
Time differentiation → sum of weight
Error for “Walking” and “Running”:
Time intervals need to be adjusted
“Sitting” “Sitting” and “Running”“Sitting” “Walking”Error:
Activity

17. Conclusion
 Offers the performance of human daily activity detection
 Promises benefits for unconstrained tool
Perceptive shoes using hetero-core optical fiber
Weight based determination
 Provide practical information
 Allow extraction of features from five human activities
• Improve determination performance for various subjects
• Recognition for more complex activities
• Comparative testing against conventional inertial sensors
Future works

18. Thank you for your attention.