1. Navigation and Instrumentation Research Group
Mostafa Elhoushi
Ph.D. Candidate, Queen’s University, Kingston, Canada
Department of Electrical
& Computer Engineering
PhD Thesis Defense
Advanced Motion Mode Recognition for Portable
Navigation
2. • This work was done under collaboration with
Trusted Positioning Inc. (TPI) - later acquired
by InvenSense Inc. - and Royal Military
College (RMC) of Canada through funds from
Mitacs, NSERC CRD grant, and TPI.
1/8/2016 NavINST Group - Queen's University 2
3. Contents
• Introduction
– Background
– Motivation
– Problem Statement
– Objective
– Prior Work
– Contributions
• Proposed Methodology
• Experimental Results
• Summary
1/8/2016 NavINST Group - Queen's University 3
4. Introduction
Background
• Navigation:
– the techniques of determining the position, velocity, and
attitude of a moving body
• Portable Navigation:
– the modified navigation techniques for a person moving
using a portable navigation device
• Applications of Portable Navigation:
– cell phone navigation for trips
– cell phone localization for advertising
– dismounted soldier and first responder localization
– handheld portable surveying systems in urban
environments
1/8/2016 NavINST Group - Queen's University 4
5. Introduction
Portable Navigation
• GNSS is very accurate BUT:
– Not available indoors and urban canyons
– Power consuming
• Other absolute positioning methods are less
accurate and need infrastructure
• Other relative positioning methods use sensors
which are erroneous
• Solution: Sensor Fusion
1/8/2016 NavINST Group - Queen's University 5
WiFiRFID Gyroscopes
Accelerometers
Magnetometer
GNSS
Barometer
Cell towers
Relative MeasurementsAbsolute Measurements
6. Introduction
Motivation
• Portable navigation is becoming increasingly popular
• Accurate, reliable, cost effective personal positioning is
needed
• There is a need for seamless outdoor/indoor portable
navigation
• A variety of commercial and military applications need
such technology
1/8/2016 NavINST Group - Queen's University 6
7. Introduction
Problem Statement
• The portable device (e.g., cell phone) is
untethered
– able to move freely without constraints within
another moving platform (person or vehicle)
• The system has to be environment
independent, i.e., work seamlessly
outdoor/indoor/urban
• The system has to work autonomously in
different modes of transit (e.g., walking,
driving, train, elevator, …)
1/8/2016 NavINST Group - Queen's University 7
8. Introduction
Objective
• Automatically detect the following Motion
Modes of a portable device on a user/platform:
– for various device types
– utilizing low-cost MEMS sensors
– in real-time
– for arbitrary Device Usage
– for arbitrary orientation of the device
– for an arbitrary user/platform
1/8/2016 NavINST Group - Queen's University 8
9. Introduction
Need for Motion Mode Recognition
• Each motion mode can have its own optimized
navigation algorithm/constraints
1/8/2016 NavINST Group - Queen's University 9
Detect Motion
Mode
Walking PDR with Walking Parameters
Running PDR with Running Parameters
Driving Driving Algorithms
Stationary Apply Zero-Update Velocity
Elevator
Fix 2D Position
If Map Matching Used -> Correct
2D Position
Escalator
Correct Along Track
Distance/Position
… …
Motion Mode
Algorithm / Constraints /
Optimization
10. Introduction
Prior Work
• Most of prior work had limited robustness
– portable navigation device tethered in much of the
work
– covered limited device usages
– covered limited orientations
• Most of prior work covered a small number of
motion modes
• Some depended on:
– GNSS signal availability → don’t work indoors
– Wi-Fi positioning → need special infrastructure
1/8/2016 NavINST Group - Queen's University 10
11. Introduction
Contributions
• Detected large number of motion modes
• Detected for the first time new motion modes:
• Recognition is robust:
– independent to device usage
– independent to device orientation
• Implemented in real-time on consumer devices
• Only mandatory inputs are signals from self-
contained low-cost sensors
1/8/2016 NavINST Group - Queen's University 11
12. Methodology
• Pattern recognition is
used to detect motion
modes.
• The figure shows the
steps of the pattern
recognition process.
• The next slides
explain what is
performed in each
step in the pattern
recognition process.
1/8/2016 NavINST Group - Queen's University 12
14. Methodology
• Pre-Processing
– Raw sensor readings have little meaning
– Need to process them to come up with more
meaningful variables
• Levelled Vertical Acceleration: 𝒂 𝑢𝑝 =
𝒂− 𝒂 ∙ 𝒂
𝒂∙ 𝒂
𝒂
• Magnitude of Levelled Horizontal Plane Acceleration: 𝒂ℎ =
𝒂 − 𝒂 𝑢𝑝
• Compensated Norm of Angular Rotation Components:
𝜔 = 𝜔 𝑥 − 𝑏 𝑥
2 + 𝜔 𝑦 − 𝑏 𝑦
2
+ 𝜔𝑧 − 𝑏 𝑧
2
• Vertical Velocity: differentiation of smoothed altitude
1/8/2016 NavINST Group - Queen's University 14
15. • Feature Extraction
– Statistical Features
• mean, median, mode, variance, standard deviation, 75th percentile,
inter-quartile range, average absolute difference, and binned
distribution, skewness, kurtosis
– Energy, Power, and Magnitude Features
• energy, sub-band energies, sub-band energy ratios, and signal
magnitude area
– Time-Domain Features
• zero-crossing rate and number of maximum peaks
– Frequency-Domain Features
• absolute values of short-time Fourier transform, FOS spectral
analysis, power spectral centroid, frequency domain entropy, average
of continuous wavelet transform
– Other
• cross-correlation between leveled vertical and horizontal acceleration
components
• ratio between vertical velocity and number of peaks in levelled
vertical acceleration
Methodology
1/8/2016 NavINST Group - Queen's University 15
16. • Classification
– Classification method used is Decision Tree
– Optimized using pruning
Methodology
1/8/2016 NavINST Group - Queen's University 16
Land-Based
Vessel
Bicycle
Land-Based
Vessel
F5>Thr5
F2>Thr2 F3>Thr3
F1>Thr1
F6>Thr6
Walking
Running
Bicycle
Walking F4>Thr4Bicycle Running
17. Methodology
• Classification (cont.)
– Separate classifiers for various groups of motion
modes
– Required classifier invoked based on certain
conditions
1/8/2016 NavINST Group - Queen's University 17
18. Methodology
• Post-Classification Refining
– In the full solution, further enhancements are used
after machine learning, such as:
• Majority Selection
• Context Exclusion
• Map Information
• GNSS Velocity
1/8/2016 NavINST Group - Queen's University 18
20. Experimental Results
• Data Collection (cont.)
1/8/2016 NavINST Group - Queen's University 20
Data Collected
>2300 trajectories, >225 hours
divided into:
Training Data Evaluation Data
Features extracted from it are used to train
classifier to generate classifier model
Features extracted from it are fed into
generated classifier model to evaluate its
performance
21. Experimental Results
• Data Collection (cont.)
– Various users with various genders, heights,
weights, motion dynamics, speeds, and gaits
– Various device usages, and orientations
– Different Elevators/Escalators/Moving Walkways in
different places and cities
– Land-based vessels included:
• Different types of: Car, Truck, Bus, Train, Light-rail Train
• Sitting, Standing, and placing device On Platform
• Moving in busy areas, quiet neighborhoods, and highways
in different cities
1/8/2016 NavINST Group - Queen's University 21
23. Experimental Results
• Results shown are of Classification only
– i.e., without Post-Classification Refining
• Performance Measure is Confusion Matrices of
evaluating classifiers on Evaluation Data (not
Training Data)
– Another evaluation was made on trajectories after
inserting GNSS outage
1/8/2016 NavINST Group - Queen's University 23
24. Experimental Results
1/8/2016 NavINST Group - Queen's University 24
Actual Motion
Mode
Predicted Motion Mode
90.5% 9.5%
2.1% 97.9%
Average Recall Rate: 94.2%
Actual Motion
Mode
Predicted Motion Mode
90.6% 9.4%
1.3% 98.7%
Average Recall Rate: 94.65%
GNSS-Outaged Trajectories
26. Experimental Results
1/8/2016 NavINST Group - Queen's University 26
Actual Motion
Mode
Predicted Motion Mode
97.2% 2.8%
15.8% 84.2%
Average Recall Rate: 90.7%
Actual Motion
Mode
Predicted Motion Mode
90.2% 9.8%
26.9% 73.1%
Average Recall Rate: 81.65%
27. Experimental Results
1/8/2016 NavINST Group - Queen's University 27
Actual Motion
Mode
Predicted Motion Mode
96.2% 3.8%
5.9% 94.1%
Average Recall Rate: 95.15%
Actual Motion
Mode
Predicted Motion Mode
90.2% 9.8%
21.7% 78.3%
Average Recall Rate: 84.25%
28. Experimental Results
1/8/2016 NavINST Group - Queen's University 28
Actual Motion
Mode
Predicted Motion Mode
87.07% 12.93%
30.94% 69.06%
Average Recall Rate: 78.06%
Actual Motion
Mode
Predicted Motion Mode
90.22% 9.78%
30.28% 69.72%
Average Recall Rate: 79.97%
GNSS-Outaged Trajectories
29. Summary
• Conclusion
– Robust detection of wide range of motion modes is
possible with self-contained low-cost sensors
• GNSS unavailability had minor – if not positive - effect
– Decision trees best choice for real-time:
• high accuracy and low computation requirements
– Solution commercialized and implemented in
consumer devices in the market
• Future Work
– Improve detection of:
– Add new motion modes
• E.g., Crawling, Airplane, Marine-Based Vessel
1/8/2016 NavINST Group - Queen's University 29
30. Publications
1/8/2016 NavINST Group - Queen's University 30
Patents:
[1] Inventors: M. Elhoushi, J. Georgy, and A. Noureldin, Assignee: Invensense Inc., “Method and System for
Estimating Multiple Modes of Motion”, U.S. Serial No. 14/528,868, filing date: 30 October 2014.
Accepted Journal Publications:
[2] M. Elhoushi, J. Georgy, A. Noureldin, and M. Korenberg, “Motion Mode Recognition for Indoor
Pedestrian Navigation using Portable Devices,” in IEEE Transactions on Instrumentation and Measurement
Published Conference Publications:
[3] M. Elhoushi, J. Georgy, A. Wahdan, M. Korenberg, and A. Noureldin, “Using Portable Device Sensors to
Recognize Height Changing Modes of Motion,” in 2014 IEEE International Instrumentation and
Measurement Technology Conference (I2MTC), 2014, pp. 477 – 481.
[4] M. Elhoushi, J. Georgy, M. Korenberg, and A. Noureldin, “Robust Motion Mode Recognition for Portable
Navigation Independent on Device Usage,” in 2014 IEEE/ION Position, Location and Navigation
Symposium - PLANS, 2014, pp. 158–163.
[5] M. Elhoushi, J. Georgy, M. Korenberg, and A. Noureldin, “Broad Motion Mode Recognition for Portable
Navigation,” in Proceedings of the 27th International Technical Meeting of The Satellite Division of the
Institute of Navigation (ION GNSS+ 2014), 2014, pp. 1768–1773.
Submitted Journal Publications:
[6] M. Elhoushi, J. Georgy, A. Noureldin, and M. Korenberg, “Online Motion Mode Recognition for Portable
Navigation using Low-Cost Sensors,” in NAVIGATION
[7] M. Elhoushi, J. Georgy, A. Noureldin, and M. Korenberg, “A Survey on Approaches of Motion Mode
Recognition Using Sensors,” in IEEE Transactions on Intelligent Transportation Systems