4. Navigation/Localisation is a fundamental technology
Cornerstone of the physical internet
person
chair
chair
robot
drone
bike
person
table
Vision
5. Navigation/Localisation is a fundamental technology
Cornerstone of the physical internet
Active RF 802.11mc
person
chair
chair
robot
drone
bike
person
table
Vision
6. Navigation/Localisation is a fundamental technology
Cornerstone of the physical internet
Active RF 802.11mc
person
chair
chair
robot
drone
bike
person
table
Vision
Passive RF-Pose3D
7. Navigation/Localisation is a fundamental technology
Cornerstone of the physical internet
Active RF 802.11mc
person
chair
chair
robot
drone
bike
person
table
Vision
Passive RF-Pose3D
…
9. Name of game: Robustification, robustification, robustification
10. The future of computing is spatial
Name of game: Robustification, robustification, robustification
11. The future of computing is spatial
multi-modal
Name of game: Robustification, robustification, robustification
12. The future of computing is spatial
multi-modal, multi-layered
Name of game: Robustification, robustification, robustification
13. The future of computing is spatial
multi-modal, multi-layered, and semantic
Name of game: Robustification, robustification, robustification
14. The future of computing is spatial
multi-modal, multi-layered, and semantic
Lest …
Name of game: Robustification, robustification, robustification
15. The future of computing is spatial
multi-modal, multi-layered, and semantic
Lest …
Name of game: Robustification, robustification, robustification
16. What do we need to do?
Inertial
Radar Lidar
Vision
Acoustic Infrared
…
… …
…
17. What do we need to do?
Inertial
Radar Lidar
Vision
Acoustic Infrared
…
… …
…
18. What do we need to do?
Inertial
Radar Lidar
Vision
Acoustic Infrared
…
… …
…
Ground 0
• independent of environmental dynamics
• i.e. very useful
19. What do we need to do?
Inertial
Radar Lidar
Vision
Acoustic Infrared
…
… …
…
Ground 0
• independent of environmental dynamics
• i.e. very useful
T
r
i
c
k
s
20. What do we need to do?
Inertial
Radar Lidar
Vision
Acoustic Infrared
…
… …
…
Ground 0
• independent of environmental dynamics
• i.e. very useful
T
r
i
c
k
s
DL
21. What do we need to do?
Inertial
Radar Lidar
Vision
Acoustic Infrared
…
… …
…
Ground 0
• independent of environmental dynamics
• i.e. very useful
Comparison between DL (IONet),
pedestrian dead reckoning (PDR), and
traditional strapdown inertial navigation
systems (SINS)
Why Deep Learning?
T
r
i
c
k
s
22. What do we need to do?
Inertial
Radar Lidar
Vision
Acoustic Infrared
…
… …
…
Ground 0
• independent of environmental dynamics
• i.e. very useful
T
r
i
c
k
s
Comparison between DL (IONet),
pedestrian dead reckoning (PDR), and
traditional strapdown inertial navigation
systems (SINS)
Chen et al. "Ionet: Learning to cure the curse of drift in inertial odometry."
arXiv preprint arXiv:1802.02209 (2018).
Why Deep Learning?
T
r
i
c
k
s
23. What do we need to do?
Inertial
Radar Lidar
Vision
Acoustic Infrared
…
… …
…
Ground 0
• independent of environmental dynamics
• i.e. very useful
T
r
i
c
k
s
Comparison between DL (IONet),
pedestrian dead reckoning (PDR), and
traditional strapdown inertial navigation
systems (SINS)
Chen et al. "Ionet: Learning to cure the curse of drift in inertial odometry."
arXiv preprint arXiv:1802.02209 (2018).
Why Deep Learning?
T
r
i
c
k
s
Major improvements over classical methods
25. DL, is that it?
System-level innovations needed for the real-world
26. DL, is that it?
System-level innovations needed for the real-world
• ultra-low power tags
• compute-, energy-, and form factor-constrained
27. DL, is that it?
System-level innovations needed for the real-world
• ultra-low power tags
• compute-, energy-, and form factor-constrained
• deal with DL model fragility
• signal characteristic variabilities in the field
• i.e. hard to generalise
28. DL, is that it?
System-level innovations needed for the real-world
• ultra-low power tags
• compute-, energy-, and form factor-constrained
• deal with DL model fragility
• signal characteristic variabilities in the field
• i.e. hard to generalise
• configurability and scalability
• use cases/market segments
29. IMULet
LSTM stack
states
estimates delayline
Edge Cloud
raw 9-DOF IMU signals
acceI
gyro
magneto
Displacement
Estimate
On-device
compressed
representation
FC
FC
Adaptation
z K
layer 1
CNN
layer 0
Latent space tracking
s
30. IMULet
LSTM stack
states
estimates delayline
Edge Cloud
raw 9-DOF IMU signals
acceI
gyro
magneto
Displacement
Estimate
On-device
compressed
representation
FC
FC
Adaptation
z K
layer 1
CNN
layer 0
Latent space tracking
s
31. IMULet
LSTM stack
states
estimates delayline
Edge Cloud
raw 9-DOF IMU signals
acceI
gyro
magneto
Displacement
Estimate
On-device
compressed
representation
FC
FC
Adaptation
z K
layer 1
CNN
layer 0
Latent space tracking
s
32. IMULet
LSTM stack
states
estimates delayline
Edge Cloud
raw 9-DOF IMU signals
acceI
gyro
magneto
Displacement
Estimate
On-device
compressed
representation
FC
FC
Adaptation
z K
layer 1
CNN
layer 0
Latent space tracking
s
33. IMULet
LSTM stack
states
estimates delayline
Edge Cloud
raw 9-DOF IMU signals
acceI
gyro
magneto
Displacement
Estimate
On-device
compressed
representation
FC
FC
Adaptation
z K
layer 1
CNN
layer 0
Latent space tracking
s
(1) ultra-low power
34. IMULet
LSTM stack
states
estimates delayline
Edge Cloud
raw 9-DOF IMU signals
acceI
gyro
magneto
Displacement
Estimate
On-device
compressed
representation
FC
FC
Adaptation
z K
layer 1
CNN
layer 0
Latent space tracking
s
(1) ultra-low power
35. IMULet
LSTM stack
states
estimates delayline
Edge Cloud
raw 9-DOF IMU signals
acceI
gyro
magneto
Displacement
Estimate
On-device
compressed
representation
FC
FC
Adaptation
z K
layer 1
CNN
layer 0
Latent space tracking
s
(2) configurable
36. IMULet
LSTM stack
states
estimates delayline
Edge Cloud
raw 9-DOF IMU signals
acceI
gyro
magneto
Displacement
Estimate
On-device
compressed
representation
FC
FC
Adaptation
z K
layer 1
CNN
layer 0
Latent space tracking
s
(2) configurable
39. IMULet
LSTM stack
states
estimates delayline
Edge Cloud
raw 9-DOF IMU signals
acceI
gyro
magneto
Displacement
Estimate
On-device
compressed
representation
FC
FC
Adaptation
z K
layer 1
CNN
layer 0
Latent space tracking
s
Promising
early performance
40. IMULet
LSTM stack
states
estimates delayline
Edge Cloud
raw 9-DOF IMU signals
acceI
gyro
magneto
Displacement
Estimate
On-device
compressed
representation
FC
FC
Adaptation
z K
layer 1
CNN
layer 0
Latent space tracking
s
Promising
early performance
41. IMULet
LSTM stack
states
estimates delayline
Edge Cloud
raw 9-DOF IMU signals
acceI
gyro
magneto
Displacement
Estimate
On-device
compressed
representation
FC
FC
Adaptation
z K
layer 1
CNN
layer 0
Latent space tracking
s
Promising
early performance
Check out paper for details
42. In Summary
IMULet is a cloudlet for inertial tracking
• DL + edge + hooks for efficiency and generalisability
43. In Summary
IMULet is a cloudlet for inertial tracking
• DL + edge + hooks for efficiency and generalisability
Key component towards infrastructure-less localisation
• as mandated by physical internet/spatial computing
44. In Summary
IMULet is a cloudlet for inertial tracking
• DL + edge + hooks for efficiency and generalisability
Key component towards infrastructure-less localisation
• as mandated by physical internet/spatial computing
Scaling evaluation in-progress