1.
Lessons Learned:Ten years
of MotionTracking for Mixed
Reality applications
Tom Calloway
Advanced Programs Manager
ThalesVisionix – Massachusetts, USA
Thales Visionix, Inc. #AWE2016 1

2.
Outline
I. MotionTracking for Mixed Reality
II. Lessons to Remember
III. The Future ofAugmented Reality
Thales Visionix, Inc. #AWE2016 2

3.
MotionTracking: From Nice to Necessary
Thales Visionix, Inc. #AWE2016 3
The utility of motion tracking has ranged from useful to absolutely
necessary at points along the entire mixed reality continuum

4.
MotionTracking for Mixed Reality
Thales Visionix, Inc. #AWE2016 4
“Tracking any combination of the degrees of freedom
(DOF) can be useful for a given application.
e.g. 1, AR for pilots (3DOF orient. or 6DOF pose)
e.g. 2,VR (3DOF orient. or 6DOF pose)
e.g. 3,Tracking a tool (3DOF pos. or 6DOF pose)
e.g. 4,Tagging an object (6DOF pose + 1DOF range)
e.g. 5, Deep see diving (1DOF depth)

5.
MotionTracking Approaches are Diverse
All Kinds of Hardware:
• Gyroscopes
• Accelerometers
• Ultrasonic transmitters and receivers
• Magnetic sources and sensors
• Cameras (mono, stereo, panoptic)
• Depth sensors (structured light, time of flight)
• Markers and fixed references (visible light,
reflective, LED)
All Kinds of Algorithms:
• Kalman filtering (EKF, UKF, MSCKF)
• ICP, RANSAC, particle filtering, smoothing
• Image processing and computer vision
• Centroid refinement, natural feature detection,
machine learning
• SLAM,VINS,VIO
• Enhancement filters
• Prediction, virtual synchronization, perceptual
enhancement
Thales Visionix, Inc. #AWE2016 5

6.
Lessons to Remember
1. Understand your user and your application
2. Know your environment and your technology
3. Don’t settle for sub-optimal performance
Unless you truly have no choice
Thales Visionix, Inc. #AWE2016 6

7.
Understand your User and your Application
Thales Visionix, Inc. #AWE2016 7
Firefighters “kick” their way up stairs to
test their stability and integrity.
Shoe mounted inertial tracking
algorithm needs to be robust against,
and even make use of that knowledge.

8.
Understand your User and your Application
Thales Visionix, Inc. #AWE2016 8
Air Force pilots come in all sizes…

9.
Know your Environment and yourTechnology
Thales Visionix, Inc. #AWE2016 9
Stinger Missile DomeTrainer withWireless ultrasonic-inertial motion
tracking system, Courtesy of AAI
Tracking systems fusing ultrasonic
and inertial data can work extremely
well in environments where other
systems fail.
Doesn’t work so well in the presence
of microphone-saturating explosion
sounds though…

10.
Don’t Settle for Sub-optimal Performance
Thales Visionix, Inc. #AWE2016 10
A tracking installation wasn’t proceeding
optimally due to sighting equipment slipping
in simulated Martian sand.
We switched to a fiducial auto-mapping
approach that we use for aircraft cockpit
augmented reality.
This prompted us to further improve our
algorithms. We now get ~20 micron of RMS
mapping precision for pilot augmented reality.

11.
So remember once again..
1. Understand your user and your application
2. Know your environment and your technology
3. Don’t settle for sub-optimal performance
Unless you truly have no choice
Thales Visionix, Inc. #AWE2016 11

12.
The Future of MotionTracking in AR
Enabling Augmented Reality Anywhere
1. Inertial Sensors will become more accurate and dominate the approaches
2. Computational power will continue improving (ASICs, 3D chips, local + cloud, etc.)
3. Utility of AR will expand in tandem with hardware and content
4. Major software efforts including OS re-writes (ROS -> CyOS)
5. Perfect storm of wearables, IoT, better algorithms, content, data and hardware
6. The net result will be extremely positive, fun, and profitable!
Thales Visionix, Inc. #AWE2016 12

13.
Thales Visionix, Inc. #AWE2016 13
A glimpse of Keiichi Matsuda’s Hyper Reality dystopia

14.
Thank you!
Come see the ThalesVisionix booth in the AR Section of the Expo
- See technology demonstrations (natural feature tracking and AR)
- Let us help you solve your motion tracking challenges
Thales Visionix, Inc. #AWE2016 14