A talk from the Develop Track at AWE USA 2018 - the World's #1 XR Conference & Expo in Santa Clara, California May 30- June 1, 2018. Klony Lieberman (Sixdof Space): Ultra Fast 6DOF Tracking We present a prototype inside-out optical 6DOF tracking system using an optical image compression technology that greatly enhances the response time of the optical tracking while simultaneously reducing the overall computational requirements by an order of magnitude. http://AugmentedWorldExpo.com

1. Slide 1
Ultra-Fast 6DOF Tracking
Dr. Klony Lieberman
VP R&D
Ultra-High-Speed 6DOF SLAM using Optical Compression

2. Slide 2
The story with consumer XR* tracking
Inside-out tracking is the best approach but…
…Visual SLAM is a very, very difficult problem
• Uncontrolled environment
• Complex scenes
• Random lighting environments
• Moving imaging platform -> motion blur
• Huge data pipeline with massive amounts of data
• Single 1 Megapixel camera at 50 FPS generates >50MByte of data per second
• Computationally intensive requiring dedicated CPU/DSP/GPU
* Choose your preference… AR / VR / MR etc.

3. Slide 3
1st Trick: Reduce the amount of data!
1. Concentrate on bright and high contrast objects
2. Perform image compression in the optical domain
3. Maintain high directional visual entropy during compression
4. Use a linear optical sensor to convert the image data to electrical signals at
high speed
Optically compressing the entire room image into a handful of 1D data sets
reduces the 2D image processing nightmare into something more like an audio
processing challenge

4. Slide 4
2nd Trick: Run fast!
1. Capture bright and high contrast objects with shutters <1ms
2. With motion blur under control, SLAM can run during movement
3. Small data sets allow SLAM iterations as rapidly as every 1ms
4. At 1KHz frame rates user movements are typically well below 1o angular and
<1cm translational
The faster you can iterate SLAM, the less guesswork is required at each iteration
and therefore the faster you can iterate SLAM!

5. Slide 5
Data reduction technique
Given a 2D sparse image matrix:
• Sum rows to get a 1D vertical elevation representation
• Sum columns to get a 1D horizontal azimuth representation
Capture
resolution
2D total
pixels
2 x 1D total
pixels
Data
reduction
30 x 15 450 45 10 x
640 x 480 307,200 1,120 274 x
1024 x 512 524,288 1,536 341 x
2048 x 1024 2,097,152 3,072 682 x
4096 x 2048 8,388,608 6,144 1,365 x
Pixel count scales linearly with resolution
Compressing the image electronically reduces the amount of data sent to the host, but still requires
reading out and electronically processing an entire 2D image sensor

6. Slide 6
Optical data reduction with Dome Lens
• Light from any azimuth angle β arriving at a
given elevation (α) will be integrated at the
same pixel
• Light from different elevation angles α will
be mapped to a unique pixel on the sensor
• The result is an instantaneous 1-dimensional
representation of the 2-dimensional image
scene

7. Slide 7
Dome Lens prototype module
• Simple to manufacture and assemble
• Great proof-of-feasibility
• But … poor trade off between resolution optical
efficiency
 Requires very narrow slit
 Low light collection efficiency

8. Slide 8
Aspheric Toroid Compression Lens
Diamond turned optical prototype
Same basic principle as the dome lens with dramatically improved optical performance
yet little additional manufacturing complexity

9. Slide 9
Toroid Lens Module Performance
• Large field of view (>120 degrees in both axes)
• High optical efficiency – large NA
• Very high image resolution
• Single element optic design
• 1K pixel low-cost linear CMOS sensor
• 0.5 mSec shutter or less
• Up to 1KHz readout rate
• At 8-bit quantization, still only 1MByte/sec of data per sensor
Cost-effective and convenient for mass-market consumer device

10. Slide 10
Development board - 3 sensors
• Horizontal sensors give azimuth angles to features
• Two sensors used to deliver distance via triangulation
• Single vertical sensor provides vertical elevation angles
Power and data connection to host system require only a single USB2.0 connection

11. Slide 11
1K real-time absolute angle measurement
• 1000 frames of 1KB per second = 1MB data transfer per second
• Basic processing done in real time on the on-board 200MHz CPU
• Raw or processed data easily transferred over regular USB-2 to host
Tracking rotation rates of > 500 degrees per second

12. Slide 12
SLAM analysis with Kalman Filter
• Small number of individual objects is well suited to EKF analysis
• Each feature is reduced to a signature with unique defining characteristics
• With multiple lights in view, multiple tentative solutions exist for X,Y co-ordinates
• Addressed by disambiguation especially by behaviour with movement
• High speed readout (1 mSec image update rate) means:
• Very small change in pose between frames
• Can synchronize perfectly with MEMS data rates – no ‘fusion’ required
• MEMS sensor integration for continuous tracking with only a single light source in view

13. Slide 13
Mounted on commercial 3DOF headsets
Samsung Gear VR Sony PSVR
Well-positioned for bringing 6DOF to 3DOF devices

14. Slide 14
6DOF tracking results – comparison with HTC-VIVE Lighthouse

15. Slide 15
6DOF tracking results – comparison with HTC-VIVE Lighthouse

16. Slide 16
6DOF tracking results – comparison with HTC-VIVE Lighthouse

17. Slide 17
System Architecture - designed for low processing requirements
• Front end - Feature extraction
• Back end - SLAM processing
• Both processes require a low memory footprint (<<1MB)
• No DSP
• No GPU
• Can be implemented using fixed-point arithmetic (no FPU)
• Expect to run at speeds above 250 frames/second on a single 32-bit processor core
clocked at 1.5GHz
Designed to easily run on a single core of a Snapdragon 835

18. Slide 18
Conclusions
• Optical Compression Visual SLAM
• Uses existing room lighting as reference beacons
• Low-cost, low-power and installation-free
• Large field of view with very low data bandwidth
• Tracking rates up to 1KHz
• End-to-end latency (motion to absolute pose data) as
low as 2ms