The original slides of my ISMAR 2014 presentation were we presented our paper: Comprehensive Workspace Calibration for Visuo-Haptic Augmented Reality, Authors: Ulrich Eck, Frieder Pankratz, Christian Sandor, Gudrun Klinker, Hamid Laga

1. Comprehensive Workspace Calibration
for Visuo-Haptic Augmented Reality
Ulrich Eck1, Frieder Pankratz2, Christian Sandor3, Gudrun Klinker2, Hamid Laga1
1 PBRC, University of South Australia, Australia
2 FAR, Technische Universität München, Germany
3 IMD Lab, Nara Institute of Science and Technology, Japan
!
10/09/2014, IEEE International Symposium on Mixed and Augmented Reality, Munich
University of
South Australia
1

2. Visuo-Haptic Augmented Reality (VHAR)
University of
South Australia
2
[Sandor C., TEDx Adelaide, 2010]

3. Novel algorithm for orientation calibration: >40% improvement
Position calibration: ~20% improvement
Time-delay compensation between sensors: faster and better calibration procedure
University of
South Australia
Contributions
3
Previous work: position calibration only Improved: with orientation calibration

4. Early examples of visuo-haptic co-location
University of
South Australia 4
Related Work
[Vallino, J. et al., ICMCS 1998] [Sandor, C. et al., IEICE 2007]

5. PHANToM Haptic Device: Joint Angles
University of
South Australia
5

6. PHANToM Haptic Device: Gimbal Angles
University of
South Australia
6

7. PHANToM Haptic Device: Sensor Errors
University of
South Australia
7

8. Compensation of sensor errors
University of
South Australia 8
Related Work
[Ikits, M. et al., EG 2003] [Harders, M. et al., TVCG 2009]

9. Missing: Orientation Calibration
University of
South Australia
9
Position calibration only With orientation calibration
Orientation Error in Degrees

10. University of
South Australia
Haptic Device Calibration
External tracker provides reference pose
Forward kinematics model calculates
stylus pose from sensor measurements
Sensor errors are modelled as linear or
quadratic function
Sensor errors are compensated by
minimizing the distance to the reference
Complete coverage of sensor ranges
required
Accurate pose correspondences
required for good calibration results
10
External Tracker (ET)
ETorigin
HIPpose
HIPtarget
Haptic Device (HD)
Ctarget
Camera (C)
HDorigin

11. Challenge: Unsynchronized Sensor Input
Unknown delay between haptic device and external tracker
Received measurements do not match during movement
Related work
tedious point and hold process for sampling corresponding
measurements
Our solution
determine time-delay between sensors first and
compensate it in software to allow users to move freely
during the calibration process
University of
South Australia
11

12. University of
South Australia
Calibration Procedure
Tooltip
(Tuceryan et al., 1995)
Absolute Orientation
(Horn, 1987)
Time-Delay Estimation
(Huber et al., 2012)
Joint-Angles
(Harders et al., 2009)
Reference Orientation
Gimbal-Angle Errors
12
External Tracker (ET)
ETorigin
HIPpose
HIPtarget
Haptic Device (HD)
Ctarget
Camera (C)
HDorigin

13. University of
South Australia
Calibration Procedure
Tooltip
(Tuceryan et al., 1995)
Absolute Orientation
(Horn, 1987)
Time-Delay Estimation
(Huber et al., 2012)
Joint-Angles
(Harders et al., 2009)
Reference Orientation
Gimbal-Angle Errors
13
External Tracker (ET)
ETorigin
HIPpose
HIPtarget
Haptic Device (HD)
Ctarget
Camera (C)
HDorigin

14. University of
South Australia
Time-Delay Estimation
14
External Tracker (ET)
ETorigin
HIP
Haptic Device (HD)
HDorigin

15. University of
South Australia 15
Time-delay estimation
[Huber, M et al., ISMAR 2009]

16. Time-aware dataflow network
execution model
[Puskta, D et al., ISMAR 2007]
University of
South Australia 16

17. University of
South Australia
Calibration Procedure
Tooltip
(Tuceryan et al., 1995)
Absolute Orientation
(Horn, 1987)
Time-Delay Estimation
(Huber et al., 2012)
Joint-Angles
(Harders et al., 2009)
Reference Orientation
Gimbal-Angle Errors
17
External Tracker (ET)
ETorigin
HIPpose
HIPtarget
Haptic Device (HD)
Ctarget
Camera (C)
HDorigin

18. Orientation Reference Estimation
University of
South Australia
18
External Tracker (ET)
ETorigin
HIP
Haptic Device (HD)
HDorigin

19. University of
South Australia 19
Reference Orientation
REFzaxis
x
y
z
z
x y
HIPpose
HIPtarget
x
y
z
x
y
z
HIPor igin
REFzaxis

20. Reference Orientation
Tracked markers travel on circular paths around the z-axis
Transform marker positions with inverse forward kinematic
pose using only the first 5 angles
Fit circles to trajectories to find centers and fit a line through
the centres and the HIP position
University of
South Australia
20

21. University of
South Australia
Calibration Procedure
Tooltip
(Tuceryan et al., 1995)
Absolute Orientation
(Horn, 1987)
Time-Delay Estimation
(Huber et al., 2012)
Joint-Angles
(Harders et al., 2009)
Reference Orientation
Gimbal-Angle Errors
21
External Tracker (ET)
ETorigin
HIPpose
HIPtarget
Haptic Device (HD)
Ctarget
Camera (C)
HDorigin

22. Gimbal-Angle Error Compensation
University of
South Australia
22
External Tracker (ET)
ETorigin
HIP
Haptic Device (HD)
HDorigin

23. University of
South Australia
Gimbal-Angle Calibration
Use reference z-axis to determine the errors of the first two
gimbal sensors
Model errors as linear or quadratic function
Minimize error using Levenberg-Marquardt optimization
Exploit mechanical limits of the last gimbal sensor by finding
the gaps in the circular paths to compensate the offset
23

24. University of
South Australia
Evaluation
Numerical Evaluation of calibration results:
Two VHAR workspace setups:
Low-fidelity (LF): PHANToM Omni, OptiTrack
High-fidelity (HF): PHANToM Premium, ART-Tracking
Visual Evaluation:
Unwrapped spherical heatmap of recuded errors
Visual overlays on haptic stylus in calibrated system
24

25. No Calib Harders Our Work No Calib Harders Our Work
No Calib Harders Our Work No Calib Harders Our Work
University of
South Australia
Numerical Evaluation
25
Reduced position errors by ~ 20%
Reduced orientation errors by > 40%

26. Improved Orientation Accuracy
University of
South Australia
26
Reduction of Orientation Errors in Degrees

27. ISMAR 2014
Live Demo
You can:
- perform a calibration
- paint on a fish
We’re in Room 00.07.014
27

28. Summary
Contributions:
Novel algorithm for orientation calibration (>40% improvement)
Improved position calibration (~20% improvement)
Compensated time-delay for faster and better calibration
University of
South Australia
28
REFzaxis
x
y
z
z
x y
HIPpose
HIPtarget
x
y
z
x
y
z
HIPorigin
REFzaxis
Thank You!! Questions ?

29. University of
South Australia
References
Harders, M. & Szekely, G., 2009. Calibration, Registration, and Synchronization for High Precision
Augmented Reality Haptics. IEEE Transactions on Visualization and Computer Graphics, 15(1), pp.138–
149.
Fu, M.J. et al., 2011. Effect of Visuo-Haptic Co-location on 3D Fitts' Task Performance. In Proceedings of
the International Conference on Intelligent Robots and Systems (IROS), 2011 IEEE/RSJ. San Francisco,
CA, USA: IEEE, pp. 3460–3467.
Rhienmora, P. et al., 2010. Augmented Reality Haptics System for Dental Surgical Skills Training. In
Proceedings of the 17th ACM Symposium on Virtual Reality Software and Technology. Hong Kong: ACM,
pp. 97–98. Ikits, M., Hansen, C.D. & Johnson, C.R., 2003. A Comprehensive Calibration and Registration
Procedure for the Visual Haptic Workbench. In the workshop. New York, NY, USA: ACM, pp. 247–254.
Vallino, J. & Brown, C., 1999. Haptics in Augmented Reality. In Proceedings of the IEEE International
Conference on Multimedia Computing and Systems. pp. 195–200.
Horn, B.K.P., 1987. Closed-form Solution of Absolute Orientation using Unit Quaternions. Journal of the
Optical Society of America, 4(4), pp.629–642.
Tuceryan, M., Greer, D.S. & Whitaker, R.T., 1995. Calibration Requirements and Procedures for a Monitor-
Based Augmented Reality System. Visualization and …, 1(3), pp.255–273.
Pustka, D. et al., 2006. Spatial Relationship Patterns: Elements of Reusable Tracking and Calibration
Systems. In Proceedings of the International Symposium for Mixed and Augmented Reality. Santa Barbara,
CA, USA, pp. 88–97.
Sandor, C. & Uchiyama, S., 2007. Exploring Visuo-Haptic Mixed Reality, IEICE.
Huber, M., Michael, S. & Klinker, G., 2012. Temporal Calibration in Multisensor Tracking Setups. 9.
Workshop der GI-Fachgruppe Virtuelle und Erweiterte Realität (VR-AR 2012), pp.201–212.
29