Technische Universität München      Haptic CommunicationsFernanda Brandi, Rahul Chaudhari, Burak Cizmeci, Julius Kammerl, ...
Technische Universität MünchenThe quest for immersive communication: telepresence                                  Network...
Technische Universität MünchenTelepresence + Haptics = Teleaktion (Telemanipulation)                         Local Control...
Technische Universität MünchenHaptic interaction in shared (virtual) environments   Subjective sense of togetherness in sh...
Technische Universität MünchenCollaborative Haptics27.03.2012              Eckehard Steinbach et al.                      ...
Technische Universität MünchenHaptics         Kinesthetic Perception                                           Tactile Per...
Technische Universität MünchenHaptic Communications                    Position/Velocity                            Intern...
Technische Universität MünchenProperties of haptic data streams§  Data format:      §  Degrees of freedom: between 1 and...
Technische Universität MünchenOutline§  Early work in haptic data compression / reduction§  Perceptual online data reduc...
Technische Universität MünchenEarly work in haptic data compression / reduction§  Lossy compression of haptic payload (di...
Technische Universität MünchenOutline§  Early work in haptic data compression / reduction§  Perceptual online data reduc...
Technische Universität München    Weber’s law  Just Noticeable Difference (JND)                ΔI                   = cons...
Technische Universität MünchenPerceptual haptic data reduction§  Exploit limits of human haptic perception§  Only transm...
Technische Universität MünchenPerceptual deadband coding                                            ΔISender:             ...
Technische Universität MünchenPredictive coding and filtering                                  Position + Velocity        ...
Technische Universität MünchenCombination with predictive codingA                                           A             ...
Technische Universität MünchenResults for 1 DoF experiment                            1000                                ...
Technische Universität MünchenAlternative prediction approach: local object model                                         ...
Technische Universität MünchenLocal surface models27.03.2012             Eckehard Steinbach et al.                        ...
Technische Universität MünchenExample27.03.2012   Eckehard Steinbach et al.                                    20
Technische Universität MünchenResults             Rate-Quality Performance                                               X...
Technische Universität MünchenOutline§  Early work in haptic data compression / reduction§  Perceptual online data reduc...
Technische Universität MünchenMulti-DoF extension§  Independent usage of 1-DoF deadband on    components of multi-DoF dat...
Technische Universität MünchenMulti-DoF isotropic deadzone                    y                                      y    ...
Technische Universität MünchenDirection Adaptive Perceptual Force Deadzone                                            Base...
Technische Universität MünchenOutline§  Early work in haptic data compression / reduction§  Perceptual online data reduc...
Technische Universität München    Haptic recording and replaySensAble Technologies                                        ...
Technische Universität MünchenHaptic recording and replay§  Challenge: Simultaneous display of force and position/    mot...
Technische Universität München27.03.2012   Eckehard Steinbach et al.                                    29
Technische Universität MünchenDeadband-based offline compression of recorded hapticsignals                                ...
Technische Universität MünchenOutline§  Early work in haptic data compression / reduction§  Perceptual online data reduc...
Technische Universität MünchenError-resilient haptic data communication                           Artifacts             Bo...
Technische Universität München     Markov Decision Process (Binary Tree)                                        Markov    ...
Technische Universität MünchenSummary§  Perceptual haptic data reduction§  Based on Weber’s law of just noticeable diffe...
Technische Universität MünchenOutlook: Selected open issues§  How to integrate temporal aspects in human haptic    percep...
Technische Universität MünchenAcknowledgments§  Current and former PhD students: P. Hinterseer, J. Kammerl, F.    Brandi,...
Upcoming SlideShare
Loading in …5
×

Haptic Communications

2,269 views

Published on

Published in: Technology, Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,269
On SlideShare
0
From Embeds
0
Number of Embeds
174
Actions
Shares
0
Downloads
0
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Haptic Communications

  1. 1. Technische Universität München Haptic CommunicationsFernanda Brandi, Rahul Chaudhari, Burak Cizmeci, Julius Kammerl, Clemens Schuwerk, Eckehard Steinbach, Xiao Xu Institute for Media Technology, TU Munich Sandra Hirche, Iason Vittorias Institute of Automatic Control Engineering, TU MunichF t Klagenfurt University, March 27, 2012
  2. 2. Technische Universität MünchenThe quest for immersive communication: telepresence Network audiovisual communication Although conversational services are bidirectional, audiovisual data communication is 2x unidirectional IEEE Signal Processing Magazine, vol. 28, no. 1, January 2011 Special Issue on Immersive Communication Guest editors: Y. Altunbasak, J. Apostolopoulos, P. Chou, and B. H. Juang27.03.2012 Eckehard Steinbach et al. 2
  3. 3. Technische Universität MünchenTelepresence + Haptics = Teleaktion (Telemanipulation) Local Control Loop Local Control Loop Sensors & Network Actuators Operator with audio-visual-haptic Teleoperator in Human-System-Interface communication remote environment Operator performance increases significantly in telemanipulation of remote objects when haptic feedback is provided Haptic communication is by definition bidirectional [Cohen Loeb 1983; Hannaford et al. 1993; Hirzinger et al. 1994; Srinisavan et al. 1997; Dennerlein et al. 2000; Basdogan et al. 2000; Cockburn et al. 2005; Tholey et al. 2005; Hokayem et al. 2006; El Saddik 2007] R. Ferrell and T. B. Sheridan, “Supervisory control of remote manipulation,” IEEE Spectrum, vol. 4, no. 10, pp. 81–88, October 1967.27.03.2012 Eckehard Steinbach et al. 3
  4. 4. Technische Universität MünchenHaptic interaction in shared (virtual) environments Subjective sense of togetherness in shared environments is significantly improved when haptic feedback is provided [C. Basdogan et al., 2000]27.03.2012 Eckehard Steinbach et al. 4
  5. 5. Technische Universität MünchenCollaborative Haptics27.03.2012 Eckehard Steinbach et al. 5
  6. 6. Technische Universität MünchenHaptics Kinesthetic Perception Tactile Perception Image Source: Katsunari Sato, Dept. of MEIP, The University of Tokyo/Japan Position & Forces sense of touch of the skin     Percepon  of   form,  posion,  surface  texture,  sffness,  fricon,  temperature,  etc.    27.03.2012 Eckehard Steinbach et al. 6
  7. 7. Technische Universität MünchenHaptic Communications Position/Velocity Internet Force Feedback 1000 Hz sampling rate27.03.2012 Eckehard Steinbach et al. 7
  8. 8. Technische Universität MünchenProperties of haptic data streams§  Data format: §  Degrees of freedom: between 1 and >20 §  Sampling frequency: up to 1000Hz §  Sampling resolution: up to 16bit§  Transmission properties: §  Very strict delay constraints (stability) §  Control loop is closed by communication system §  High packet rates (up to 1000 pkts/s) Block-based coding not feasible ! §  Bad payload/header ratio27.03.2012 Eckehard Steinbach et al. 8
  9. 9. Technische Universität MünchenOutline§  Early work in haptic data compression / reduction§  Perceptual online data reduction of haptic signals§  Extension to multi-DoF§  Perceptual offline coding of haptic signals§  Error-resilient haptic communication27.03.2012 Eckehard Steinbach et al. 9
  10. 10. Technische Universität MünchenEarly work in haptic data compression / reduction§  Lossy compression of haptic payload (different sampling, quantization and entropy coding schemes) §  Hikichi et al., ICME 2001 §  Shahabi et al., ICME 2002 §  Ortega and Liu, Prentice Hall 2002 §  Borst, WorldHaptics 2005 Packet rate reduction is not addressed !§  Packet rate reduction §  Otanez et al., American Control Conf., 2002. Human haptic perception is not exploited !27.03.2012 Eckehard Steinbach et al. 10
  11. 11. Technische Universität MünchenOutline§  Early work in haptic data compression / reduction§  Perceptual online data reduction of haptic signals§  Extension to multi-DoF§  Perceptual offline coding of haptic signals§  Error-resilient haptic communication27.03.2012 Eckehard Steinbach et al. 11
  12. 12. Technische Universität München Weber’s law Just Noticeable Difference (JND) ΔI = constant IStimulus Intensity 10g 10g Ernst Heinrich Weber (1795-1878) 1kg 1kg+10g 100g 110g Image Source: Max Planck Institute for the History of Science, Berlin http://vlp.mpiwg-berlin.mpg.de/people/data?id=per154 27.03.2012 Eckehard Steinbach et al. 12
  13. 13. Technische Universität MünchenPerceptual haptic data reduction§  Exploit limits of human haptic perception§  Only transmit packets which cause a perceivable change§  Based on Weber’s Law of Just Noticeable Differences (JND)§  Examples of JND [Jones et al. 1992, Burdea 1996] §  Arm position: 8% §  Forces at a finger: 5-14% §  Arm velocity: 8% §  Moments at a finger: 13%27.03.2012 Eckehard Steinbach et al. 13
  14. 14. Technische Universität MünchenPerceptual deadband coding ΔISender: JND (Weber) = constant I A t Signal UpdatesReceiver: A t P. Hinterseer et al., IEEE Trans. on Signal Processing, 2008.27.03.2012 Eckehard Steinbach et al. 14
  15. 15. Technische Universität MünchenPredictive coding and filtering Position + Velocity LP Prediction Prediction Filter Deadband HSI TOP Prediction LP Prediction Filter Deadband Force§  Deadband applied to difference between true and predicted signal§  Low-pass filtering of input signals §  Noise reduction §  Removal of not perceivable or not displayable frequencies P. Hinterseer et al., IEEE Trans. on Signal Processing, 2008.27.03.2012 Eckehard Steinbach et al. 15
  16. 16. Technische Universität MünchenCombination with predictive codingA A t t Predicted Signal Input Signal Only samples that differ from the predicted signal by more than the Weber JND have to be encoded27.03.2012 Eckehard Steinbach et al. 16
  17. 17. Technische Universität MünchenResults for 1 DoF experiment 1000 Velocity LP + LinPred 900 Force LP + LinPred 800 Velocity LinPred Mean perception threshold Force LinPred Packet rate [pkts/s] 700 of the test persons 600 500 400 300 200 ≈ 90% 100 0 ≈ 94% 0 5 10 15 20 25 30 35 40 Deadband [%]27.03.2012 Eckehard Steinbach et al. 17
  18. 18. Technische Universität MünchenAlternative prediction approach: local object model Environment OP Model Network Model HSI TOP DB Define a local model of the geometric structure and impedance properties of the currently touched surface. è Haptic rendering based on local surface model27.03.2012 Eckehard Steinbach et al. 18
  19. 19. Technische Universität MünchenLocal surface models27.03.2012 Eckehard Steinbach et al. 19
  20. 20. Technische Universität MünchenExample27.03.2012 Eckehard Steinbach et al. 20
  21. 21. Technische Universität MünchenResults Rate-Quality Performance X. Xu et al., IEEE HAVE 201127.03.2012 Eckehard Steinbach et al. 21
  22. 22. Technische Universität MünchenOutline§  Early work in haptic data compression / reduction§  Perceptual online data reduction of haptic signals§  Extension to multi-DoF§  Perceptual offline coding of haptic signals§  Error-resilient haptic communication27.03.2012 Eckehard Steinbach et al. 22
  23. 23. Technische Universität MünchenMulti-DoF extension§  Independent usage of 1-DoF deadband on components of multi-DoF data? §  Packet rate is determined by lowest magnitude §  Low efficiency §  Performance decreases with increasing number of degrees of freedom§  Alternative: multi-DoF deadband27.03.2012 Eckehard Steinbach et al. 23
  24. 24. Technische Universität MünchenMulti-DoF isotropic deadzone y y 2-DoF: x x y y 3-DoF: x x z J. Drösler 2000 z Discard haptic sample Encode haptic sample27.03.2012 Eckehard Steinbach et al. 24
  25. 25. Technische Universität MünchenDirection Adaptive Perceptual Force Deadzone Based on the psychophysical findings on Force Feedback Discrimination presented in H. Tan et al., “Force-direction discrimination is not influenced by reference force direction and amplitude,” Haptics-e, 2006. H. Pongrac et al., “Limitations of human 3d force discrimination,” in Proc. Human-Centered Robotics Systems 2006. Direction-adaptive force deadzone J. Kammerl et al., IEEE HAVE 201027.03.2012 Eckehard Steinbach et al. 25
  26. 26. Technische Universität MünchenOutline§  Early work in haptic data compression / reduction§  Perceptual online data reduction of haptic signals§  Extension to multi-DoF§  Perceptual offline coding of haptic signals§  Error-resilient haptic communication27.03.2012 Eckehard Steinbach et al. 26
  27. 27. Technische Universität München Haptic recording and replaySensAble Technologies Source: DHZ/SFB 453 Position + Velocity Recorder / Player Force Feedback Video Operator with Teleoperator in Human-System Inferface Compression remote environment Data Storage 27.03.2012 Eckehard Steinbach et al. 27
  28. 28. Technische Universität MünchenHaptic recording and replay§  Challenge: Simultaneous display of force and position/ motion§  Realization of playback §  Position guidance [Crossan et al. 2006, El Saddik et al. 2007] §  (Visual) substitution of competing haptic signals [Henmi et al. 1998, Williams 2004, Corno et al. 2006 ]§  Application scenarios §  posterior performance analysis / documentation §  training and teaching §  entertainment27.03.2012 Eckehard Steinbach et al. 28
  29. 29. Technische Universität München27.03.2012 Eckehard Steinbach et al. 29
  30. 30. Technische Universität MünchenDeadband-based offline compression of recorded hapticsignals 100 - no difference 75 – perceptable, but not disturbing 50 – slighly disturbing 25 – disturbing•  High transparency up to a deadband size of k=0.4•  More than 95% of samples dropped J. Kammerl and E. Steinbach, ACM Multimedia 200827.03.2012 Eckehard Steinbach et al. 30
  31. 31. Technische Universität MünchenOutline§  Early work in haptic data compression / reduction§  Perceptual online data reduction of haptic signals§  Extension to multi-DoF§  Perceptual offline coding of haptic signals§  Error-resilient haptic communication27.03.2012 Eckehard Steinbach et al. 31
  32. 32. Technische Universität MünchenError-resilient haptic data communication Artifacts Bouncing Roughness Glue Effect F. Brandi, J. Kammerl, and E. Steinbach, ACM Multimedia 201027.03.2012 Eckehard Steinbach et al. 32
  33. 33. Technische Universität München Markov Decision Process (Binary Tree) Markov Channel Decision Predictor Model Tree ACKs Perceptual DB Position / Velocity HSI TOP Force Feedback Perceptual DB ACKs Operator with Markov Predictor Teleoperator in ChannelHuman-System Interface Model Decision remote Environment Tree F. Brandi, J. Kammerl, and E. Steinbach, ACM Multimedia 2010. 27.03.2012 Eckehard Steinbach et al. 33
  34. 34. Technische Universität MünchenSummary§  Perceptual haptic data reduction§  Based on Weber’s law of just noticeable differences§  1-dof, multi-dof§  90-95% packet rate reduction§  Similar performance for recording and replay§  Error-resilient haptic data communication27.03.2012 Eckehard Steinbach et al. 34
  35. 35. Technische Universität MünchenOutlook: Selected open issues§  How to integrate temporal aspects in human haptic perception?§  Haptic communication for area-based haptic sensing and actuation (including tactile information)§  Objective measures for immersiveness?§  Perceptual coding of wave variables?§  Joint data reduction and multiplexing for audio/video/ haptics?§  …27.03.2012 Eckehard Steinbach et al. 35
  36. 36. Technische Universität MünchenAcknowledgments§  Current and former PhD students: P. Hinterseer, J. Kammerl, F. Brandi, R. Chaudhari, X. Xu, B. Cezmici, C. Schuwerk§  Collaborators §  S. Chaudhuri (IIT Bombay) §  M. Buss, S. Hirche and I. Vittorias (Institute of Control Eng. @ TUM) §  B. Färber and V. Nitsch (University of Armed Forces Munich) §  A. El Saddik and J. Cha (University of Ottawa) §  B. Hannaford and H. King (University of Washington)§  Funding §  DFG SFB 453: High-fidelity telepresence and teleaction §  DFG STE 1093/4-1 §  ERC Grant 258941 “ProHaptics” §  European-Brazilian Network for Academic Exchange EUBRANEX27.03.2012 Eckehard Steinbach et al. 36

×