The Image-based data glove presentation

4,507 views

Published on

Slides of the image-based data glove presented during SVR 2008.

Project Page: http://vitorpamplona.com/wiki/The%20Image-Based%20Data%20Glove

Reference: Vitor F. Pamplona, Leandro A. F. Fernandes, João Prauchner, Luciana P. Nedel e Manuel M. Oliveira. The Image-Based Data Glove . Proceedings of X Symposium on Virtual Reality (SVR'2008), João Pessoa, 2008. Anais do SVR 2008, Porto Alegre: SBC, 2008, (ISBN: 857669174-4). pp. 204-211

1 Comment
3 Likes
Statistics
Notes
  • Reference: Vitor F. Pamplona, Leandro A. F. Fernandes, João Prauchner, Luciana P. Nedel e Manuel M. Oliveira. The Image-Based Data Glove . Proceedings of X Symposium on Virtual Reality (SVR'2008), João Pessoa, 2008. Anais do SVR 2008, Porto Alegre: SBC, 2008, (ISBN: 857669174-4). pp. 204-211
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
4,507
On SlideShare
0
From Embeds
0
Number of Embeds
14
Actions
Shares
0
Downloads
184
Comments
1
Likes
3
Embeds 0
No embeds

No notes for slide
  • The Image-based data glove presentation

    1. 1. The Image-Based Data Glove Vitor F. Pamplona Leandro A. F. Fernandes João L. Prauchner Luciana P. Nedel [email_address] Manuel M. Oliveira
    2. 2. Outline <ul><li>Data gloves </li></ul><ul><li>Our idea </li></ul><ul><li>The tracking system </li></ul><ul><li>The virtual hand representation and animation </li></ul><ul><li>The prototype </li></ul><ul><li>Device evaluation </li></ul><ul><li>Results and conclusion </li></ul>
    3. 3. Outline <ul><li>Data gloves </li></ul><ul><li>Our idea </li></ul><ul><li>The tracking system </li></ul><ul><li>The virtual hand representation and animation </li></ul><ul><li>The prototype </li></ul><ul><li>Device evaluation </li></ul><ul><li>Results and conclusion </li></ul>
    4. 4. Data Gloves P5 Glove $59.00 6 DOF of motion Fingers flexion 0.5° resolution (90 ° ) 60Hz refresh rate Pinch Glove $1,899.00 6 DOF of motion Pinch gestures only DG5-VHand Glove $485.00 Fingers flexion (10-bit per finger)
    5. 5. Data Gloves MidiGlove $1,200.00 Fingers flexion (10-bit per finger) 100-200Hz refresh rate X-IST Data Glove $1,200.00-$1,700.00 Pitch and roll motion (+ $125.00) Fingers flexion (10-bit per finger) 100-200Hz refresh rate 5DT Data Glove $895.00-$1495.00 Pitch and roll motion Fingers flexion (8-bit per finger) 75Hz-200Hz refresh rate
    6. 6. Data Gloves CyberGlove II 6 DOF of motion 0.5° resolution 90 records per second ShapeHand $7,500.00-$16,200.00 6 DOF of motion Fingers flexion and adduction
    7. 7. Outline <ul><li>Data gloves </li></ul><ul><li>Our idea </li></ul><ul><ul><li>The Image-based data glove (IBDG) </li></ul></ul><ul><li>The tracking system </li></ul><ul><li>The virtual hand representation and animation </li></ul><ul><li>The prototype </li></ul><ul><li>Device evaluation </li></ul><ul><li>Results and conclusion </li></ul>
    8. 8. The Idea <ul><li>A low cost full-featured device </li></ul><ul><li>Markers on each finger tip </li></ul><ul><li>A single web cam fixed on hand </li></ul><ul><li>A software to </li></ul><ul><ul><li>track the position of those markers </li></ul></ul><ul><ul><li>estimate the position of the finger joints </li></ul></ul><ul><ul><li>returns the full status of the user fingers </li></ul></ul>Webcam Visual Marker
    9. 9. Research Challenges <ul><li>Build an image-based data glove </li></ul><ul><li>Finger tips tracking with flexure and adduction </li></ul><ul><li>Suitable for existing manipulation techniques </li></ul><ul><li>Requirements </li></ul><ul><ul><li>Low Cost </li></ul></ul><ul><ul><li>Comfort </li></ul></ul><ul><ul><li>Accuracy and Precision </li></ul></ul><ul><ul><li>Real time performance </li></ul></ul>
    10. 10. Contributions <ul><li>Single camera per hand </li></ul><ul><li>“ Continuous” tracking of fingers position </li></ul><ul><li>Reproduction of fingers flexion and adduction </li></ul><ul><li>Low cost, real time and precise </li></ul>
    11. 11. Problems to be solved <ul><li>Tracking system </li></ul><ul><ul><li>The markers are small </li></ul></ul><ul><ul><li>The camera has blur and focus lost </li></ul></ul><ul><ul><li>The tracking system needs to be real time </li></ul></ul><ul><li>Virtual hand representation and animation </li></ul><ul><ul><li>The dimensions of the virtual hand must be real </li></ul></ul><ul><ul><li>The inverse kinematics should be real time </li></ul></ul><ul><li>Prototype assembling (hardware) </li></ul><ul><ul><li>The camera position and orientation can not change </li></ul></ul><ul><ul><li>The markers need to be planar </li></ul></ul><ul><ul><li>The markers position and orientation can not change </li></ul></ul>
    12. 12. Outline <ul><li>Data gloves </li></ul><ul><li>Our idea </li></ul><ul><li>The tracking system </li></ul><ul><li>The virtual hand representation and animation </li></ul><ul><li>The prototype </li></ul><ul><li>Device evaluation </li></ul><ul><li>Results and conclusion </li></ul>
    13. 13. Tested Tracking Systems <ul><li>ARToolKit </li></ul><ul><ul><li>Misclassification among patterns </li></ul></ul><ul><ul><li>Sensitive to focus changes </li></ul></ul><ul><ul><li>Unstable tracking (jitter) </li></ul></ul><ul><li>ARTag </li></ul><ul><ul><li>Needs at least 2 different markers to detects the same object </li></ul></ul><ul><li>ARToolKitPlus </li></ul><ul><ul><li>Improved pattern recognition system </li></ul></ul><ul><ul><li>Stable tracking (less jitter) </li></ul></ul><ul><ul><li>Automatic thresholding </li></ul></ul><ul><ul><li>Improved camera calibration model </li></ul></ul>
    14. 14. Tested Tracking Systems 10x10 mm, ARToolKitPlus BCH patterns with thin borders 10x10 mm, ARToolKitPlus simple patterns 7x7 mm, ARToolKit patterns
    15. 15. Tested Webcam Genius Trek 310 Genius VideoCam NB Creative WebCam PD1001 Creative WebCam Live! Ultra CIF CMOS PC Camera Logitech QuickCam for Notebooks Pro
    16. 16. Tracking System and Calibration M k M c M h
    17. 17. Outline <ul><li>Data gloves </li></ul><ul><li>Our idea </li></ul><ul><li>The tracking system </li></ul><ul><li>The virtual hand representation and animation </li></ul><ul><li>The prototype </li></ul><ul><li>Device evaluation </li></ul><ul><li>Results and conclusion </li></ul>
    18. 18. Virtual Hand Representation <ul><li>Hand model from V-ART library </li></ul><ul><li>Blender Inverse Kinematics Module </li></ul>
    19. 19. Outline <ul><li>Data gloves </li></ul><ul><li>Our idea </li></ul><ul><li>The tracking system </li></ul><ul><li>The virtual hand representation and animation </li></ul><ul><li>The prototype </li></ul><ul><li>Device evaluation </li></ul><ul><li>Results and conclusion </li></ul>
    20. 20. The Prototype <ul><li>Weight 195g </li></ul>
    21. 21. The Dices!
    22. 22. The FLEA Camera <ul><li>Point Grey Research </li></ul><ul><li>Developers Experience </li></ul><ul><li>30 × 31 mm size </li></ul><ul><li>High quality images (1024 × 768 pixels) at 30 fps </li></ul><ul><li>3.5-10.5 mm Computar Varifocal Lenses. </li></ul><ul><li>Total weight: 115 g </li></ul>
    23. 23. Using The Prototype
    24. 24. Summary <ul><li>Hardware </li></ul><ul><ul><li>Camera to track finger tips </li></ul></ul><ul><ul><li>Paper and printer to make visual markers </li></ul></ul><ul><ul><li>Glove, leather and sticks to build the camera support </li></ul></ul><ul><ul><li>Dices and thimbles </li></ul></ul><ul><li>Software </li></ul><ul><ul><li>C++ </li></ul></ul><ul><ul><li>ARToolKitPlus library </li></ul></ul><ul><ul><li>V-ART library </li></ul></ul><ul><ul><li>Blender inverse kinematics module </li></ul></ul>
    25. 25. Outline <ul><li>Data gloves </li></ul><ul><li>Our idea </li></ul><ul><li>The tracking system </li></ul><ul><li>The virtual hand representation and animation </li></ul><ul><li>The prototype </li></ul><ul><li>Device evaluation </li></ul><ul><li>Results and conclusion </li></ul>
    26. 26. Device Evaluation <ul><li>Testbed application </li></ul><ul><li>15 Subjects </li></ul><ul><ul><li>2 women and 13 men </li></ul></ul><ul><ul><li>13 right handed and 2 left handed </li></ul></ul><ul><ul><li>21 to 38 years old </li></ul></ul><ul><li>Goals : </li></ul><ul><ul><li>to verify the quality of the finger tips tracking </li></ul></ul><ul><ul><li>to verify the gesture reproduction </li></ul></ul><ul><ul><li>to collect some impressions of the users about IBDG </li></ul></ul>
    27. 27. Testbed Application <ul><li>Given a hand gesture, the user must imitate it. </li></ul>
    28. 28. Methodology <ul><li>Pre-test questions </li></ul><ul><li>Calibration section </li></ul><ul><li>Warm up section </li></ul><ul><ul><li>Unlimited time for training </li></ul></ul><ul><li>Task description </li></ul><ul><ul><li>Six pre-defined hand poses </li></ul></ul><ul><ul><li>The user imitates them </li></ul></ul><ul><ul><li>The movements are logged </li></ul></ul><ul><li>Post-test questions </li></ul>
    29. 29. Data Collection and Analysis <ul><li>Store </li></ul><ul><ul><li>User identification </li></ul></ul><ul><ul><li>Task parameters </li></ul></ul><ul><ul><li>Position and orientation of each finger along time </li></ul></ul><ul><ul><li>Elapsed time </li></ul></ul><ul><li>Analyze </li></ul><ul><ul><li>Precision </li></ul></ul><ul><ul><li>Performance </li></ul></ul><ul><ul><li>Subjective measures (post-test questions) </li></ul></ul>
    30. 30. Outline <ul><li>Data gloves </li></ul><ul><li>Our idea </li></ul><ul><li>The tracking system </li></ul><ul><li>The virtual hand representation and animation </li></ul><ul><li>The prototype </li></ul><ul><li>Device evaluation </li></ul><ul><li>Results and conclusion </li></ul>
    31. 31. Precision Analisis <ul><li>Mean observed error: 0.59 </li></ul><ul><li>Confidence interval of 99%; </li></ul>
    32. 32. Usability <ul><li>Latency about 237ms (2.4 GHz PC with 2gb of memory) </li></ul><ul><li>Velocity about 23 fps (640 x 480) </li></ul><ul><li>Weight about 195g (58% because of the FLEA camera) </li></ul><ul><li>Maximum size of hands for using our prototype 21.5 cm </li></ul><ul><li>User opinions to our prototype </li></ul><ul><ul><li>Comfort: 2.73 (5 means comfortable and 1 uncomfortable) </li></ul></ul><ul><ul><li>Precision: 2.87 (5 means precise and 1 means imprecise) </li></ul></ul><ul><li>80% of the volunteers said they can build their own IBDG </li></ul>
    33. 33. Discussion <ul><li>The tracking system (ARToolkitPlus) worked well with </li></ul><ul><ul><li>Smooth variations in lighting conditions </li></ul></ul><ul><ul><li>Shadows over the visual patterns </li></ul></ul><ul><li>For dark places one can use an infrared camera </li></ul><ul><li>FLEA Camera can be replaced by a newest web cams </li></ul><ul><ul><li>Improving comfort with a lighter one </li></ul></ul><ul><ul><li>Decreasing the prototype cost with a cheaper one </li></ul></ul><ul><ul><li>Wireless or bluetooh enabled camera </li></ul></ul><ul><li>Thumb can be tracked using a device with a greater field of view </li></ul><ul><li>The prototype needs a second version! </li></ul>
    34. 34. Questions? <ul><li>Luciana Porcher Nedel </li></ul><ul><li>[email_address] </li></ul>The Image-Based Data Glove

    ×