This is the final presentation about our research in interactive surfaces that use for collaborative works. This is the presentation for the ATMM (Advanced Topics in MultiMedia) course which is instructed by Dr. Benoit Huet. This course has given me a wide scale of many interesting topics that multimedia can involve in.

1. +
INTERACTIVE
SURFACES
Instructor: LE Khanh-Duy
LE Anh-Dung
Dr. Benoit Huet

2. +
AGENDA
1. INTRODUCTION
2. APPROACH
3. DEMONSTRATION
4. CONCLUSION

3. +
AGENDA
1. INTRODUCTION
2. APPROACH
3. DEMONSTRATION
4. CONCLUSION

4. +
USER INTERFACE AT A GLANCE

5. +
MOTIVATION
 We want
 Draw/touch/interact freely
 Artistically
 Collaboration
 Enhancement in education

6. +
LIKE THIS ?
[1] [3]
[2]

7. +
AGENDA
1. INTRODUCTION
2. APPROACH
3. DEMONSTRATION
4. CONCLUSION

8. +
Surface Setup Methods
 Infrared-illumination-based method
 Depth-sensing-camera-based method
 Using digital pen

9. + Infrared-illumination-based method[8]
 Two most popular approaches
Frustrated total internal reflection (FTIR) Rear diffuse illumination (Rear DI)
Captured image

10. +
Infrared-illumination-based method
Rear DI setup in reality

11. +
Infrared-illumination-based method
 Preprocess the captured image by following steps
 Background subtraction
 Binarization
 Spot size filtering
 Dilation
 Detect and track the blobs to identify user’s interactions

12. + Infrared-illumination-based method
Captured image Background subtraction Binarization
Tracking Dilation Spot size filtering

13. +
Depth-sensing-camera-based
method[4]
 Depth-sensing camera: combination
of infrared and regular camera
 Example: Microsoft Kinect
 Extract 3D skeletons of users and 3D
position of display surface

14. +
Depth-sensing-camera-based method
 Map 3D positions of user’s hands in space to 2D position of
them on the surface
Symbolic representation of the projection surface as seen by the
Kinect (left); Desirable target representation of a hand hovering over the
projection surface (right)

15. +
Using Digital Pen [5]
 Interactive surface covered by a specially printed paper with
pattern of tiny dots
 Digital pen is a pen with embedded IR camera to track the
movement
Printed paper with pattern of tiny dots
Digital pen with IR camera

16. +
Authentication problem in
collaborative works
 Problem: who is interacting with a region or an object on the
surface?

17. +
Authentication problem in
collaborative works
 Some solutions
 Attaching RFID tags to users
 Tracking digital pen by ID

18. + Attaching RFID tags to users [4]
 Applied in depth-sensing camera-
based system
 Each user in the system has a tag
 Assigning a tag to a skeleton
 Tracking tagged skeletons to monitor
interactions of users.
 Users need to register with system by
interacting with area of RFID reader

19. +
Tracking digital pen by ID
 Applied in system using digital pen and pattern-printed
surface
 Each digital pen is identified by an ID.
 Movement of each digital pen will be recorded

20. +
AGENDA
1. INTRODUCTION
2. APPROACH
3. DEMONSTRATION
4. CONCLUSION

21. +
MEDUSA [7]

22. +
SHARED DESIGN SPACE [5]

23. +
AGENDA
1. INTRODUCTION
2. APPROACH
3. DEMONSTRATION
4. CONCLUSION

24. +
CONCLUSION
 Detect not only the touch, but also the hover action, the
appearance of the user
 Track each person by using RFID
 Still some limitations:
 Design
 Responsiveness
 Cost

25. +
ADDITIONAL DEMOS

26. +
References
[1] MIT Sketching: http://www.youtube.com/watch?v=NZNTgglPbUA
[2] SandCanvas: http://sandcanvas.com/
[3] MemTable: http://fluid.media.mit.edu/projects/memtable-integrated-system-capture-and-recall-shared-
histories-group-workspaces
[4] F. Klompmaker, H. Fischer and H. Jung, "Authenticated Tangible Interaction using RFID and Depth-Sensing
Camera Supporting Collaboration on Interactive Tabletops," 2012.
[5] M. Haller, P. Brandl, D. Leithinger, J. Leitner, T. Seifried and M. Billinghust, "Shared Design Space: Sketching
ideas using digital pens and a large augmented tabletop setup".
[6] E. Larson, G. Cohn, S. Gupta, X. Ren, B. Harrison, D. Fox and S. N.Patel, "HeatWave: Thermal Imaging for
Surface User Interaction".
[7] M. Annett, T. Grossman, D. Wigdor and G. Fitzmaurice, "Medusa: A Proximity-Aware Multi-touch Tabletop”.
[8] L.-w. Chan, Y.-f. Chuang, Y.-f. Chia, Y.-p. Hung and H. Jane, "A New Method for Multi-Finger Detection Using a
Regular Diffuser".

27. THANK YOU FOR YOUR
LISTENING