This document discusses surface computing and multi-touch display devices. It describes how multi-touch devices allow direct interaction through multiple simultaneous touches rather than using a keyboard and mouse. It explains the hardware and software components of resistive, capacitive, and optical touchscreen technologies. It also outlines the process of building a basic multi-touch table using an infrared camera and laser projection plane, and developing applications using the CCV and TUIO protocols.

1. SURFACE COMPUTING
&
MULTI-TOUCH
DISPLAY DEVICES

2. Surface Computing Advertisement

3. Contents
 Introduction to Surface Computing
 Description of MT Display Devices
 Various Sub Systems
 H/W & S/W Required
 Usage areas of MT Devices
 Practical approach to multi-touch devices.

4. Introduction
 Surface computing is the term for the
use of a specialized computer GUI in
which traditional GUI elements are
replaced by intuitive, everyday objects.
Instead of a keyboard and mouse, the
user interacts directly with a touch-
sensitive screen.

5. Description
 The multi-touch device is a user interface
device that will allow a user to interact with
a computer through use of a multi-touch
display. Thus, it is a substitute to the
keyboard and mouse. In this user can
directly interact with the system with their
fingers and not only this but with multiple
touches at the same instant.

6. How Does a Touchscreen Work?
A basic touch-screen has three main
components:
1. Touch sensor;
2. Controller;
3. Software driver.
The touch-screen is an input device, so it needs
to be combined with a display and a PC or
other device to make a complete touch input
system.

7. Touchscreen Technology
Resistive
Capacitive
Optical

8. Resistive touchscreen
Structure
Resistive touch screens
consist of a glass or
acrylic panel that is
coated with electrically
conductive and resistive
layers made with indium
tin oxide (ITO) .The thin
layers are separated by
invisible spacers.

9. Projected-capacitive touchscreen
 During a touch, capacitance forms between the
finger and the sensor grid.The embedded serial
controller in the touchscreen calculates touch
location coordinates and transmits them to the
computer for processing.

10. Optical touch-screen

11. TERMINOLOGY
 Diffuser: Screen surface for projection.
 IR Lasers: Laser emitting IR Beam.
 IR Led: Led emitting IR.
 Blob - A bright luminescent object
 Blob tracking - Assigning each blob an ID (identifier).
 TUIO - A protocol used for communicating the
position, size, and relative velocity of blobs. (Tangible
User Interface Object)
 CCV: An open source software for blob tracking.
(Community Core Vision)
 MT: Multitouch

12. TECHNIQUES
 FTIR :Frustrated Total Internal
Reflection
 DI: Diffused Illumination
 DSI: Diffused Surface Illumination
 LED LP: Led Laser Plane
 LLP: Laser Light Plane

13. PRINCIPLE OF WORKING
 Mesh of IR is generated on screen.
 Frustration is created on the surface if
touched.
 Detected by the camera.
 Blobs are created and sent to tracker.
 Tracker communicates with
application.

14. FTIR

15. Diffused Illumination

16. Diffused Surface Illumination

17. Led LP

18. Building LLP Set Up
 Table with glass top.
 Camera (PS3 eye/ MS Lifecam)
 Projector
 Mirror
 Four 780 nm IR lasers
 Four Line generators 89 degree each
 Rosco grey/Tracing paper
 USB cable for powering lasers
 Under exposed – undeveloped negative roll
 Cello tape/knife/wire/insulating tape..etc.

19. IR LASER PLANE

20. BLOCK DIAGRAM

23. Input Subsystem: The Camera
 We are using a
simple web camera
easily available in
the market.
 It can be made
capable of detecting
IR by simply
removing the filter
from the lens

24. The Projector
 A rear projection setup allows the projector to
be placed inside the multimedia table.
However, with the size of the multimedia table
in mind, there is a strict requirement on the
“throw” distance of the projector.
 An extremely short throw distance projector
was found to overcome the problem.

25. Output Subsystem
 A diffusion layer is
needed to provide a
surface for the
projected image to
be displayed.
 To diffuse the
projected image
through rear
projection a sheet of
tracing paper will
be used.

26. Cabinet Sub system

27. Computer Subsystem
 A personal computer with 20 GB free hard disk
space, 1GB RAM and PIV + processor can be
efficient enough.
 The Windows Vista operating system was chosen
because the CCV package is highly supported in
Windows.

28. Programming for the MT
 CCV is an open source C++ software package
that contains code to analyze data from a
camera, detect IR blobs from within the
data, interpret that IR blob data, and generate
TUIO events for the application to interpret.
 Since most operating systems only expect one
mouse click at any single time, a new event and
protocol must be used to interpret any number of
touches, the TUIO protocol.

29. CCV: Hardware–Software Interface
 CCV acts as a driver for the multi-touch table device
so that the computer embedded inside can interpret
touches from the user or users.
 Individual applications can be written to interpret
the data supplied from CCV and provide feedback
to the display for the user or users.

30. CCV Calibration
 The CCV library comes
with an interface to
calibrate and test the
blob detection for a
multi-touch setup.
 Also, CCV comes with
software to calibrate
and test the camera that
will be used with the
multi-touch table.

31. Application Development
 Many different platforms can be used to
develop multi-touch applications utilizing
TUIO events. Such platforms include:
 Adobe Flash
 Java
 C++
 Python
 Our demo application has been developed
with ADOBE FLASH and is an open source
s/w.

32. BUILDING MINI MT
 Cardboard box
 Cellophane sheet
 Camera
 Knife/cello-tape/A-4
sheets

33. APPLICATIONS OF MT
 Personal Computing
 Graphics
 Audio Mixing
 Video Playing
 Computer Gaming
 Slide Presentations
……and many more!

34. India’s first Multi-Touch Device
“ZUGARD”

35. QUESTIONS

36. That’s All….
Thank you…
For any assistance in building your own setup
or for any other help feel free to contact:
www.facebook.com/princejerry06
www.facebook.com/technicabar
www.twitter.com/princejerry06
www.twitter.com/technicabar
rishabha.garg06@gmail.com