2. Output Devices
The human senses need specialized interfaces
Graphics displays for visual feedback;
3-D audio hardware for localized sound;
Haptic interfaces for force and touch feedback;
Interested in smell and not yet in taste feedback.
3. Output Devices
Definition:
A graphics display is a computer interface
that presents synthetic world images to
one or several users interacting with the
virtual world.
4. Output Devices
Graphics Displays
Human stereo viewing;
Personal displays;
Large volume displays:
–Active glasses
– Workbenches;
– Microsoft Surface
– Caves;
– Walls;
5. Output Devices
Human Visual System
Vision is the dominant sensorial channel;
Depth perception in mono images is based
- on occlusion (one objects blocks another
from view;
- on shadows, textures and motion parallax
(closer images appear to move more than
distant ones)
6. Human Visual System-continued
Depth perception in stereo is based on seteropsis
when the brain registers and fuses two images;
Image parallax means that the two eyes
register different images (horizontal shift);
The amount of shift depends on the “inter-
pupillary distance” (IPD) (varies for each person in
the range of 53-73 mm);
Works in the near field (to a few meters from the
eye)
8. Output Devices (same principle
used in new 3D HDTVs)
Left eye image
Right eye image
9. Output Devices
Implications for Stereo Viewing devices
Need to present two images of the same VR
environment;
The two images can be presented at the same
time on two displays (HMD);
The two images can also be presented time-
sequenced on one display (active glasses);
The two images can also be presented spatially-
sequenced on one display (auto-stereoscopic
displays).
10. Output Devices
Personal Displays
Definition:
A graphics display that outputs a virtual
scene destined to be viewed by a single
user. Such image may be monoscopic or
stereoscopic, monocular (for a single eye)
or binocular (displayed on both eyes).
11. Output Devices
Personal Displays
Head Mounted Displays;
3-D Binoculars (hand supported);
Booms (floor supported);
Virtual windows (floor supported);
Auto-stereoscopic displays (desk supported).
18. Output Devices
HMD Characteristics (Summary)
A field of view of at least 120x50 degrees.
At least 1600x1200 resolution, but preferably HD
1080.
Bright displays with a very fast dynamic response.
No more than 250 grams (8-10 oz) in weight.
Easy user interface and cable management
(based on responses from 84 universities)
19. AMLCD display,
Resolution: 267x225
FOV: 30x23 degrees–
Equivalent to 62 in at 2 m
Weight: 100 grams
Can be worn over glasses
Olympus Eye Trek Face Mounted Display (FMD 200)
20. Olympus Eye Trek Head Mounted Display Optics
– uses free-form lens to compensate for aberrations;
- an eccentric optical system to reduce size (eliminate 45 degree mirror)
24. Organic LEDs (OLED)
Active-matrix OLED display, each pixel can be addressed independently via the
associated TFT’s and capacitors in the electronic back plane. Each pixel element
can be selected to stay “on” during the entire frame time. Since OLED is an
emissive device, the display aperture factor is not critical.
There are no intrinsic limitations to the pixel count, resolution, or
size of an active-matrix OLED display, leaving the possibilities for commercial use
open to our imagination. Also, because of the TFT’s in the active-matrix design, a
defective pixel produces only a dark effect, which is considered to be much less
objectionable than a bright point defect, like found in LCD’s.
25. Organic LEDs (OLED)
Robust Design - OLED’s are tough enough to use in portable devices
Viewing Angles –up to 160 degrees screens provide a clear image, even in bright
light.
High Resolution –Each pixel can be turned on or off independently to create
multiple colors in a fluid and smooth edged display.
“Electronic Paper” – OLED’s are paper-thin. Due to the exclusion of certain
hardware goods that normal LCD’s require, OLED’s are as thin as a dime.
Production Advantages –20% to 50% cheaper than LCD processes.
Video Capabilities –handle streamlined video, which could revolutionize the PDA and
cellular phone market.
Hardware Content – Lighter and faster than LCD’s. out of plastic and bendable.
do not need lamps, polarizers, or diffusers. Takes less power to run (2 to 10 volts).
26. 5DT Head Mounted Display
800x600 pixels
40o diagonal view
Organic LED
Frame sequential stereo
600 grams
$4k
29. Sensics piSight panoramic HMDs
Uses Organic LED
A series of micro-displays with
special optics to generate a
panoramic view
Weight 2 lbs (1 Kg)
SVGA input
resolution (2400x1729 pixels)
Field of view 179ο horizontal
by 58ο vertical
Binocular overlap 82ο
Cost? USD
sensics.com
30. Sensics xSight panoramic HMDs
Uses Organic LED
A series of micro-displays with
special optics to generate a
panoramic view
Weight 0.35 kg
DVI input
resolution (1680x1050 pixels)
Field of view 123ο horizontal
by 58ο vertical
Binocular overlap 123ο
Cost? USD
sensics.com
31. Sensics wireless HMD
A wireless video link optimized HMDs
Low latency (<30 msec)
Support for HD1080p high definition video at
full 60Hz frame rates.
Ability to use multiple transmitter/receiver pairs
simultaneously to drive multiple independent HMDs
Available battery option for both HMD and wireless video
link.
Integrated stereo audio
High-quality H.264 MPEG-4 compression
Wi-Fi wireless N technology which does not require line of
sight and works over large distances
32. Sensics wireless smart goggles
•On-board 1.2 GHz dual-core processor with
graphics and 3D co-processor running Android 4.0
•Allows execution of on-board applications without
requiring connection to a PC or a gaming console.
•First-person hand tracking provides real-time
tracking and location information of the user’s
hands.
•Hand position can be used to drive user interface,
identify gestures and interact with the game.
•Embedded head tracker for head angular position and
linear acceleration
•Dual SXGA (1280×1024) OLED displays.
•64 degree field of view for excellent immersion
•Embedded stereo audio and microphone
•Battery operated On-board WiFi and Bluetooth services.
35. 21” LCD display,
Resolution: SXGA
(1600x1200)
Weight: Counter-
Balanced;
No dead space but
High latencies due to
Third-party tracker
Virtual Window 3-D Display (courtesy of Virtual Research Co.)
36. Output Devices
Auto-stereoscopic displays
Do not require use of special glasses;
Passive auto-stereoscopic displays do not track
user’s head and thus restrict user’s position;
Active auto-stereoscopic displays track the head
motion and give more freedom of motion.
38. 18.1” LCD display,
Resolution:
1280x1024 (mono)
640 x 1024 (stereo)
Weight: 11.25 kg
Auto-stereoscopic 3-D Display(courtesy of Dimension Technologies Co.)
39. 40” LCD display,
Resolution:
1280x768 pixels
70o horizontal viewing
(7 to 15 feet)
Weight: 33.2 kg
SynthaGram 404 (courtesy of StereoGraphics Co. - $12,000)
40. 20” LCD display,
Resolution:
1600x1200 (mono)
100o horizontal viewing
(1.5 to 6 feet)
Weight: 8.4 kg
SynthaGram 204 (courtesy of StereoGraphics Co. - $3,000)
41. 18” LCD display,
Resolution:
1280x1024 (mono)
640 x 1024 (stereo)
Weight: 17 kg
Active auto-stereoscopic 3-D Display (courtesy of Dresden 3D Co.)
42. Single-user Auto-stereoscopic display…
The display redirects the appropriate frames to the right and left eye
so that each eye can only see the relevant frame.
The tracker locates each eye and sends the information to the
control box.
The control box then tells the LCD screen what pixels to display.
Through the optics system in the screen, the image will get directed
through the TFT directly to the appropriate eye.
A split second later it would do the same to the other eye.
Hence, creating a 3D image.
44. Multi-user Auto-stereoscopic display…
Multiple users can be tracked simultaneously and more pixels
can be opened up at any given time allowing light beams to be
directed simultaneously to more than one eye and more than one 3D
user.
Position finders already track pupils of multiple viewers with very
small delay. Good resolution but still shows some flicker.
OLED’s becoming mainstream can help eliminate flicker
What needs to be done?
Better displays (100Hz…120Hz)
Complete the multi-user concept
47. Autostereoscopic cell phones!
Ocuity (UK) and NEC make 2.5” diameter autostereoscopic cell
phones. InTouch mobile handset (TTPCom)
2.1” Transflective 2D/3D TFT-LCD
132xRGBx176 pixel display
Automatic control of 2D to 3D switching function
Running TTPCom WGE 3D stereo game demonstration
48. Holographic displays
• The image source is based on standard flat panel technology of
which the image is seen upon a nine optical layer glass panel.
Objects will appear to float in space.
•For the maximum 3D effect, the background seen through the
display should be several feet behind the display and dark in color.
http://www.eonreality.com/files/
brochures/eon_icrystal_hd.pdf
49. Holographic displays – EON TouchLight
• Bare-hand 3D interaction virtual reality display system
• VR scene can be zoomed, panned and rotated with both hands
• Uses image processing techniques to combine the output of two
video cameras placed behind a semi-transparent plane in front of
the user. Incorporates IR cameras and image processing board
http://www.eonreality.com/files/brochures/eon_touchlight_hd.pdf
50. Output Devices
Large Volume Displays
Allow several co-located users to view a
monoscopic or stereoscopic view of the virtual
world;
Can be classified as monitor-based large
volume displays or projector-based large
volume displays.
Allow more freedom of motion vs. personal
displays.
51. Output Devices
Monitor-based Large Volume Displays
Use active or passive glasses;
Several users can look at a monitor;
Can have a single monitor, or multiple side-by-
side monitors;
If side-by-side, image continuity becomes an
issue.
52. Untracked
and wireless
Tracked
and wireless
Active glasses
53. Output Devices Active glasses vs. FMDs
Some advantages:
no cables if head position is not tracked;
light and ergonomic (can be used over vision glasses);
work well with large volume displays.
allows full screen resolution 1280x1024
Some disadvantages:
lose 2/3 of image light intensity through LCD filtering;
require special CRT “stereo ready” that has twice
the hardware refresh rate (Hz) 120 Hz or more;
require direct line of sight for IR controller;
different viewing metaphor “through the window”.
54. Wireless – old model Active glasses
Wireless – new model
Wired to the synchronizing
jack of the graphics card
I-O Display Systems Inc. $99 vs. $1000 for StereoGraphics
wireless glasses
55. Wired/wireless glasses need a “stereo enabler” when connected
to a VGA card without a 3-pin mini DIN output jack)
60. Through the window metaphor
The projection factor is changes by a factor K which
such that
K = r (u – U) + U
Where: r is the responsiveness factor (optimally 1.25);
u is the current head distance from the screen;
U is the default distance (say 30 cm).
Unfortunately tracker jitter is amplified as well
64. Output Devices
Projector-based Large-Volume Displays
Old technology is CRT-based
(analog) three projector tubes (R, G, B);
Requires special “fast green” coating to avoid the
fogging due to fast switching (at 120 Hz);
Suffer from low luminosity problems
(200-300 lumens)
65. Output Devices
Projector-based Large-Volume Displays
Technology made transition from CRT-based
(analog) to Digital Micro-mirror Device (DMD)
(digital) projectors;
Workbench-type displays (Fakespace Responsive
Workbench, Barco Baron, V-desk, etc.)
Cave-type display (CAVE, RAVE)
Wall-type displays
Domes
66. Output Devices
Digital Micro-mirror Device
Display
Light intensities are much larger
than for CRT-based projectors
300 lumens to 1500 or more lumens
Thus ambient light does not
hinder image quality
67. Tilted surface
Viewing Cone
Reflector mirror
Floor CRT projector
(not shown)
The old Fakespace “ImmersaDesk” workbench
68. IR Controllers CRT Projector
Mirrors Tilting mechanism
Baron workbench (courtesy of BARCO Co.)
70. CRT Projector
Screen
Mirror
CAVE 3-D large volume display (courtesy of Fakespace Co.)
71. CAVE 3-D large volume display (courtesy of Fakespace Co.)
72. RAVE (“Re-configurable Virtual Environment”)
Modular construction that allows various viewing
configuration, from flat wall, to angled theater, to
CAVE;
Vertical wall image 2.3 m X 2.4 m;
Several CRT projectors (260 lumens, 1280x1024
resolution);
Takes 30 minutes or less to reconfigure
73.
74. New types of stereo displays
Such as BARCO Trace
Driven by Barco Galaxy Stereo DLP projectors 3000 Lumens;
800:1 contrast ratio
WARP geometry distortion for edge matching;
1400 x 1050 pixel resolution
70 inch diagonal screen
active stereo glasses
75. Microsoft SURFACE
one large display (projector)
five infrared cameras
tracks user’s finger contact with the surface
PC included in the enclosure
76. 40” diagonal Samsung SUR 40 new SURFACE
1920 x 1080 resolution
Dual-core CPU, AMD HD6750M GPU
Viewing angle 178 degrees
PixelSense™ to sense fingers and objects touching the
screen. It sees and reacts to light – taking sixty pictures every
second in a way that is similar to a movie camera.
77. MultiTouch technologies
* Two layers of electrodes. Electrodes
parallel in same layer and orthogonal
to to the other layer
* Capacitive sensors can be constructed
from copper or Indium tin oxide
(ITO).
* Methods for measuring capacitance
are Relaxation Oscillator(in figure),
Charge Time, Voltage Divider, Charge
Transfer, Sigma-Delta Modulation.
* Self capacitance, mutual capacitance
78. Resistive
* Two ITO patterned plates separated by spacing dots.
The top layer has ITO columns and bottom layer has
ITO rows.
* 5V is applied to a is applied to a column, and voltage
is measured at every horizontal sensing line. In order to
prevent masking, inactive sensing lines are set to high
impedance
Multi touch Optical technologies
Images from www.touchuserinterface.com
Optical Imaging : two optical sensors track the
movement of any object close to the surface by
detecting the interruption of an infra-red light source.
79. Multi touch Optical technologies
Rear Diffused Illumination : Infrared light is shone
from the opposite side of the touch surface. Finger
interrupts the infrared light, reflects back to the
camera .
Frustrated Total Internal Reflection (FTIR) : floods
the inside of a piece of acrylic with infrared light by
trapping the light rays within the acrylic.
Kinect : Uses near infrared transmitter creates a
pattern of near-infrared dots.
In-Cell : 2D grid of retro-reflective optosensors
which are placed behind an LCD panel.
http://www.articlesbase.com/electronics-articles/technologies-of-multitouch-part-3-optical-technologies-
5485162.html
80. Output Devices
Wall-type displays
Accommodate more users
Using a single projector on a large wall means small
image resolution;
Thus tiled displays place smaller images side-by-side
so they need multiple projectors;
Images need to have overlap, to assure continuity;
However overlap from two projectors means intensity
discontinuity (brighter images in the overlap areas)
Projectors need to modulate intensities to dim their
light for overlap pixels.
86. Wall and Dome-type displays
Advantages:
Accommodate more users (tens to hundreds)
Give users more freedom of motion;
Disadvantages:
Large cost (up to millions of dollars);
Even with multiple projectors, resolution is much
lower than for CRTs (because the area is large).
Example PanoWall has 200,000 pixels/m2 while a
monitor has 18,200,000 pixels/m2
To have equal numbers of pixels/unit are – more
projectors (military)
87.
88. Output Devices
3-D Audio Displays
Definition:
Sound displays are computer interfaces
that provide synthetic sound feedback to
the user interacting with the virtual world.
The sound can be monoaural (both ears
hear the same sound) or binaural (each
ear hears a different sound).
89. Output Devices
3-D Audio Displays
3-D audio should not be
confused with stereo sound;
Human hearing model;
HRTF-based 3-D sound;
Convolvotron;
3-D sound cards.
95. Output Devices
NASA again a pioneer in 3-D sound
put microphones in dummy heads;
played localized sound and measured
signal;
Determined the HRTF;
Worked on first circuitry;
101. Cross-talk effect
Sound from one speaker reaches both ears:
[ ] [ ][ ]
Yleft
Yright
=
Hl,l Hl,r
Hr,l Hr,r
Sleft
Sright
where Hl,l is the HRTF between the left speaker and the left ear,
Hl,r is the HRTF between the right speaker and the left ear,
Yleft is the sound reaching the left ear
Yright is the sound reaching the right ear
102. Cross-talk effect cancellation
Sound from both speakers is adjusted such that:
[ ] [ ][ ]
-1
Sleft Hl,l Hl,r Yleft
=
Sright Hr,l Hr,r Yright
where Yleft and Yright are known (the output if the convolving
process)
103. Commercial 3D Sound Cards
What they have to offer:
Digital Output
Multi-speaker compatibility
(7.1 channel format allows for 8 speakers)
Positional Audio
offers 3D dimensions of sound
104. USB 3D Sound Adapter
•Supports 3-dimensional sound
•Virtual 5.1 sound effects
•USB powered, no external power required
•Digital Class-B power amplifier
•27 Environment sound effects
•10 Band / Pre-set equalizer
•$30
105. Sabrent 7.1 PCI Sound Card
Internal connectors
8-channels of audio,
a PCI interface,
3D sound and great quality
$14
PCI Bus connector
Audio ports
HRTF-base 3D positional audio, supporting
DirectSound 3D, EAX and A3D interface.
MIDI Game port Support multi-speaker output to
2/2.1/4.1/5.1/7.1 speakers. Support Karaoke
key, Echo sound effects.
106. Creative Labs Sound Blaster Audigy 4 Pro
Aureal 3D (A3D)
•Two Versions
•1.0 was very similar to DS3D plain Microsoft
DirectX component for positional audio
•2.0 could more accurately simulate how sound
sources in a complex environment behave
•An extension of DS3D (by itself,
its just reverb)
•PC environmental reverb
standard created by Creative Lab
•All sound cards can have EAX
capability
•$150
108. Output Devices
Haptic Interfaces
Haptics…
Comes from Greek Hapthai meaning the
sense of touch;
Groups touch feedback and force feedback
109. Output Devices
Touch Feedback
Relies on sensors in and close to the skin;
Conveys information on contact surface
geometry, roughness, slippage, temperature;
Does not actively resist user contact motion;
Easier to implement than force feedback.
110. Output Devices
Force Feedback
Relies on sensors on muscle tendons and
bones/joints proprioception;
Conveys information on contact surface
compliance, object weight, inertia;
Actively resist user contact motion;
More difficult to implement than touch feedback
(no commercial products until mid 90s).
111. Haptic Interfaces
Human touch sensing mechanism
Most touch sensors are on the hand (much less
density on other parts of the body);
Four primary types of sensors:
40 % are Meissner’s corpuscles – detect movement
across the skin – velocity detectors
25% are Merkel’s disks – measure pressure and vibrations
13 % are Pacinian corpuscles – deeper in skin (dermis) –
acceleration sensors. Most sensitive to vibrations of about 250 Hz
19% are Rufini corpuscles – detect skin shear and temperature
changes
113. Haptic Interfaces
Sensorial adaptation
Measure the decrease in electrical signals from
the skin sensor over time, for a constant stimulus;
If the sensor produces a constant electrical
discharge for a constant mechanical stimulus –
It is called “Slow Adapting” (SA);
If the rate of electrical discharge drops rapidly
over time for a constant stimulus – called
“Rapidly Adapting” (RA)
114. Haptic Interfaces
Spatial resolution
Measure the receptive field size of a sensor;
If the sensor has a large receptive field – it
has low spatial resolution (Pacinian and Ruffini)
SA-II, RA-II
If the receptive field is small – has high
spatial resolution (Meissner and Merkel) SA-I,
RA-I
118. Haptic Interfaces
Maximum and sustained force exertion
Maximum force exerted during “power” grasp
Averages 400 N (male) and 225 N (female);
Looking at body location, force output
Grows from 50 N at PIP finger joint, to 100 N
at shoulder;
Sustained force feedback is much smaller than
maximum, owing to fatigue and pain
119. Haptic Interfaces
Fatigue measured as a function of % Maximum
Voluntary Contraction (MVC) and rest cycle
120. Haptic Interfaces
Haptic feedback actuators
Need to maximize power/weight ratio;
Need to have high power/volume ratio;
Need to have high bandwidth;
Need to have high dynamic range
(fidelity);
Need to be safe for the user
- None of the current actuator technology
satisfies all these requirements
126. Output Devices
Temperature feedback
Added simulation realism by simulating
surface thermal “feel”;
No moving parts;
Uses thermoelectric pumps made of solid-state
materials sandwiched between “heat source” and
“heat sink”;
Single pump can produce 65°C differentials;
128. User comfort zone
13-46°C
If system fails
Heat travels back
Through the pump
and can burn
skin
Temperature feedback actuator control
129. Output Devices
Force Feedback Interfaces…
Need mechanical grounding to resist user
motion;
Can be grounded on desk, wall, or on user body;
More difficult to construct and more expensive
Than tactile feedback interfaces
135. NOVINT FALCON
3 DOF Force Feedback
• 3 DOF (right-left,
forward/backward, up/down) 75 ×75
× 75 mm
• rumble, vibrations
•3D exploration and textures
• Dynamic effects (inertia, weight,
momentum)
• Cost less than $300
• Resolution >0.06 mm
• Max continuous force 10 N
• Stiffness 5 N/mm
• Connectivity USB 2.0
• 1 kHz control bandwidth
•http://home.novint.com/
136. The Haptic Master
3 DOF cylindrical robot
Max force output 250 N
Stiffness 50 N/mm
Uses force-in, position-out
arrangement
137. Exoskeleton
over CyberGlove)
Cables and pulleys
16 N/finger (continuous?); Weight 539 grams;
remote electrical actuators in a control box.
The CyberGrasp force feedback glove
140. 6 DOF mechanical arm
Wrist position and
Force feedback –
No need for a tracker
Allows simulation of weight and inertia, not possible with
glove-only interfaces CyberForce interface (introduced recently)
143. Olfactory Feedback
Olfactory feedback refers to the inclusion of the sense of
smell as an output in a virtual world or simulation.
Aroma therapy has proven to be an effective form of
treatment for rehabilitation and treatment.
The notion of “olfactory-evoked recall” can be vital to
immersion in a VR world. “Smell has the greatest impact
on human emotions.” (Enviroscent)
144. Challenges in Smell Technologies
No “primary smells” unlike colors, makes smell synthesis
difficult.
Consumable.
More difficult to modulate and control compared to sound
and light.
Size.
145. Applications of Olfactory Stimuli
Used by Dr. Albert Rizzo at USC for treatment of
PTSD patients.
USAF researched FiVe FiRe training system for fire fighters which
includes multiple odors for burning objects.
Entertainment purposes; the original
Smell-o-Vision was used in theaters to
add additional movie going enjoyment.
146. Technologies for Olfactory Stimuli
Enviroscent has a line of smell machines such
as the ES-1 Fragrance Machine.
Scents in Metagel dispersion cartridges
-8 different scents gunpowder, diesel, burning
rubber, body odor, etc.)
- through which compressed air is pumped to
carry the smell temporarily into the users
simulation space.
- researchers think PTSD that this adds to the
sense of presence. Since the olfactory bulb is
closely linked to areas of the limbic system
implicated to be involved with memory and
emotion
147. Smell as Input
Smell is actually a relatively common input source with
applications in medicine, security, and more.
Can be used to detect pollutants, identify diseases,
or even used to identify an individual.
Typically designed as an array of chemical sensors.
151. VR for Addict Rehabilitation
Virtually Better, a company focused on VR
rehabilitation techniques, in cooperation with
addiction researcher Dr. Bordnick of the
University of Houston, have created VR
Simulations of addict tempting environments.
Video Courtesy ScienCentral, Inc.
152. Further Reading and Resources
Albert Rizzo article
Smell Cannon
UoW Document on Olfaction for VR
Sony/Matsushita Smellovision
Using VR with Olfaction for Addiction Rehabilitation
Robot Nose
