Marcelo Knörich Zuffo Universidade de São Paulo Escola Politécnica Laboratório de Sistemas Integráveis Campus Party São Paulo, 26 de Janeiro de 2010

1. The VR Continuum: From CAVEs to Digital
TV, How VR and AR will change everyday
life in the next decade
Marcelo Knörich Zuffo
Universidade de São Paulo
Escola Politécnica
Laboratório de Sistemas Integráveis
Campus Party
São Paulo, 26 de Janeiro de 2010

2. Virtual Reality
Virtual Reality
Immersion
Immersion
Interaction Interaction
Imagination
Realism
(Burdea, 2003) (Lopes e Zuffo, 2006)
SIACG 2009 2

3. Historical Perspective - Immersion
– Rupestrial Painting in caves during the Paleolithic age
– flares and smoke completed the artificial feeling of immersion.
– Pictures of immersive environments in the cave of Lascaux in
France, magdalenian men 15.000 B. C.
SIACG 2009 3

4. REALIDADE VIRTUAL
Historical Perspective - Immersion
• Pictorial Allegory of the Cave of Plato, 427 BC, The Republic
Book VII
SIACG 2009 4

5. REALIDADE VIRTUAL
Historical Perspective - Immersion
• Gothic cathedrals, XII century to the end of the XV century.
• Candles, Vitrals, choirs, organs, incense burning and priest
talks integrates a fully immersive
SIACG 2009 5

6. Historical Perspective - Immersion
• TV Evolution in the Past 85 Years
• Family Oriented
• Interactivity is limited
SIACG 2009 6

7. Where we will stop ?
• 108 Inches LCD TV Set
SIACG 2009 7

8. Where we will stop ?
• Projections upon the
Banespa Building in São
Paulo City
SIACG 2009 8

9. Historical Perspective – Interactive
Immersion
• First immersive and interactive electronic systems
Sensorama Sketchpad Ultimate Display
(Morton Heilig,1962) (Sutherland MIT,1963) (Sutherland MIT, 1967)
SIACG 2009 9

10. REALIDADE VIRTUAL
Historical Perspective - Immersion
• Evolution of Computer Graphics and its applications
1970
Desenho Assistido por
Computador
1980
Desenho e Design Assistido
por Computador
1990
Fotorealismo e
Simulação Física
(Virtualização)
SIACG 2009 10

11. Historical Perspective – The Cave
• The first CAVE (Cave Automatic Virtual Environment) was
established at the University of Illinois, Chicago, by
researchers from Thomas Fanti, Daniel Sandin and Carolina
Cruz-Neira in 1991 (Cruz-Neira, 1993)
• Motivation
– Scientific Visualization
• First Fully Immersive VR
Picture of The First Cave 1991
SIACG 2009 11

12. Microelectronics advances in past 30 years
Materials used for Integrated Circuit Manufacturing (Source: Intel)
MOS Transistor Size 2001-2011 (Source: Intel)
SIACG 2009 12

13. Digital Light Processing
• DLP Technology (Digital Light Processing) was invented in
1987 by Dr. Larry Hornbeck at Texas Instruments
• The DLP projection systems are devices MEMSs (Micro-
Electro-Mechanical Systems) called DMD (Digital Micromirror
Device).
Chip DMD DMD Surface DMD in Operation
fonte: Texas
SIACG 2009 13

14. REALIDADE VIRTUAL
A evolução das CPUs e GPUs
• GPU Graphics Processing Unit
• CPU Computer Processing Unit
Lei de
Moore
Evolução dos microprocessadores (CPUs) Evolução das CPUs e GPUs
SIACG 2009 14

15. CPU x GPU: Desempenho
– GPU: Visualização
• Ex.: Placa Gráfica ATI X800XT (R420):
– 520 MHz * 16 pipes * 4 wide * 1 flop/inst * 1 inst/cycle
– 66.5 GFlops (109 operações de ponto flutuante/segundo)
– CPU: Simulação
• Ex.: Microprocessador Pentium P4, 3GHz
– 6 GFlops
Foto do Chip da GPU NVIDIA G6800
Placa Dual CPU 2 Core Dual x 4 GPU em processo de 65µm
SIACG 2009 15

16. CAVERNA Digital
• 5 Screens
• 1 Mpixel per screen
• Pixel size 3mm x 3mm
• 600 lumens/projetor
• total 3.000 lumens
• Frame Rate 60Hz Upper View
General View General View User
SIACG 2009 16

17. CAVERNA Digital
VRcluster 2001
VRcluster 2003
VRcluster 2006
Internet 2 FAPESP
Kyatera
Central de Controle
Rastreador
Projetores
SIACG 2009 17

18. VR Clusters Evolution at USP
(, SIGGRAPH
2002)
(IEEE VR 2003)
2001 – 1a Geração: 2003 - 2a Geração: 2006 – 3a Geração:
• 6 nós • 8 nós • 8 nós
• dual Pentium Xeon 1.76GHz • dual Pentium Xeon 3.0GHz • dual Opteron dual Core 3.0GHz
• 1 GB RAM • 2 GB RAM • 8 GB RAM
• placa de video Wildcat 7210 • placa de video NVidia • duas placas de video NVidia
(256MB), AGP PRO, Quadro FX 3000G - 256MB - Quadro FX 4400 (SLI) - PCI
• 2 HDs 80GB SCSI AGP 8x Express x16 dual
• 1 saída de vídeo por nó • 2 HDs 80 SCSI • 2 HDs SATA 160GB
• 2 saídas de vídeo por nó • 4 saídas de vídeo por nó
SIACG 2009 18

19. CONSIDERAÇÕES FINAIS
VRClusters on a Chip
Multiple GPUs and CPUs Integrated in the same die
• 8 – 64 CPUS
• 8 - 64 GPUS
• 1GB Unified cache
• 50 - 250 Bgates
• Integrated Energy Management
• Integrated Memory Management
• Dual CPU-GPU core to be announced
by AMD in 2009
SIACG 2009 19

20. Volume Visualization for anatomical analysis
Real Time Visible HumanTM analysis
SIACG 2009 20

21. Molecular visualization
Electrofisiology
learning
FAPESP
Kyatera-Tidia
Software: Jinx
Potassium chanel
polipeptidic chain
User at the cave Diffusion Hemolisina-tree
SIACG 2009 21

22. Cultural Heritage
reconstruction of São Paulo 1911
Baruel Building
College Square (1st São Paulo Civilized Location)
SIACG 2009 22

23. The Digital TV Transition in Brazil
• It started in 2003
• In our model, the University had a key role
discussing and proposing inovations.
• Digital TV = Embedding a Computer on a TV
• So TVs will be musch more interactive and
immersive in the followng years.
Marcelo Knörich Zuffo Depto. de Engenharia de Sistemas Eletrônicos 23
Escola Politécnica da USP

24. Analog Terrestrial TV in Brazil
• Major source of information
and entertainment
• Universal distribution
• 2,3 TV sets / house
• Near 100 Million TV sets
• 7 hours a day
SIACG 2009
Source: IBGE-PNAD/2003

25. Analog Scenario
Emissora de TV Aberta Analógica Cidadão
Distribuição
Produção Transmissão Recepção
Analógico
Digital Analógica Analógica
Digital
SIACG 2009

26. Digital Scenario
Emissora de TV Digital Cidadão
Produção Distribuição Transmissão
Recepção
Digital Digital
Digital Digital
SIACG 2009

27. The 4901 ACT
• Government ACT4901, 26 November 2003
– The Brazilian Digital TV System is established
• Arguments
– Wide Access to population
– Interoperable
– Mobile and portable
– Need to consider regional differences
– Usability and Accessibility
– Social Implications
• To promote the digital inclusion and social inclusion
SIACG 2009

28. Research Network in 2004 and 2005
• Between 2004 and 2005 we created the largest
research network ever in Brazil:
– 22 Consortia
– Near 100 R&D Groups
– Near 1.500 Researchers involved
– Everything was debated, detailed and demonstrated at
USP Campus in 9 December 2005.
SIACG 2009

29. Some Snapshots from that period
Some prototypes
Huge Press Coverage
A lot of collaboration
29

30. The 5820 ACT
• 2006 was an election year
• So things moved fast:
-The conclusions are:
We will write an standard considering
– I Multiformat: HDTV (High-def), SDTV (Standard-Def)
and LDTV (Low-def)
– II Multiprograming- multiple programs in the same
channel
– III - Interactivity (middleware and connectivity).
– Comercial launch in 3 Dezember 2007
– Forum creation: Industry/University/Broadcast/Goverment
SIACG 2009 30

31. The SBTVD Standard
Figura: Normas que definem o Sistema Brasileiro de TV Digital
31

32. N01 Transmission System
 BST-OFDM modulation
 Channel of 6 MHz
 13 segments of 428.57 kHz for
information + 1 divided into two
bands of custody
 3 modes of operation with
different spacing between
carriers
BST-COFDM Modulation no SBTVD
 Displacement of 1 / 7 MHz for
the top carriers
Serviço Modulação Taxa mínima (kbps) Taxa máxima (kbps)
LDTV (1 segmento) QPSK 280,85 595,76
SDTV (3 segmentos) 16-QAM 1685,13 3574,56
HDTV (9 segmentos) 64-QAM 7583,13 16085,52
Service Configuration upon a DTV Channel

33. Multiprogramming
• Several programs in the
same channel
– Around 19Mbits/second/
channel
– We do data multiplexing
– 70 channels means 1.4
Tbits/second
– Near 540Tbytes can be
broadcast on a single day
SIACG 2009

34. N01 – Transmission System
Figura: Esquema de transmissão no SBTVD
Figura: Faixa de freqüências da TV Digital no Brasil

35. Multiformat
240 linhas
• 1080 lines(HD) LDTV
4:3 480 linhas
• 720 lines(HD) SDTV
4:3
720linhas
• 480 lines(SD)
1080 linhas
EDTV
16:9
• 240 lines(LD)
HDTV
16:9

36. interactivity
• From 2009 will viewers can interact with the TV via
remote control:
– Local interactivity: no connectivity
• Station sends data that can be navigated
– Full interactivity with intermittent connectivity
• User sends the broadcast short messages via mobile or fixed
– Full interactivity with permanent connectivity
• User connected permanently to the station via a broadband
Internet

37. DTV Receiver
256Mbytes
32 Bit CPU Navigation and
4 USBs Voting buttons
1080i
OpenGL
Ethernet
Ethernet Access
RF Input
USB Port
Analog Output
HDMI Interface
37

38. Future perspectives
Interactive electronics will support
Virtual Reality
– More immersiveness immersion
– More interaction
– More modular and flexibe
– More complex and integrated
– More simple and accessible Interaction
– More cheaper
– And more and more popular
Realism
• Converging DTV and VR: interactive 3DTV and make that
accessible for all
• Basically TV Sets will follow now the MOORE LAW !
38

39. Immersive and Interactive 3DTV
• More simple and accessible
• Low cost display and interactive devices
• Adoption of HDTV 1080i on the Brazilian DTV system
Interactive Remote 3DTV HDTV 1080p LDC Display Auto-stereoscopic display
Control Phillips – Lenticular Using lenticular lenses
Sansung Active Stereo
Mitshubishi Active Stereo
39

40. Visual cluster
• Clustering projectors for ultrarealistic immersive
spaces
• “Casual projector alocation
SIACG 2009 40

41. Visual Cluster Demos
Google Earth Cisne Branco
Rio de Janeiro View Port Resizing
41

42. 3D and Panoramic Video Display
Virtual Rio
HD3D
Virtual Rio 240o
SIACG 2009 42

43. Receptor de TVDigital Acessível
• Parceria com a Secretaria do Estado dos Direitos
da Pessoa com Deficiência (SEDPcD)
• Projeto visa universalizar o SBTVD pela inclusão
dos brasileiros com deficiência

44. Obrigado !
mkzuffo@lsi.usp.br
Marcelo Knörich Zuffo Depto. de Engenharia de Sistemas Eletrônicos 44
Escola Politécnica da USP