2015 crs4-seminar-massive-models-full

www.crs4.it/vic/
Visual Computing 1:
Scalable Graphics
Marco Agus
Marcos Balsa
Enrico Gobbetti
Fabio Marton
Jose Díaz
CRS4/ViC
June 2015

E. Gobbetti, F. Marton, Massive models, June 2015
CRS4
• Center for Research, Development, and
Advanced Studies in Sardinia
– Interdisciplinary research center focused on computational
sciences
• Strong research program in CS + apps to IS / E&E / Bio
• Leading computational + bioprocessing facilities
– Located in the POLARIS Science and Technology Park (Pula,
Sardinia, Italy)
– Operational since 1992, RTD staff of ~130 people

CRS4: Visual Computing Group
• Main activities
– Visual Computing RTD
– 3D scanning
• Staff
– Director: Enrico Gobbetti
– Main lab: Marco Agus, Marcos
Balsa, Fabio Bettio, Jose Díaz,
Fabio Marton, Gianni Pintore,
Ruggero Pintus, Antonio
Zorcolo, Roberto Combet,
Emilio Merella, Alex Tinti
– Secretariat: Katia Brigaglia,
Cinzia Sardu

CRS4: Visual Computing Group
• Externally funded research program
– Many EU projects (4FP-7FP) and national projects
– Industrial projects from GEXCEL, DIES, ENEL, …;
• Active presence in the scientific community
– Many scientific collaborations: ISTI, UZH, JHU, HOL, KAIST, …
– Over 150 refereed publications
– Conference organization
• Software products and technology transfer
– “Industrial software”: point cloud visualizers, surgical simulators,
sciviz tools,
– “Public software”: geoviewing, neuroimaging tools, …
– “Cultural Heritage Installations”, …
• Educational Activities
– University courses; PhD students; Tutorials; ITNs

Research
• RTD (mostly)
connected to 3D
massive models…
– Many domains:
simulations, multimedia
engineering, medical
imaging, 3D scanning,
geospatial data, …
– RTD covers acquisition,
processing, distribution,
rendering, exploration
• … plus other topics
– Simulators, 3DTV, …
6

Massive research…
• How to efficiently acquire/create massive models?
– Computational photography, 3D scanning pipe-lines, 3DTV
• How to efficiently process them?
– Stream-processing, multiresolution, external memory algorithms,
parallel programming, GPGPU
– Specific processing methods
• How to efficiently store and distribute them?
– Multiresolution, adaptive streaming, compression
• How to efficiently render/interact with them?
– Multiresolution, adaptive rendering/collisions, out-of-core methods,
GPU programming, parallelization, rasterization, ray-casting
• How to efficiently explore them?
– Novel 3D displays, specific interaction techniques
– Portable devices
7

Effective acquisition of color and
shape of 3D models
• Combining acquired
colorimetric and
geometric
information using
multiple sensors
– acquiring 3D models in
highly cluttered
environments
– mapping photographic
datasets to dense 3D
models acquired with
laser scanning
– capturing indoor scenes
8
SELECTED PUBLICATIONS:
ACM JOCCH 2015 A Fast and Robust Framework for
Semi-Automatic and Automatic Registration of
Photographs to 3D Geometry
ACM JOCCH 2015 Mont'e Scan: Effective Shape and
Color Digitization of Cluttered 3D Artworks
C&G 2014 Automatic Room Detection and Reconstruction
in Cluttered Indoor Environments with Complex Room
Layouts.

• Large scale terrain mapping
– Parallel color/geometry fusion,
compression, Online regional
systems
• Streaming reconstruction
– Streaming MLS implementation
– Parallel, fast, scalable to gigabyte-
sized models
• Registration and blending
– Auto registration geometry to
geometry and photo to geometry
– Streaming seamless photo
blending on massive point clouds
and triangle meshes
Scalable surface processing
9
ACM JOCCH 2015 A Fast and Robust Framework for
Semi-Automatic and Automatic Registration of
Photographs to 3D Geometry
ACM JOCCH 2015 Mont'e Scan: Effective Shape and
Color Digitization of Cluttered 3D Artworks
C&G 2014 Automatic Room Detection and Reconstruction
in Cluttered Indoor Environments with Complex Room
Layouts.

Rendering and streaming of terrains
and urban environments
• Batched Dynamic
Adaptive Meshes
– First GPU-accelerated
seamless variable
resolution methods for
terrain rendering
(*-BDAM)
• Compressed LODs for
massive urban
models
– Blockmaps framework
10
EG 2007: Ray-Casted BlockMaps for Large Urban
Models Visualization.
EG 2006: C-BDAM - Compressed Batched Dynamic
Adaptive Meshes for Terrain Rendering.
EG 2003: BDAM - Batched Dynamic Adaptive
Meshes for High Performance Terrain Visualization.
Second Best Paper Award.

Processing, distribution, and rendering of
massive dense 3D meshes and point clouds
• Coarse grained multiresolution
model based on hierarchical
volumetric decomposition
– First GPU bound technique for
massive meshes
– General framework based on an
extension of the Multi-Triangulation
(GPU-MT)
– Optimized representation based on a
conformal hierarchy of tetrahedra
– Packed representation based on
quadrangulation (QuadPatches)
• Demonstrated on a number of
test cases, including all Digital
Michelangelo and Mont’e Prama
models
– 11
IEEE VIS 2005, Batched Multi Triangulation. In
Proceedings IEEE Visualization
C&G 2004, Layered Point Clouds - a Simple and Efficient
Multiresolution Structure for Distributing and Rendering
Gigantic Point-Sampled Models.
SIGGRAPH 2004, Adaptive TetraPuzzles Efficient Out-of-
core Construction and Visualization of Gigantic Polygonal
Models.

massive dense 3D meshes and point clouds
• Coarse grained multiresolution
model based on hierarchical
volumetric decomposition
– First GPU bound technique for
massive meshes
– General framework based on an
extension of the Multi-Triangulation
(GPU-MT)
– Optimized representation based on a
conformal hierarchy of tetrahedra
– Packed representation based on
quadrangulation (QuadPatches)
• Demonstrated on a number of
test cases, including all Digital
Michelangelo and Mont’e Prama
models
– 12
IEEE VIS 2005, Batched Multi Triangulation. In
Proceedings IEEE Visualization
C&G 2004, Layered Point Clouds - a Simple and Efficient
Multiresolution Structure for Distributing and Rendering
Gigantic Point-Sampled Models.
SIGGRAPH 2004, Adaptive TetraPuzzles Efficient Out-of-
core Construction and Visualization of Gigantic Polygonal
Models.

huge complex 3D models
• Visualization of very large
arbitrary surface models
(CAD/simulation) based on
volumetric multi-scale
approximations
– Far Voxels, …
• Coherent Hierarchical
Culling, Ray Tracing
– Far Voxels, CHC+RT
• Image-based methods
– ExploreMaps, …
13
Eurographics 2015 CHC+RT: Coherent Hierarchical
Culling for Ray Tracing
Eurographics 2014: Efficient Construction and Ubiquitous
Exploration of Panoramic View Graphs of Complex 3D
Environments
Siggraph 2005 Far Voxels - A Multiresolution Framework
for Interactive Rendering of Huge Complex 3D Models on
Commodity Graphics Platforms

Massive volumetric compression and
rendering
• Render models of potentially
unlimited size on current
GPU platforms.
• Methods based on
– adaptive out-of-core
multiresolution techniques
– visibility feedback
– single-pass GPU raycasting
framework
• Novel compression
techniques based on
– tensor decomposition
– sparse coding
• Fully interactive
performance on datasets of
many GVoxels
14
CGF 2014 State-of-the-art in Compressed GPU-Based
Direct Volume Rendering.
Eurovis 2012 COVRA: A compression-domain output-
sensitive volume rendering architecture based on a sparse
representation of voxel blocks.
IEEE VIS 2011 Interactive Multiscale Tensor
Reconstruction for Multiresolution Volume Visualization.

Interactive visualization on remote,
web, and mobile devices
• Compact multiresolution
mesh and point-cloud
representations for
embedded platform and
web scripting
– Asymmetric compression
framework, LODs, constrained
techniques, image-based
representations
– ExploreMaps, Adaptive Quad
Patches, Compressed
TetraPuzzles
– WebGL, Android, iOS
15
Eurographics 2014 ExploreMaps: Efficient Construction
and Ubiquitous Exploration of Panoramic View Graphs of
Complex 3D
Web3D 2013 Compression-domain Seamless
Multiresolution Visualization of Gigantic Meshes on Mobile
Devices.
Web3D 2012 Adaptive Quad Patches: an Adaptive
Regular Structure for Web Distribution and Adaptive
Rendering of 3D Models. Best Paper Award

• 3D multi-projector display with
special holographic screen
– HW by Holografika
– Objects appear floating in space
– Developed calibration method,
rendering systems, MCOP technique
for rendering
– Cluster-parallel visualization
• Special rendering techniques
for surfaces and volumes
– Illustrative methods on view-
dependent displays
• Special navigation techniques
– Maintain focus on display’s sweet spot
– Reduce DOFs to support casual users
Parallel multiresolution visualization
on light field displays
16
The Visual Computer 2010 View-dependent
Exploration of Massive Volumetric Models on Large
Scale Light Field Displays.
Eurographics 2008 GPU Accelerated Direct Volume
Rendering on an Interactive Light Field Display.
C&G 2008 Scalable Rendering of Massive Triangle
Meshes on Light Field Displays.

Novel user interfaces for exploring
3D models
• Natural interfaces for
scene navigation and
data exploration
– Assisted 3D scene
exploration
– Information presentation
in scenes with
annotations
– Exploration tools for
volumetric data
– Device-specific UIs (light-
field display, dual-display
setups, mobile touch
screens, …)
17
EuroVis 2015 Adaptive Recommendations for Enhanced
Non-linear Exploration of Annotated 3D Objects.
JOCCH 2014 IsoCam: Interactive Visual Exploration of
Massive Cultural Heritage Models on Large Projection
Setups
C&G 2012 Natural exploration of 3D massive models on
large-scale light field displays using the FOX proximal
navigation technique

TODAY’S SEMINARS
18

Today’s seminars
9:30 Enrico Gobbetti Opening
9:45 Fabio Marton Tecniche per la visualizzazione in tempo reale
di modelli 3D di grandi dimensioni
11:00 Break
11:30 Marcos Balsa
Marco Agus
Mobile Graphics: panoramica di applicazioni
grafiche mobili e integrazione con soluzioni
multi-risoluzione
13:00 Break
14:30 Enrico Gobbetti
Fabio Marton
State-of-the-art in Compressed
GPU-Based Direct Volume Rendering: part 1
Models and Preprocessing
16:00 Break
16:30 Jose Díaz
Enrico Gobbetti
State-of-the-art in Compressed
GPU-Based Direct Volume Rendering: part 2
Rendering
18:00 Questionario di valutazione
19

Follow-up on 30/6/2015 dedicated to
Cultural Heritage Computing
9:30 Enrico Gobbetti Opening
9:45 Ruggero Pintus
Gianni Pintore
Shape Modeling and acquisition
11:30 Break
12:00 Fabio Bettio Effective Shape and Color Digitization of
Cluttered 3D Artworks
12:30 Gianni Pintore Simple Acquisition and Reconstruction of Multi-
room Indoor Structures
13:00 Break
14:30 Marcos Balsa
Marco Agus
Exploration of Complex and Annotated 3D
Models
16:00 Break
16:30 Enrico Gobbetti
Ruggero Pintus
Geometric Analysis for Cultural Heritage
17:30 Questionario di valutazione
20

… AND MORE!
For more information: www.crs4.it/vic/

www.crs4.it/vic/
Massive models exploration:
a short overview
Marco Agus
Marcos Balsa
Enrico Gobbetti
Fabio Marton
Jose Díaz
CRS4/ViC
June 2015

MASSIVE MODEL RENDERING
A bit of context…
23

Context and Motivation
• Explosion of data in all areas of
science, engineering, health and
business applications, driven by
improvements in hardware and
information processing technology
– Acquisition: 3D imaging, remote sensing,
range scanners, massive picture collections,
ubiquitous sensing devices, …
– Computing: modeling, simulations…
• Need for novel tools, techniques, and
expertise!
• Our focus is 3D data
– Wide and deep impact on a variety of
domains
24

Application domains / data sources
• Many important
application domains
• Today’s models
exceed
– O(108-1010) samples
– O(109-1011) bytes
• Varying
– Dimensionality
– Topology
– Sampling distribution
Terrain Models
2.5D – Flat/Spherical – Dense
regular sampling
Urban models
2.5-3D – Block structure
Laser scanned models
3D – Moderately simple topology –
low depth complexity - dense
CAD models
3D – complex topology – high depth
complexity – structured - ‘ugly’ mesh
Natural objects / Sim. results
3D – complex topology + high depth
complexity + unstructured/high
frequency details
Volumetric models
3D/4D – semitransparent volumes

Massive model rendering
• To explore massive 3D scenes we need to
transform them at interactive into a synthetic
image that can be displayed on the screen
• Two main families of algorithms
– Raytracing algorithms
– Rasterization-based algorithms
I/O
Storage Screen
10-100 Hz
O(N=1M-100M) pixels
O(K=unbounded) bytes
(triangles, points, …)
Limited bandwidth
(network/disk/RAM/CPU/PCIe/GPU/…)
View parameters
Projection + Visibility + Shading

Basic Ray Tracing vs. Rasterization
• Rasterization
– Project scene to image
samples
• Ray Tracing
– Project image samples to
scene
For each image pixel p:
make a ray r
For each scene primitive o:
if intersect(r,o) then
find color for o
color p with it
find where o falls on screen
rasterize 2D shape
for each produced pixel p:
find color for o
color p with it Lighting
Projection

Basic Ray Tracing vs. Rasterization
• Rasterization
– Project scene to image
samples
• Ray Tracing
– Project image samples to
scene
For each image pixel p:
make a ray r
if intersect(r,o) then
find color for o
color p with it
find where o falls on screen
rasterize 2D shape
for each produced pixel p:
find color for o
color p with it

Scalability
• Traditional HPC, parallel rendering definitions
– Strong scaling (more nodes are faster for same data)
– Weak scaling (more nodes allow larger data)
• Our interest/definition: output sensitivity
– Running time/storage proportional to size of output
instead of input
• Computational effort scales with visible data and screen
resolution
• Working set independent of original data size
29

A real-time data filtering problem!
• Models of unbounded complexity on limited
computers
– Need for output-sensitive techniques (O(N), not O(K))
• We assume less data on screen (N) than in model (K )
– Need for memory-efficient techniques (maximize cache hits!)
– Need for parallel techniques (maximize CPU/GPU core usage)
I/O
Storage Screen
10-100 Hz
O(N=1M-100M) pixels
Limited bandwidth
View parameters

A real-time data filtering problem!
• Models of unbounded complexity on limited
computers
– Need for output-sensitive techniques (O(N), not O(K))
• We assume less data on screen (N) than in model (K )
– Need for memory-efficient techniques (maximize cache hits!)
– Need for parallel techniques (maximize CPU/GPU core usage)
I/O
Storage Screen
10-100 Hz
O(N=1M-100M) pixels
Limited bandwidth
View parameters
Small
Working Set

Output-sensitive techniques
• At preprocessing time:
build MR structure
– Data prefiltering!
– Visibility + simplification
– Compression
• At run-time: selective
view-dependent
refinement from out-
of-core data
– Must be output sensitive
– Access to prefiltered data
under real-time constraints
– Visibility + LOD
COARSE
FINE

Output-sensitive techniques
• At preprocessing time:
build MR structure
– Data prefiltering!
– Visibility + simplification
– Compression
• At run-time: selective
view-dependent
refinement from out-
of-core/remote data
– Must be output sensitive
– Access to prefiltered data
under real-time constraints
– Decoding, Visibility + LOD
Occluded / Out-of-view
Inaccurate
Accurate
FRONT

Our contributions: GPU-friendly
output-sensitive techniques
• Chunk-based multiresolution structures
– Amortize selection costs over groups of primitives
– Combine space partitioning + level of detail
– Same structure used for visibility and detail culling
• Seamless combination of chunks
– Dependencies ensure consistency at the level of chunks
• Complex rendering primitives
– GPU programming features
– Curvilinear patches, view-dependent voxels, …
• Chunk-based external memory management
– Streaming, compression/decompression, block transfers,
caching

35MPixel displays 72 projectors
35
1GTri model on Light Field Displays…

… and on Mobile Terminals
iPhone4 / iPad
36

… and we can do volumes, too
Direct Volume Rendering of 64GVoxel
37

REAL-TIME ADAPTIVE
MESHES
38

Real-time adaptive meshes
• The problem:
efficiently create view-
dependent meshes
• Constraints:
– must approximate original
surface with controlled
screen-space error
– must preserve continuity
(conforming meshes)
– must handle meshes of
varying topology
– must be efficiently
rendered

Chunked multiresolution structures
• Mix and match chunks
– Amortize CPU work!
• Two approaches
– Adaptive coarse subdivision
• Multiresolution by combining a
variable number of fixed-size
patches
– Chunked-MT
TetraPuzzles
*-BDAM
– Fixed coarse subdivision
• Fixed number of patches,
multiresolution inside patches
– Adaptive QuadPatches

Chunked Multi Triangulations
The Multi Triangulation Framework
• Theoretical basis
– MT multiresolution framework
(Puppo 1996)
• Our contribution
– GPU friendly implementation
based on surface chunks with
boundary constraints
– Optimized implicit
specializations
(TetraPuzzles/V-Partitions)
– Parallel out-of-core pre-
processing and out-of-core
run-time
Partitioning
and
simplification
Adaptive
rendering GPU
Cache
Multiresolution
structure
(data+dependency)
Off-line On-line
Network /
Bus
References: EG 2003, 2006; IEEE
Viz 2003, 2005; SIGGRAPH 2004;
SPBG/C&G 2004; VAST
2009,2012; PG 2010, …

• Consider a sequence of
local modifications over
a given description D
– Each modification replaces a
portion of the domain with a
different conforming portion
(simplified)
• f1 floor / g1 the new
fragment
D’=D f g
Di+1=Di gi+1

• Dependencies
between
modifications can be
arranged in a DAG
– Adding a sink to the DAG
we can associate each
fragment to an arc
leaving a node

MT Cuts
• A cut of the DAG
defines a new
representation
– Collect all the fragment
floors of cut arcs and you
get a new conforming
mesh
D*=D0  g1  g4 = f0  f02  f03  f13  f1  f4

GPU Friendly MT
• Chunked MT assume
fragments are
triangle patches with
proper boundary
constraints
– DAG << original mesh
(patches composed by
thousands of tri)
– Structure memory +
traversal overhead
amortized over
thousands of triangles
– Per-patch optimizations

GPU Friendly MT
• Chunked MT assume
regions provide good
hierarchical space-
partitioning
– Compact
• Close-to-spherical
– Used for computing fast
projected error upper
bounds
– Used for visibility queries

GPU Friendly MT
• Construction
– Start with hires triangle
soup
– Partition model
– Construct non-leaf cells by
bottom-up recombination
and simplification of lower
level cells
– Assign model space errors
to cells
• Rendering
– Refine graph, render
selected precomputed cells
– Project errors to screen
– Dual queue
Adaptive
rendering GPU
Cache
On-line

Construction methods and specialized solutions
• Not all MT-graphs are good!
– Need good aspect ratios, no cascading dependencies
• Many subdivision structures and construction
methods proposed
– 3D surfaces
• TetraPuzzles: Partitioning based on conformal hierarchy of
tetrahedra
• V-Partition: General solution based on Voronoi space partitions
• Q-VDR, …
– Terrains
• *-BDAM: 2.5D specialization based on 4-8 tiling, supports
heavy compression
• …

Adaptive TetraPuzzles
• Construction
soup
– Partition model using a
conformal hierarchy of
tetrahedra
level cells
• Rendering
– Refine conformal hierarchy,
render selected
precomputed cells

Adaptive TetraPuzzles
Overview
• Construction
soup
– Partition model using a
conformal hierarchy of
tetrahedra
level cells
• Rendering
– Refine conformal
hierarchy, render selected
precomputed cells
View dependent mesh
refinement

TetraPuzzles rendering of Digital Michelangelo Models.
St Matthew 370 M-Triangles
NVIDIA 6800GTS (2004)

Adaptive Quad Patches
Simplified Streaming and Rendering for the Web
• Constrained
environments
– Lightweight, interpreted,
scripted
– Generic 3D models may still be
too heavy
• Need to implement mesh
codecs and dual queue
adapters
• Many models are heavy
but topologically simple
– Scanning / Modeling constraints
• Reuse components!
Javascript!

Adaptive Quad Patches
Simplified Streaming and Rendering for the Web
• Solution: represent
models as collections of
multiresolution quad
patches
– Image representation allows
component reuse!
– Natural multiresolution model
inside each patch
– Adaptive rendering handled
totally within shaders!
• CAVEAT: Does not work
for “generic” models
Javascript!
Best paper, WEB3D2012
SEE SEMINAR ON MOBILE
GRAPHICS FOR DETAILS

SAMPLE-BASED SOLUTIONS
55

Advantages of mesh-based
multiresolution models
• First GPU bound methods
for very large meshes
– Adaptive conforming
meshes
• Reduced overdraw
– Extensive optimization
• Stripification, cache coherence,
compression, …
– State of the art
performance
• GPU bound, >4Mtri/frame at
>60 fps on modern GPUs
• Extremely high quality for
large dense models with
“well behaved” surface

Limitations of mesh-based
• Hard to apply to models
with high detail and
complex topology and high
depth complexity!
– Error measured on
boundary surfaces
– LOD construction based on
local surface
coarsening/simplification
operations
– LOD construction unaware
of visibility (view-
independent
approximations)

• Sampled representations
• First coarse-grained multiresolution
point hierarchy (LPC)
• Far voxels for Multi-scale modeling
of appearance rather than
geometry, tight integration of
visibility and LOD construction
• Exploits GPU programmability for
accelerated rendering
• Many test cases, ranging
from laser scans, to
isosurfaces, to extremely
large CAD models
Sampled representations
C&G 2004, SIGGRAPH 2005,
VAST 2009, PG 2010, VC 2012,
VAST 2012, …

Far Voxels
The Far Voxel Concept
• Assumption: opaque surfaces,
non participating medium
• Goal is to represent the
appearance of complex far
geometry
– Near geometry can be
represented at full resolution
• Idea is to discretize a model
into many small volumes
located in the neighborood of
surfaces
– Approximates how a small
subvolume of the model reflects
the incoming light
=> View-dependent cubical voxel

Far Voxels
• Assumption: opaque surfaces,
non participating medium
• Goal is to represent the
appearance of complex far
geometry
– Near geometry can be
represented at full resolution
• Idea is to discretize a model
into many small volumes
located in the neighborhood of
surfaces
– Approximates how a small
subvolume of the model reflects
the incoming light
=> View-dependent voxel

Far Voxels
• A far voxel returns color
attenuation given
– View direction
– Light direction
• Rendered using a
customized vertex shader
executed on the GPU
Shader = f (view direction, light direction)

Far Voxels
Construction overview

Far Voxels
Construction overview: Inner nodes
• Sample a model subvolume
to build a grid of far voxels
• Voxels are far
– Project to worst case max
– Viewed not closer than dmin
D min
Section of the 3D grid of far voxels
 max

Far Voxels
Construction overview: Inner nodes
• Sample a model subvolume
to build a grid of far voxels
• Voxels are far
– Project to worst case max
– Viewed not closer than dmin
• Raycasting samples
original model and
identifies visible voxels
D min
Section of the 3D grid of far voxels
 max

Far Voxels
Construction overview: Far Voxel
• Consider voxel subvolume
• Samples gathered from
unoccluded directions
– Sample:
• (BRDF, n) = f(view direction)

Far Voxels
Construction overview: Far Voxel
• Consider voxel subvolume
• Samples gathered from
unoccluded directions
– Sample:
• (BRDF, n) = f(view direction)
• Compress shading
information by fitting
samples to a compact
analytical representation

Far Voxels
Construction overview: Far Voxel Shaders
• Build all the K different far
voxels representations
– K = flat, smooth..
– Principal component analysis
• Evaluate each representation
error
– Compare real values (samples)
with the voxel approximations
from the sample direction
• Choose approximation with
lowest error
…
Flat proxy:
2 components
Smooth proxy:
6 components
Others…
Err(k) =

Far Voxels
Rendering
• Hierarchical traversal with coherent culling
– Stop when out-of view, occluded (GPU
feedback), or accurate enough
• Leaf node: Triangle rendering
– Draw the precomputed triangle strip
• Inner node: Voxel rendering
– For each far voxel type
• Enable its shader
• Draw all its view dependent primitives using
glDrawArrays
– Splat voxels as antialiased point primitives
– Limits
• Does not consider primitive opacity
• Rendering quality similar to one-pass point splat
methods (no sorting/blending)
Triangles
Far Voxels

Far Voxels
Results
• Tested on extremely complex heterogeneous surface
models
– St.Matthew, Boeing 777, Richtmyer Meshkov isosurf., all at
once
• Tested in a number of situations
– Single processor / cluster construction
– Workstation viewing, large scale display
373M triangles
14.5 GB
350M triangles
13.7 GB
472M triangles
18.4 GB
1.2G triangles
46.6 GB

Far Voxels
Results
• Xeon 2.4GHz, 70GB SCSI 320 Disk, GeForce FX6800GT AGP
8x (hardware dated back to 2005)
• Window size: from video resolution to stereo projector
display
– St.Matthew, Boeing, Isosurface: 640 x 480
– All at once: 640 x 480 and Stereo 2 x 1024 x 768
• Pixel tolerance: [Target 1 | Actual ~0.9 | Max ~10]
• Resident set size limited to ~200 MB
45 Fps
51 MPrim/s
44 Fps
42 MPrim/s
34 Fps
41 MPrim/s
2 x 1024 x 768
20 Fps
40 MPrim/s
640 x 480
20 Fps
42 MPrim/s

Far Voxels rendering of complex models. NVIDIA 6800 GTS (2005)

Far Voxels
Conclusions
• General purpose
technique that targets
many model kinds
– Seamless integration of
• multiresolution
• occlusion culling
• out-of-core data management
– High performance
– Scalability
• Main limitations
– Slow preprocessing
– Non-photorealistic rendering
quality
Intel Xeon 2.4GHz 1GB, GeForce 6800GT AGP8X

3D VOLUMETRIC MODELS
AND COMPRESSION
78

Volumetric models
• Voxelized representation
– Regular grid structure
– Simple scalar grid or sampled
representation/voxel
• Data on surfaces and/or
interior of objects
– Advanced semitransp. shading
• Increasingly common
– SciViz / Medical imaging
– Off-line rendering for movies
– Gaming: voxel engines!
• Need for compression and
efficient rendering

• Visualization of massive
scalar volumes without size
limitations
– A single-pass raycasting
technique working out-of-
core on GPU parallel
architectures
• Compress data to facilitate
data streaming and 4D
visualizations
– Novel compression
architecture and novel
compression methods
Volumetric models
References: EG 2008, Visual Computer
2008, Visual Computer 2010, VG 2010,
TVCG 2011, EUROVIS 2012…
80

• Visualization of massive
scalar volumes without size
limitations
– A single-pass raycasting
technique working out-of-
core on GPU parallel
architectures
• Compress data to facilitate
data streaming and 4D
visualizations
– Novel compression
architecture and novel
compression methods
Volumetric models
81
Crassin - Gigavoxels
References: EG 2008, Visual Computer
2008, Visual Computer 2010, VG 2010,
TVCG 2011, EUROVIS 2012…
(STAR EG 2013)

Order dependentOrder independent
Accumulation
Empty space skipping
Early ray termination
Pixel
82
Massive Volumes Visualization
Volume rendering problem

Volume rendering problem
• Current interactive
solutions are based
on GPU architectures
– Massive parallelism
– Huge memory bandwidth
• E.g. GeForce GTX 780
Ti
– has a 336 GB/s of
bandwidth
– Has 5 GFLOPs
[ hardwareinsight.com ]
83

• We introduced a novel method based on the
following basic ideas:
– Multi-resolution out-of-core representation based on a
octree of volume bricks
– Adaptive CPU loading of the data from local/remote
repository cooperates with separate render thread fully
executed in the GPU
– Stackless traversal of an adaptive working set
– Exploitation of the visibility feedback
Multiresolution out-of-core DVR
84
Real-time deblocked GPU rendering of compressed volumes VMV , 2014.
View-dependent exploration of massive volumetric models on large-scale
light field displays. The Visual Computer, 26, 2010.
A single-pass GPU ray casting framework for interactive out-of-core rendering
of massive volumetric datasets. The Visual Computer, 24, 2008.

• An adaptive cut of a multi-
resolution octree structure is
traversed on the GPU, leading
to a method which …
–  is scalable and fully adaptive
–  increases performance and reduces
overhead
–  produces simple and flexible code
(single-pass)
85

• Use CPU for …
– Creation & loading
– Octree refinement
– Encode current cut using
an spatial index
• Use GPU for …
– Stackless octree traversal
• Using neighbour pointers
– Rendering
• Flexible ray traversal /
compositing strategies
• Improved visibility
feedback
86
Architecture overview
Neighbour pointer navigation

volume
render
adaptive loader
storage
preprocessing
octree node
database
visibility
feedback
has current working set
enough accuracy?
yes
octree refinement
prepare to render
no
GPUCPU
[ creation and maintainance ] [ rendering ]
offline
Method overview
87

• The adaptive loader maintains in-core a view-and-
transfer function dependent cut of the out-of-core
octree structure
– Uses it to update the GPU cache and Spatial Index.
– Uses CUDA scatter write capability on a 8bit CUDA-array.
• Basic principles:
– Update at each frame the visibility status of the nodes in the
graph based on rendering feedback
– Refine nodes marked as visible during the previous frame and
considered inaccurate and non-empty according to the current
transfer function
– Pull-up visibillity data to inner nodes by recursively
recombination
• The cost amortized over full brick traversal is
negligible (<1ms on Nvidia GPU 8800GTS 640MB)
Visibility feedback
88

• Impact of the
visibility culling
– Visibility culling reduces
the working set from 1731
to 1035 bricks in a almost
opaque case, and from
1984 bricks to 1789 bricks
when surfaces get more
transparent. The window
size used for rendering
was of 1024x576 pixels
Visibility feedback
89

Results
90
Interactive exploration of a 16bit 2GB CT volume on a
consumer NVidia 8800 GTS graphics board with 640MB (2008)

Introducing Compression
• Long data transfer times and GPU memory size
limitations motivate LOD and compression
– LOD (Flat-MR blocking, single-pass MOVR, gigavoxels)
• Compression is fully exploited if data is
maintained in compressed form through the
entire pipe-line
– Compression-domain volume renderers + deferred filtering
• Highly asymmetric encoding/decoding schemes
– We can afford slow offline compression and precomputation
– Fast real-time data decoding, interpolation and shading
– Spatially independent random-access to data
• SEE COMPRESSION SEMINAR THIS AFTERNOON
91

INTERACTION AND 3D
DISPLAY
92

Massive volumetric model on light-field display
72 projector (35MPixel) Holografika light field display driven by 36 NVIDIA 8800GTS
graphics boards (2010)
93

Rendering on Light-field Displays
Light-field display overview
• The key feature characterizing 3D
displays is direction-selective light
emission
• 3D display based on commercially
available hardware developed by
Holografika (software by CRS4!)
– Specially arranged projector array
and a holographic screen
– Side mirrors increase the available
light beams count
– Each projector emits light beams
toward a subset of the points of the
holographic screen
Projector
Screen
Light field
94

Light-field display physical behavior
• Selective horizontal
light transmission,
wide vertical
scattering
– Homogeneous light
distribution
• Continuous 3D view
simulated by
controlling color of
each ray (= projector
pixel)
– Parameters found in
calibration step
95
References: TVC 2010, under
review

Projection technique
• Screen pixels have the
same color when viewed
from all vertical viewing
angles
• We introduce a “virtual
observer”, fixing the
viewer´s height and
distance from screen
– The resulting MCOP technique is
exact for all viewers at the
same distance from the screen
and height as the virtual
observer
– It proves in practice to be a
good approximation for all
viewing positions in the display
workspace
96
References: C&G 2006, EG 2008,
TVC 2010

• The view-dependent characteristics of the display can be
exploited to develop specialized interactive illustrative
techniques designed to improve spatial understanding
• Simple head motions can reveal new aspects of the
inspected data
Novel view-dependent illustrative tools
97

Detph dependent resolution
• The size of the
smallest voxel that
can be reproduced
depends on the
distance from the
screen and from the
beam angular size
• We use this estimate
to determine the
resolution for
sampling the volume
98
References: C&G 2006, EG 2008,
TVC 2010

Natural interaction: requirements
• Light field display
constraints
– Depth-dependent
spatial resolution,
calibration errors,
angular bounds
• Interaction
metaphor should
be simple
– Reduced number of
DOFs
– Short learning time

FOX interface: components
• Translation and
rotation
• Automatic
zooming
• Automatic
hotspot
placement References: VRCAI 2011, C&G
2012

Parallel rendering
• We employ a GPU cluster for rendering
• Sort first parallel rendering approach
– Adaptive out-of-core GPU rendering vs Replicating data
– Static assignment: rendering process images.
• Good load balancing. Caused by the geometry of the display,
with all projectors looking at the same portion of the volume.
• The most loaded processes handle (max.) about 30% more
bricks.
• Our framework can be extended with other image partitioning
based techniques.
101

System overview
102

Volume rendering on a Holografika 72 projector (35MPixel) light field display
driven by 36 NVIDIA 8800GTS graphics boards (2009)

Interactive surface exploration on a 72 projector (35MPixel) Holografika light
field display driven by 36 NVIDIA 460GTX graphics boards. FOX interaction
technique (2011)

TIME FOR A CONCLUSION,
RIGHT?
For more information: www.crs4.it/vic/

Massive research…
• How to efficiently acquire/create massive models?
– Computational photography, 3D scanning pipe-lines, 3DTV
• How to efficiently process them?
– Stream-processing, multiresolution, external memory algorithms,
parallel programming, GPGPU
– Specific processing methods
• How to efficiently store and distribute them?
– Multiresolution, adaptive streaming, compression
• How to efficiently render/interact with them?
– Multiresolution, adaptive rendering/collisions, out-of-core methods,
GPU programming, parallelization, rasterization, ray-casting
• How to efficiently explore them?
– Novel 3D displays, specific interaction techniques
– Portable devices
106

Massive research…
• Lots of questions
– Hot topic, “big data” is a buzzword…
– “massive” for preprocessing/data analysis
– “massive” for run-time (rendering)
• Lots of answers…
– Many papers… no general solution
• …but still much work to do...
107

Some food for PhDs..
• Construction pipe-lines
– Sensor fusion, consolidation, multiresolution meshing,
editable representations, …
• Streamlined adaptive mesh models
– Flat multiresolution structures, GI/AQP not the solution…
• Model-specific (approximated) compressed
– E.g., improve blockmaps for urban models…
• Assisted navigation interfaces
– E.g., planning, precomputed paths,
• … and more!
108

www.crs4.it/vic/
Thanks for your attention!
Questions?
Enrico Gobbetti
Fabio Marton
CRS4/ViC
gobbetti@crs4.it
marton@crs4.it
www.crs4.it/vic/
109

2015 crs4-seminar-massive-models-full

Recommended

Recommended

More Related Content

What's hot

What's hot (11)

Viewers also liked

Viewers also liked (7)

Similar to 2015 crs4-seminar-massive-models-full

Similar to 2015 crs4-seminar-massive-models-full (20)

More from CRS4 Research Center in Sardinia

More from CRS4 Research Center in Sardinia (20)

Recently uploaded

Recently uploaded (20)

2015 crs4-seminar-massive-models-full