Computer Graphics Recipes for
Endowing 3D Shapes with
Semantics in Virtual Worlds
Michela Mortara, Chiara E. Catalano
CNR IMATI Genova, Italy

Outline
3D in Gaming: need of semantics
How to make semantics explicit:
Segmentation
Annotation
Applications to virtual worlds
Conclusions
3AMIGAS Workshop, Amstedam, June 16th 2009

3D in Gaming
Complex and convincing virtual worlds
realistic environment and characters
realistic interactions and behaviour
3AMIGAS Workshop, Amstedam, June 16th 2009

What’s semantics?
Knowledge related to the geometry of the 3D
shape
Knowledge related to the application domain in
which the object is manipulated
Knowledge related to the meaning of the object
represented by the 3D model
3AMIGAS Workshop, Amstedam, June 16th 2009

Adding semantics to geometry
We have to extract and associate knowledge to
digital content
Shape analysis: extracts implicit info coded into
the geometry
Numerical values (e.g. volume, area, spatial
distribution, curvature)
Shape descriptors (structuring the shape)
Segmentation
3AMIGAS Workshop, Amstedam, June 16th 2009
geometry structure semantics

Adding semantics to geometry
We have to extract and associate knowledge to
digital content and its subparts
Shape Annotation: it documents the shape
with contextual knowledge either manually or
automatically
geometry structure semantics
3AMIGAS Workshop, Amstedam, June 16th 2009

Semantic-driven segmentations
They captures semantically relevant features
automatically
what is relevant in the gaming context?
IMATI segmentation methods for triangle meshes:
Fitting primitives
Reeb Graphs
Tailor
Plumber
They extract specific kind of knowledge suitable
also for virtual worlds
3AMIGAS Workshop, Amstedam, June 16th 2009

Fitting Primitives
Hierarchical face clustering algorithm able to recognise
planes, cylinders and spheres
3AMIGAS Workshop, Amstedam, June 16th 2009

Reeb Graph
The topology of a shape is coded into the Reeb
graph respect to f, which captures the evolution of the
level sets of f on the shape
3AMIGAS Workshop, Amstedam, June 16th 2009

Plumber (and Tailor)
It defines a shape decomposition into
connected components that are either
tubular features or blob regions
Skeletons and grasping
regions
3AMIGAS Workshop, Amstedam, June 16th 2009
Landmarks

Automatic identification of human body parts
Plumber Tailor
Skeleton extraction
segmentation characterisation
Tagging human Candidate
body parts skeletal joints
3AMIGAS Workshop, Amstedam, June 16th 2009

ShapeAnnotator: general framework
Shape
A surface mesh
Abstracted
Shape
Shape A segmented mesh
CG Tools
Segmentation Plug-ins Annotator
Domain
An OWL ontology
Instance
Instance
Expert Knowledge Base
3AMIGAS Workshop, Amstedam, June 16th 2009

ShapeAnnotator: multi-segmentation
Pick the interesting features from different
shape segmentations
(b) Morse-based (c) Plumber (d) fitting primitives
3AMIGAS Workshop, Amstedam, June 16th 2009

The ShapeAnnotator: ontology browser
3AMIGAS Workshop, Amstedam, June 16th 2009

The ShapeAnnotator: knowledge base
3AMIGAS Workshop, Amstedam, June 16th 2009

Possible scenarios for virtual worlds
DSW semantic search for reuse:
Models as they are: to have large diversity of human
characters and objects
Modifying models: to create new VHs, reuse
garments, take only interesting parts
Assistive living
Simulation of domestic environments with realistic
people (e.g. with disabilities), places and danger
conditions
3AMIGAS Workshop, Amstedam, June 16th 2009

Conclusions
3D with semantics would have high impact
Easy indexing, searching and retrieval
Efficient reuse and re-adaptation
Realistic interactions among virtual characters and
objects
Still a lot of work to do
Manual annotation
No shared conceptualisation
…
3AMIGAS Workshop, Amstedam, June 16th 2009