Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Semantic-based Segmentation and Annotation of 3D Models Laura Papaleo , Leila De Floriani Department of Information and Co...
Outline <ul><li>Introduction </li></ul><ul><li>A semantic web framework for shape annotation </li></ul><ul><li>The two-lev...
Introduction <ul><li>Motivations: </li></ul><ul><ul><li>3D objects widely available on the net  and used in several discip...
Introduction ( cont’d ) <ul><li>Our purpose :  analyze and semantically annotate digital shapes  </li></ul><ul><li>Basic s...
3D Digital shape <ul><li>Digital shape: digital representation of real or  virtual objects. </li></ul><ul><li>Manifold sha...
Our proposal <ul><ul><li>[ BEyond Shape Modeling for  understAnding Real world represenTations ] </li></ul></ul><ul><li>Be...
Advantages <ul><li>Fast and semantic-driven population of digital libraries for efficient searching and retrieval </li></u...
be-SMART: the modules September, 9 2009 ICIAP09 - Laura Papaleo Modeling World Semantic Web world <ul><li>Java-based </li>...
Be-SMART:   graph-based reasoning <ul><li>Two-level segmentation graph for reasoning and annotation </li></ul>September, 9...
Segmentation phases <ul><li>Segmentation of non-manifold objects     context independent </li></ul><ul><li>Segmentation o...
Segmentation Techniques <ul><li>Automatic : several well-behaved techniques in the literature </li></ul><ul><ul><li>Fast  ...
Our Approach (automatic phase) <ul><li>Segmentation into nearly planar regions </li></ul><ul><ul><li>[ Cohen-Steiner et al...
Our Approach (manual phase) <ul><li>Merging : manual merging of regions by selection </li></ul><ul><li>Cutting : Intellige...
Manifold segmentation graph <ul><li>All steps are performed  on the manifold  segmentation graph </li></ul><ul><li>In the ...
Hierarchical semantic tagging <ul><li>Goal : add information to a region  </li></ul><ul><li>Every segmented region (node) ...
Properties <ul><li>non-triangular meshes can be segmented </li></ul><ul><li>meshes do not need to be connected </li></ul><...
Interface September, 9 2009 ICIAP09 - Laura Papaleo Xj3D browser Manifold segmentation graph <ul><li>Patch information: </...
Inspecting the segmentation graph <ul><li>Implemented as an hyperbolic graph </li></ul><ul><li>Intuitive browsing and glob...
Summary <ul><li>We need semantics: </li></ul><ul><ul><li>be-SMART, a framework  for  inspecting, structuring and  annotati...
Future work <ul><li>Geometric modeling </li></ul><ul><ul><li>Automatic computation of handles and  through holes </li></ul...
Acknowledgements <ul><li>This work has been partially supported by </li></ul><ul><ul><li>the  MIUR-FIRB  project SHALOM un...
Upcoming SlideShare
Loading in …5
×

Semantic-based Segmentation and Annotation of 3D Models

2,030 views

Published on

Presentation of be-smart a semantic web framework for digital shape understanding at ICIAP 2009

Published in: Technology, Lifestyle, Business
  • Be the first to comment

Semantic-based Segmentation and Annotation of 3D Models

  1. 1. Semantic-based Segmentation and Annotation of 3D Models Laura Papaleo , Leila De Floriani Department of Information and Computer Science University of Genova
  2. 2. Outline <ul><li>Introduction </li></ul><ul><li>A semantic web framework for shape annotation </li></ul><ul><li>The two-level segmentation graph </li></ul><ul><li>Segmentation of manifold shapes </li></ul><ul><ul><li>Automatic </li></ul></ul><ul><ul><li>Semi-manual </li></ul></ul><ul><li>Semantic-driven hierarchical tagging </li></ul><ul><li>Results and future activities </li></ul><ul><li>Conclusions </li></ul>September, 9 2009 ICIAP09 - Laura Papaleo
  3. 3. Introduction <ul><li>Motivations: </li></ul><ul><ul><li>3D objects widely available on the net and used in several disciplines </li></ul></ul><ul><ul><li>Organization of multimedia content into digital libraries </li></ul></ul><ul><ul><li>Objective: archiving 3D objects in an intelligent way (e.g. [1,2,3]) </li></ul></ul><ul><li>KEY: extract and maintain embedded knowledge </li></ul>September, 9 2009 ICIAP09 - Laura Papaleo [1] RAZDAN A., et al. 2002 [2] AIM@SHAPE, EC NoE, 2004-2007 [3] MINDSWAP, Univ. Maryland 2001
  4. 4. Introduction ( cont’d ) <ul><li>Our purpose : analyze and semantically annotate digital shapes </li></ul><ul><li>Basic step for semantic annotation: </li></ul><ul><ul><li>decomposition of a shape into “meaningful” portions </li></ul></ul><ul><ul><li>meaning is context-dependent </li></ul></ul>September, 9 2009 ICIAP09 - Laura Papaleo object perception recognition object segmentation classification
  5. 5. 3D Digital shape <ul><li>Digital shape: digital representation of real or virtual objects. </li></ul><ul><li>Manifold shape: </li></ul><ul><ul><li>object in which every point has a neighborhood homeomorphic to either an open ball (internal point), or to an open half-ball (boundary point) </li></ul></ul><ul><li>Non-manifold shape characterized by </li></ul><ul><ul><li>non-manifold joints (vertices and edges) </li></ul></ul><ul><ul><li>parts of different dimensions </li></ul></ul>September, 9 2009 ICIAP09 - Laura Papaleo Non-manifold edge Non-manifold vertex Non-manifold shape
  6. 6. Our proposal <ul><ul><li>[ BEyond Shape Modeling for understAnding Real world represenTations ] </li></ul></ul><ul><li>Be_SMART: Semantic Web framework for inspecting, structuring and annotating digital shapes and scenes </li></ul><ul><li>Within be-SMART segmentation of shapes (or parts of)  this presentation </li></ul>September, 9 2009 ICIAP09 - Laura Papaleo Extract Organize Reuse-Share Formalize
  7. 7. Advantages <ul><li>Fast and semantic-driven population of digital libraries for efficient searching and retrieval </li></ul>September, 9 2009 ICIAP09 - Laura Papaleo Info on the persons, the events, hyper-dimensional connections about concepts 2D Info on models, their geometry, the procedure of creation… hyper-dimensional connections about concepts 3D
  8. 8. be-SMART: the modules September, 9 2009 ICIAP09 - Laura Papaleo Modeling World Semantic Web world <ul><li>Java-based </li></ul><ul><li>Platform independent </li></ul><ul><li>Modular and extensible </li></ul><ul><li>Extension of PhotoStuff [1] </li></ul>[1] Photostuff, MindSwap, Univ of Maryland, 2006 Annotator (Ontology & Metadata Mngmt) context-dependent Geometry and Topology Analyzer (GTA) Topological Decomposer (TD) Automatic Segmentation (AS) Manual Segmentation (MS) context-independent
  9. 9. Be-SMART: graph-based reasoning <ul><li>Two-level segmentation graph for reasoning and annotation </li></ul>September, 9 2009 ICIAP09 - Laura Papaleo Semantic-oriented decomposition into manifold parts(*) Manifold Decomposition into meaningful parts Non-manifold segmentation graph Manifold segmentation graph Ontology-driven annotation Digital shape (*) De Floriani, Papaleo, Huang, Hendler. A semantic web environment for digital shapes understanding. LNCS, SAMT 2007. Ontology-driven processes
  10. 10. Segmentation phases <ul><li>Segmentation of non-manifold objects  context independent </li></ul><ul><li>Segmentation of manifold parts  context dependent </li></ul>September, 9 2009 ICIAP09 - Laura Papaleo shell shell wire-web petal petal petal petal pistil
  11. 11. Segmentation Techniques <ul><li>Automatic : several well-behaved techniques in the literature </li></ul><ul><ul><li>Fast  </li></ul></ul><ul><ul><li>Based on specific characteristics (curvature, texture..) </li></ul></ul><ul><ul><li>Can produce incorrect results  </li></ul></ul><ul><li>Manual : less approaches in the literature </li></ul><ul><ul><li> Freedom for the user </li></ul></ul><ul><ul><li> More work for the user </li></ul></ul><ul><ul><li> Difficult in case of complex shapes </li></ul></ul>September, 9 2009 ICIAP09 - Laura Papaleo
  12. 12. Our Approach (automatic phase) <ul><li>Segmentation into nearly planar regions </li></ul><ul><ul><li>[ Cohen-Steiner et al. Variational shape approximation , ACM SIGGRAPH 2004 ] </li></ul></ul><ul><li>Clustering of the regions according to geometric criteria </li></ul><ul><ul><li>[ Alla Sheffer: Model simplification for meshing using face clustering . Computer-Aided Design, 2001 ] </li></ul></ul>September, 9 2009 ICIAP09 - Laura Papaleo Segmentation into planar regions Region clustering
  13. 13. Our Approach (manual phase) <ul><li>Merging : manual merging of regions by selection </li></ul><ul><li>Cutting : Intelligent scissoring </li></ul><ul><ul><li>[T. Funkhouser, et al.. Modeling by example . SIGGRAPH, 2004] </li></ul></ul>September, 9 2009 ICIAP09 - Laura Papaleo Set the viewpoint Draw the cut Compute the cut Select two adjacent regions Merge & update
  14. 14. Manifold segmentation graph <ul><li>All steps are performed on the manifold segmentation graph </li></ul><ul><li>In the graph: </li></ul><ul><ul><li>Nodes are regions </li></ul></ul><ul><ul><li>Arcs maintain boundaries between regions (chain of edges) </li></ul></ul>September, 9 2009 ICIAP09 - Laura Papaleo
  15. 15. Hierarchical semantic tagging <ul><li>Goal : add information to a region </li></ul><ul><li>Every segmented region (node) C contains in the name also the names of its ancestors : ancestor1: … :ancestorn: RegionName </li></ul><ul><li>We can trace the entire segmentation process </li></ul>September, 9 2009 ICIAP09 - Laura Papaleo horse::head::leftEar
  16. 16. Properties <ul><li>non-triangular meshes can be segmented </li></ul><ul><li>meshes do not need to be connected </li></ul><ul><li>use X3D coding, and define a new <shape> for each segmented portion </li></ul><ul><li>save separately selected regions and the associated annotation </li></ul>September, 9 2009 ICIAP09 - Laura Papaleo
  17. 17. Interface September, 9 2009 ICIAP09 - Laura Papaleo Xj3D browser Manifold segmentation graph <ul><li>Patch information: </li></ul><ul><li>Semantic </li></ul><ul><li>Geometry </li></ul><ul><li>Adjacency </li></ul>
  18. 18. Inspecting the segmentation graph <ul><li>Implemented as an hyperbolic graph </li></ul><ul><li>Intuitive browsing and global visualization </li></ul><ul><li>Functionalities </li></ul><ul><ul><li>Click on a node C, focus on C, highlight of the region </li></ul></ul><ul><ul><li>All nodes always visible </li></ul></ul>September, 9 2009 ICIAP09 - Laura Papaleo
  19. 19. Summary <ul><li>We need semantics: </li></ul><ul><ul><li>be-SMART, a framework for inspecting, structuring and annotating 3D models and scenes according to ontology-driven metadata </li></ul></ul><ul><li>Segmentation is the basis for semantic annotation </li></ul><ul><ul><li>two segmentation phases in be-SMART </li></ul></ul><ul><ul><li>a simple tool for browsing the segmentation graph </li></ul></ul><ul><ul><li>A semantic driven-hierarchical tagging </li></ul></ul>September, 9 2009 ICIAP09 - Laura Papaleo
  20. 20. Future work <ul><li>Geometric modeling </li></ul><ul><ul><li>Automatic computation of handles and through holes </li></ul></ul><ul><ul><li>Extend to high-dimensional data </li></ul></ul><ul><ul><li>Add new semi-automatic segmentation methods </li></ul></ul><ul><li>Semantic Web - Reasoners - Web Services </li></ul><ul><ul><li>Improve the power of the reasoning </li></ul></ul><ul><ul><li>Towards the definition of semantic web services for reasoning on multimedia content </li></ul></ul><ul><ul><li>Ontology-driven processes performed by smart semantic agents </li></ul></ul><ul><li>Specialization of the system in application domains (biomedical, animation,…) </li></ul>September, 9 2009 ICIAP09 - Laura Papaleo
  21. 21. Acknowledgements <ul><li>This work has been partially supported by </li></ul><ul><ul><li>the  MIUR-FIRB  project SHALOM under contract number RBIN04HWR8   </li></ul></ul><ul><ul><li>the Science and Technological Park of Liguria POS.N.5−Avv.1/2006 </li></ul></ul><ul><ul><li>DISI-RPI Research Project (J. Hendler, Cognitive Science Department) </li></ul></ul><ul><li>Thank you. </li></ul><ul><li>Laura Papaleo [email_address] </li></ul>September, 9 2009 ICIAP09 - Laura Papaleo

×