• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Manual Segmentation and semantic-based hierarchical tagginf od 3d models
 

Manual Segmentation and semantic-based hierarchical tagginf od 3d models

on

  • 766 views

Today 3D objects have become widely available in different application domains, thus it is becoming fundamental to use, integrate and develop techniques for extracting and maintaining their implicit ...

Today 3D objects have become widely available in different application domains, thus it is becoming fundamental to use, integrate and develop techniques for extracting and maintaining their implicit knowledge. These techniques should be encapsulated in intelligent systems able to semantically annotate the 3D models, thus improving their usability and indexing, especially in innovative web cooperative environments. In our work, we are moving in this direction, by defining and developing data structures, methods and interfaces for structuring and semantically annotating 3D complex models (and scenes), even changing over time, according to ontology-driven metadata. In this paper, we focus on tools and methods for manually segmenting manifold 3D models and on the underline structural representation that we build and manipulate. We present also an interface from which the user can inspect and browse the segmentation, describing also the first prototype of an annotation tool which allows a hierarchical semantic-driven tagging of the segmented model.

Statistics

Views

Total Views
766
Views on SlideShare
766
Embed Views
0

Actions

Likes
1
Downloads
5
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • In this presentation I’m going to present the results in the development of a semantic web system for 3D shape understanding and annotation. The system can be located in a more wide framework of research that research group I belong to has established with the RPI Department of Cognitive Science leaded by Prof J. Hendler. this slide is just to remind that the activity we are working on in this paper is related to digital representations of 3D object and 3D objects are becoming widely available on the net and used in many disciplines. There is so a general need of organizing these representations in an intelligent way. One of the first project in this sense was a NSF project in 2002 which concentrated mainly on medical 3D objects and the way in which this objects can be analyzed and organized… to be searched well. In any case the key in this framework is to extract knowledge from digital representations and maintain that in order also to efficiently search and retrieve and eventually reason on them.
  • Semantic Web proposes annotating document content using semantic information. The result is multimedia content with machine interpretable mark-up that provide the source material with which agents and SemanticWeb services operate, thus creating annotations with well-defined semantics.
  • Qui diciamo che tutto il ragionamento relativo all’oggetto (o alla scena) viene fatto basandosi su una rappresentazione a grafo Il grafo si chiama segmentation graph ed è un two-level segmentation graph. Il primo livello (attivo nel caso di modelli non-manifold) etc Il secondo livello si chiama manifodl segmentation graph Le informazioni semantiche verranno associate ai nodi del grafo (o eventualmente, se necessario anche agli archi) e quindi alle porzioni di oggetto…
  • In our case, since X3D allows to define shapes using vertices and not edges, the use of an implementation of the original approach would have been computationally expensive. Every time the user would draw the stroke, the system should compute the intersection between a circle of radius r and all the edges present in the scene and projected on the viewplane.We decided to apply the algorithm on the vertices of the model. This change does not modify the general methodology, but it allows us to reduce the number of operations to be performed.With our choice, in fact, we do not have to compute intersections (solving linear systems) but only euclidean distances. Additionally, we have been able to extend the procedure to surface meshes which are not represented only by triangles, using in this way all the faces types defined by the X3D standard. Furthermore, we extended the method to special cases: we can treat the case in which, given a stroke (as a set of circles) there is no vertex inside it connecting the initial and final vertex of the stroke: each time we cannot find a connection inside the stroke, we search for the nearest point in the surface model and we let the path passing from it. This procedure solves also the case in which, given the stroke, the system cannot find an initial and/or final vertex. For automatic computation of the cut in the not visible part of the model, we improved the original method restricting the search of the connections to a subset of vertices. For doing this we compute the visibility of each vertex before performing the cut.
  • In our case, since X3D allows to define shapes using vertices and not edges, the use of an implementation of the original approach would have been computationally expensive. Every time the user would draw the stroke, the system should compute the intersection between a circle of radius r and all the edges present in the scene and projected on the viewplane.We decided to apply the algorithm on the vertices of the model. This change does not modify the general methodology, but it allows us to reduce the number of operations to be performed.With our choice, in fact, we do not have to compute intersections (solving linear systems) but only euclidean distances. Additionally, we have been able to extend the procedure to surface meshes which are not represented only by triangles, using in this way all the faces types defined by the X3D standard. Furthermore, we extended the method to special cases: we can treat the case in which, given a stroke (as a set of circles) there is no vertex inside it connecting the initial and final vertex of the stroke: each time we cannot find a connection inside the stroke, we search for the nearest point in the surface model and we let the path passing from it. This procedure solves also the case in which, given the stroke, the system cannot find an initial and/or final vertex. For automatic computation of the cut in the not visible part of the model, we improved the original method restricting the search of the connections to a subset of vertices. For doing this we compute the visibility of each vertex before performing the cut.
  • In our case, since X3D allows to define shapes using vertices and not edges, the use of an implementation of the original approach would have been computationally expensive. Every time the user would draw the stroke, the system should compute the intersection between a circle of radius r and all the edges present in the scene and projected on the viewplane.We decided to apply the algorithm on the vertices of the model. This change does not modify the general methodology, but it allows us to reduce the number of operations to be performed.With our choice, in fact, we do not have to compute intersections (solving linear systems) but only euclidean distances. Additionally, we have been able to extend the procedure to surface meshes which are not represented only by triangles, using in this way all the faces types defined by the X3D standard. Furthermore, we extended the method to special cases: we can treat the case in which, given a stroke (as a set of circles) there is no vertex inside it connecting the initial and final vertex of the stroke: each time we cannot find a connection inside the stroke, we search for the nearest point in the surface model and we let the path passing from it. This procedure solves also the case in which, given the stroke, the system cannot find an initial and/or final vertex. For automatic computation of the cut in the not visible part of the model, we improved the original method restricting the search of the connections to a subset of vertices. For doing this we compute the visibility of each vertex before performing the cut.
  • Some notes: using our tags - organized hierarchically - we are able to re-merge the segmented regions simply by checking the names of the regions and by merging their faces and vertices. looking at the name of a given region C we can access immediately to its history.
  • one tab collects the geometrical information, automatically extracted (geometry); another tab (adjacency) describes the adjacency information, again automatically extracted. The last working tab (semantic) is, instead, devoted to the userdefined semantic annotation.
  • La visualizzazione del grafo per adesso è per modelli manifold

Manual Segmentation and semantic-based hierarchical tagginf od 3d models Manual Segmentation and semantic-based hierarchical tagginf od 3d models Presentation Transcript

  • MANUAL SEGMENTATION AND SEMANTIC-BASED HIERARCHICAL TAGGING OF 3D MODELS Laura Papaleo*^, Leila De Floriani^ * ICT Department, Province of Genova ^Department of Information and Computer Science University of Genova
  • OUTLINE
    • Introduction
    • Be-Smart and the two-level segmentation graph
    • Related work ( segmentation )
    • Previous approach and its limitations
    • Contribution:
      • Cut-based segmentation method
      • Semantic-driven hierarchical tagging
    • Conclusions and future directions
    November, 2009 EGIT2010 - Laura Papaleo
  • INTRODUCTION
    • 3D objects widely available on the net and used in several disciplines
    • Organization of multimedia content into digital libraries
    • Objective: archiving 3D objects in an intelligent way (e.g. [1,2,3,4,5,6] and many others…)
    • KEY: extract and maintain knowledge
    November, 2009 EGIT2010 - Laura Papaleo
    • Examples
    • AIM@SHAPE, EC NoE, 2004-2007
    • MINDSWAP, Univ. Maryland 2001
    • Attene, Robbiano, Spagnuolo, Falcidieno, Semantic annotation[ …] , SAMT07.
    • De Floriani, Papaleo, Huang, Hendler, A semantic web environment […] , SAMT 2007
    • Hunter, Gerber Harvesting community annotations on 3d models[…] . Journal of Cultural Heritage (2010)
    • Moroni, Salvetti, Salvetti, Shape analysis semantic annotation […] . Pattern Recognition and Image Analysis (2010),
  • SEMANTIC WEB
    • SemWeb envisages technologies, which can make possible the generation of “intelligent” multimedia content.
    • Ingredients:
      • A content which “knows about” its own meaning
      • Automated processes can “know what to do” with such content
    November, 2009 EGIT2010 - Laura Papaleo
  • GOAL
    • Our purpose: analyze and semantically annotate digital shapes
    • Basic step for semantic annotation:
      • decomposition of a shape into “meaningful” portions
      • meaning is context-dependent
    •  Be-SMART: Semantic Web framework for inspecting, structuring and annotating digital shapes and scenes
    November, 2009 EGIT2010 - Laura Papaleo object perception recognition object segmentation classification
  • ADVANTAGES
    • Fast and semantic-driven population of digital libraries for efficient searching and retrieval
    November, 2009 EGIT2010 - Laura Papaleo Info on the persons, the events, hyper-dimensional connections about concepts 2D Info on models, their geometry, the procedure of creation… hyper-dimensional connections about concepts 3D
  • BE-SMART: GRAPH-BASED REASONING
    • Two-level segmentation graph for reasoning and annotation
    November, 2009 EGIT2010 - Laura Papaleo Semantic-oriented decomposition into manifold parts(*) Manifold Decomposition into meaningful parts Non-manifold segmentation graph Manifold segmentation graph Ontology-driven annotation Digital shape (*) De Floriani, Papaleo, Huang, Hendler. A semantic web environment for digital shapes understanding. LNCS, SAMT 2007. Ontology-driven processes
  • SEGMENTATION TECHNIQUES
    • Automatic : several well-behaved techniques in the literature
      • Fast 
        • Based on specific characteristics (curvature, texture..)
      • Can produce incorrect results 
    • Manual : less approaches in the literature
      •  Freedom for the user
      •  More work for the user
      •  Difficult in case of complex shapes
    November, 2009 EGIT2010 - Laura Papaleo
  • MANUAL SEGMENTATION TECHNIQUES
    • General purpose
    • Cut-based methods
      • allow the user to draw the cutting path on the model
    • Region-based methods
      • let the user select the interesting regions, and automatically compute the right position of the cut
    November, 2009 EGIT2010 - Laura Papaleo SHARF et al.: Snappaste: an interactive technique for easy mesh composition. The Visual Computer (2006) WU et al. sketch-based interactive framework for real-time mesh segmentation. Computer Graphics International (2007).
  • PREVIOUS APPROACH (AND LIMITATIONS) (1/2)
    • A framework able to combine automatic segmentation and merging techniques for manifold objects represented in X3D
    • Equipped with a simple manual editing tool
      • cut-based manual segmentation method: the user splits a region by manually drawing a cutting path
    November, 2009 EGIT2010 - Laura Papaleo DE FLORIANI L., PAPALEO L., CARISSIMI N.: A java3d framework for inspecting and segmenting 3d models. In Web3D (2008) ACM Segmentation into planar regions Region clustering Manual editing
  • PREVIOUS APPROACH (AND LIMITATIONS) (2/2)
    • The framework has a tool for inspecting both the segmented model and the produced segmentation graph
      • the user can navigate the graph discovering geometrical properties
      • once a region has been selected, it is possible to visualize a graphical representation of the corresponding node in the segmentation graph.
    November, 2009 EGIT2010 - Laura Papaleo
  • MANUAL SEGMENTATION BY STROKE PAINTING
    • Inspiration from Funkhouser [2004]
    • The user draws a stroke on the surface model with a width (r).
    • Cuts are considered along edges.
    • 2 edges paths (minimum paths) are computed (front and back sides)
    November, 2009 EGIT2010 - Laura Papaleo
      • [T. Funkhouser, et al.. Modeling by example. SIGGRAPH, 2004]
  • PROPERTIES (1/2)
    • We apply the algorithm on vertices of the model (not edges).
    • Equal methodology, but we reduce the number of operations to be performed
    • We to extend the procedure to surface meshes which are not necessarily triangular
    November, 2009 EGIT2010 - Laura Papaleo Set the viewpoint Draw the cut Compute the cut
  • PROPERTIES (2/2)
    • non-triangular meshes can be segmented
    • meshes do not need to be connected
    • use X3D coding, and define a new <shape> for each segmented portion
    • save separately selected regions and the associated annotation
    November, 2009 EGIT2010 - Laura Papaleo
  • MANIFOLD SEGMENTATION GRAPH
    • All steps are performed on the manifold segmentation graph
    • In the graph:
      • Nodes are regions
      • Arcs maintain boundaries between regions (chain of edges)
    November, 2009 EGIT2010 - Laura Papaleo
  • HIERARCHICAL SEMANTIC TAGGING
    • Goal: add information to a region
    • Every segmented region (node) C contains in the name also the names of its ancestors: ancestor 1 : … :ancestor n :RegionName
    • We can trace the entire segmentation process
    November, 2009 EGIT2010 - Laura Papaleo horse::head::leftEar
  • INTERFACE November, 2009 EGIT2010 - Laura Papaleo Xj3D browser Manifold segmentation graph
    • Patch information:
    • Semantic
    • Geometry
    • Adjacency
  • INSPECTING THE SEGMENTATION GRAPH
    • Implemented as an hyperbolic graph
    • Intuitive browsing and global visualization
    • Functionalities
      • Click on a node C, focus on C, highlight of the region
      • All nodes always visible
    November, 2009 EGIT2010 - Laura Papaleo
  • SUMMARY
    • We need semantics
      • Segmentation is the basis for semantic annotation
    • We presented
      • A cut-based manual segmentation tool (extensions/improvement of Intelligent scissoring).
      • An interface which allows the user to inspect both the segmented model and the segmentation graph
      • A semantic-based hierarchical tagging
      • Use of the X3D standard
    November, 2009 EGIT2010 - Laura Papaleo
  • FUTURE WORK
    • GEOMETRIC MODELING
    • SEMANTIC WEB - WEB SERVICES
    • Automatic computation of handles and through holes
    • Extend to high-dimensional data
    • Add new semi-automatic segmentation methods
    • Solution of the problem:
    • Definition of SemWeb services for reasoning
    • Ontology-driven processes performed by smart semantic agents
    • In specific applications pre-defined thesauri or dictionaries could be used
    • RDF/RDFS integration (OWL, SKOS)
    November, 2009 EGIT2010 - Laura Papaleo HCI!
  • ACKNOWLEDGEMENTS
    • Thanks to Matteo Bertucelli for the implementation of the manual segmentation tool and the interface
    • Shapes taken from the AIM@SHAPE shape repository and converted in X3D.
    November, 2009 EGIT2010 - Laura Papaleo www.mindingtheplanet.net The Semantic Web will lower the cost of processing knowledge to the same degree as the printing press lowered the cost of distributing knowledge.