This document provides a final report on the post-processing activities carried out in the 3D-ICONS project. It summarizes the completion of post-processing for 98.3% of planned 3D digitizations, resulting in 3230 3D models. The report discusses problems encountered, such as overlap between data acquisition and post-processing. It also analyzes common post-processing approaches, including geometric reconstruction, visual enrichment, and model structuring. Multiple representations were produced from some digitizations based on criteria like representation purpose and object scale. The relationship between post-processing and metadata creation is also examined.
AUTOMATIC IMAGE PROCESSING ENGINE ORIENTED ON QUALITY CONTROL OF ELECTRONIC B...sipij
We propose in this work a study of an image processing engine able to detect automatically the features of
electronic board weldings. The engine has been developed by using ImageJ and OpenCV libraries.
Specifically the image processing segmentation has been improved by watershed approach. After a
complete design of the automation processes, different test have been performed showing the engine
efficiency in terms of features extraction, scale setting and thresholding calibration. The engine provides as
outputs the storage of the cropped images of each single defects. The proposed engine together with the
post-processing 3D imaging represent a good tool for the management of the production quality of
electronic boards.
A Survey of Image Processing and Identification Techniquesvivatechijri
Image processing is always an interesting field as it gives enhanced visual data for human
simplification and processing of image data for transmission and illustration for machine preception. Digital
images are processed to give better solution using image processing. Techniques such as Gray scale
conversion, Image segmentation, Edge detection, Feature Extraction, Classification are used in image
processing.
In this paper studies of different image processing techniques and its methods has been conducted.
Image segmentation is the initial step in many image processing functions like Pattern recognition and image
analysis which convert an image into binary form and divide it into different regions. The technique used for
segmentation is Otsu’s method, K-means Clustering etc. For feature extraction feature vector in visual image is
texture, shape and color. Edge detector with morphological operator enhances the clarity of image and noise
free images. This paper also gives information about algorithm like Artificial Neural Network and Support
Vector Mechanism used for image classification. The image is categorized into the receptive class by an ANN
and SVM is used to compile all the categorized result. Overall the paper gives detail knowledge about the
techniques used for image processing and identification.
Image processing is among rapidly growing technologies today, with its applications in various aspects of a business. Image Processing forms core research area within electronics engineering and computer science disciplines too. Image Processing is a technique to enhance raw images received from satellites, space probes, aircrafts, military reconnaissance flights or pictures taken in normal day-to-day life from normal cameras. The field is becoming powerful and popular because of technically powerful personal computers, large memories of available devices as well as graphic softwares and tools available with that devices and gadgets. Image acquisition, pre-processing, segmentation, representation, recognition and interpretation are the different basic steps through which image processing is carried out. [3][4].
SEMANTIC IMAGE RETRIEVAL USING MULTIPLE FEATUREScscpconf
In Content Based Image Retrieval (CBIR) some problem such as recognizing the similar
images, the need for databases, the semantic gap, and retrieving the desired images from huge
collections are the keys to improve. CBIR system analyzes the image content for indexing,
management, extraction and retrieval via low-level features such as color, texture and shape.
To achieve higher semantic performance, recent system seeks to combine the low-level features
of images with high-level features that conation perceptual information for human beings.
Performance improvements of indexing and retrieval play an important role for providing
advanced CBIR services. To overcome these above problems, a new query-by-image technique
using combination of multiple features is proposed. The proposed technique efficiently sifts through the dataset of images to retrieve semantically similar images.
Bio medical image segmentation using marker controlled watershed algorithm a ...eSAT Publishing House
IJRET : International Journal of Research in Engineering and Technology is an international peer reviewed, online journal published by eSAT Publishing House for the enhancement of research in various disciplines of Engineering and Technology. The aim and scope of the journal is to provide an academic medium and an important reference for the advancement and dissemination of research results that support high-level learning, teaching and research in the fields of Engineering and Technology. We bring together Scientists, Academician, Field Engineers, Scholars and Students of related fields of Engineering and Technology
AUTOMATIC IMAGE PROCESSING ENGINE ORIENTED ON QUALITY CONTROL OF ELECTRONIC B...sipij
We propose in this work a study of an image processing engine able to detect automatically the features of
electronic board weldings. The engine has been developed by using ImageJ and OpenCV libraries.
Specifically the image processing segmentation has been improved by watershed approach. After a
complete design of the automation processes, different test have been performed showing the engine
efficiency in terms of features extraction, scale setting and thresholding calibration. The engine provides as
outputs the storage of the cropped images of each single defects. The proposed engine together with the
post-processing 3D imaging represent a good tool for the management of the production quality of
electronic boards.
A Survey of Image Processing and Identification Techniquesvivatechijri
Image processing is always an interesting field as it gives enhanced visual data for human
simplification and processing of image data for transmission and illustration for machine preception. Digital
images are processed to give better solution using image processing. Techniques such as Gray scale
conversion, Image segmentation, Edge detection, Feature Extraction, Classification are used in image
processing.
In this paper studies of different image processing techniques and its methods has been conducted.
Image segmentation is the initial step in many image processing functions like Pattern recognition and image
analysis which convert an image into binary form and divide it into different regions. The technique used for
segmentation is Otsu’s method, K-means Clustering etc. For feature extraction feature vector in visual image is
texture, shape and color. Edge detector with morphological operator enhances the clarity of image and noise
free images. This paper also gives information about algorithm like Artificial Neural Network and Support
Vector Mechanism used for image classification. The image is categorized into the receptive class by an ANN
and SVM is used to compile all the categorized result. Overall the paper gives detail knowledge about the
techniques used for image processing and identification.
Image processing is among rapidly growing technologies today, with its applications in various aspects of a business. Image Processing forms core research area within electronics engineering and computer science disciplines too. Image Processing is a technique to enhance raw images received from satellites, space probes, aircrafts, military reconnaissance flights or pictures taken in normal day-to-day life from normal cameras. The field is becoming powerful and popular because of technically powerful personal computers, large memories of available devices as well as graphic softwares and tools available with that devices and gadgets. Image acquisition, pre-processing, segmentation, representation, recognition and interpretation are the different basic steps through which image processing is carried out. [3][4].
SEMANTIC IMAGE RETRIEVAL USING MULTIPLE FEATUREScscpconf
In Content Based Image Retrieval (CBIR) some problem such as recognizing the similar
images, the need for databases, the semantic gap, and retrieving the desired images from huge
collections are the keys to improve. CBIR system analyzes the image content for indexing,
management, extraction and retrieval via low-level features such as color, texture and shape.
To achieve higher semantic performance, recent system seeks to combine the low-level features
of images with high-level features that conation perceptual information for human beings.
Performance improvements of indexing and retrieval play an important role for providing
advanced CBIR services. To overcome these above problems, a new query-by-image technique
using combination of multiple features is proposed. The proposed technique efficiently sifts through the dataset of images to retrieve semantically similar images.
Bio medical image segmentation using marker controlled watershed algorithm a ...eSAT Publishing House
IJRET : International Journal of Research in Engineering and Technology is an international peer reviewed, online journal published by eSAT Publishing House for the enhancement of research in various disciplines of Engineering and Technology. The aim and scope of the journal is to provide an academic medium and an important reference for the advancement and dissemination of research results that support high-level learning, teaching and research in the fields of Engineering and Technology. We bring together Scientists, Academician, Field Engineers, Scholars and Students of related fields of Engineering and Technology
Enhancing readability of digital image using image processing - Full ReportUpendra Sachan
The readability of digital images of documents (especially Black and White) can be greatly increased by removing the noise and enhancing image using adaptive threshold.
International Journal of Engineering Research and Applications (IJERA) is an open access online peer reviewed international journal that publishes research and review articles in the fields of Computer Science, Neural Networks, Electrical Engineering, Software Engineering, Information Technology, Mechanical Engineering, Chemical Engineering, Plastic Engineering, Food Technology, Textile Engineering, Nano Technology & science, Power Electronics, Electronics & Communication Engineering, Computational mathematics, Image processing, Civil Engineering, Structural Engineering, Environmental Engineering, VLSI Testing & Low Power VLSI Design etc.
COMPARISON OF RENDERING PROCESSES ON 3D MODELijcsit
Creating 3D vehicle model is complex process that requires basic knowledge of polygonal modeling. In this research, environment map is used as lighting with HDRI image.The final process of converting 3D scene to 2D image is called rendering. Image data will be obtained in four ways with various toolsets used in 3ds Max. They are: Scaneline, V-Ray, Mental Ray and Corona Renderer. At final step was made critical analysis on all of these techniques on the same computer system and excellent results were obtained.
Creating 3D vehicle model is complex process that requires basic knowledge of polygonal modeling.In this research, environment map is used as lighting with HDRI image.The final process of converting 3D scene to 2D image is called rendering.Image data will be obtained in four ways with various toolsets used in 3ds Max. They are: Scaneline, V-Ray, Mental Ray and Corona Renderer. At final step was made critical analysis on all of these techniques on the same computer system and excellent results were obtained.
Creating 3D vehicle model is complex process that requires basic knowledge of polygonal modeling. In
this research, environment map is used as lighting with HDRI image.The final process of converting 3D
scene to 2D image is called rendering. Image data will be obtained in four ways with various toolsets used
in 3ds Max. They are: Scaneline, V-Ray, Mental Ray and Corona Renderer. At final step was made critical
analysis on all of these techniques on the same computer system and excellent results were obtained.
Enhancing readability of digital image using image processing - Full ReportUpendra Sachan
The readability of digital images of documents (especially Black and White) can be greatly increased by removing the noise and enhancing image using adaptive threshold.
International Journal of Engineering Research and Applications (IJERA) is an open access online peer reviewed international journal that publishes research and review articles in the fields of Computer Science, Neural Networks, Electrical Engineering, Software Engineering, Information Technology, Mechanical Engineering, Chemical Engineering, Plastic Engineering, Food Technology, Textile Engineering, Nano Technology & science, Power Electronics, Electronics & Communication Engineering, Computational mathematics, Image processing, Civil Engineering, Structural Engineering, Environmental Engineering, VLSI Testing & Low Power VLSI Design etc.
COMPARISON OF RENDERING PROCESSES ON 3D MODELijcsit
Creating 3D vehicle model is complex process that requires basic knowledge of polygonal modeling. In this research, environment map is used as lighting with HDRI image.The final process of converting 3D scene to 2D image is called rendering. Image data will be obtained in four ways with various toolsets used in 3ds Max. They are: Scaneline, V-Ray, Mental Ray and Corona Renderer. At final step was made critical analysis on all of these techniques on the same computer system and excellent results were obtained.
Creating 3D vehicle model is complex process that requires basic knowledge of polygonal modeling.In this research, environment map is used as lighting with HDRI image.The final process of converting 3D scene to 2D image is called rendering.Image data will be obtained in four ways with various toolsets used in 3ds Max. They are: Scaneline, V-Ray, Mental Ray and Corona Renderer. At final step was made critical analysis on all of these techniques on the same computer system and excellent results were obtained.
Creating 3D vehicle model is complex process that requires basic knowledge of polygonal modeling. In
this research, environment map is used as lighting with HDRI image.The final process of converting 3D
scene to 2D image is called rendering. Image data will be obtained in four ways with various toolsets used
in 3ds Max. They are: Scaneline, V-Ray, Mental Ray and Corona Renderer. At final step was made critical
analysis on all of these techniques on the same computer system and excellent results were obtained.
Design and implement a reality-based 3D digitisation and modelling project3D ICONS Project
Design and implement a reality-based 3D digitisation and modelling project
Fabio Remondino, Fabio Menna, 3D Optical Metrology (3DOM) unit Bruno Kessler Foundation (FBK)
Anestis Koutsoudis, Christos Chamzas, Athena Research and Innovation Centre
Sabry El-Hakim, 4DHistory
Paper presented at Digital Heritage 2013
Abstract: 2D-to-3D conversion adds the binocular disparity depth cue to digital images perceived by the brain, thus, if done properly, greatly improving the immersive effect while viewing convert image in comparison to 2D image. However, in order to be successful, the conversion should be done with sufficient accuracy and correctness: the quality of the original 2D images should not deteriorate, and the introduced disparity cue should not contradict to other cues used by the brain for depth perception. If done properly and thoroughly, the conversion produces image of similar quality to "native” which is shot in stereo image and accurately adjusted and aligned in post-production.
3-Phase Recognition Approach to Pseudo 3D Building Generation from 2D Floor P...ijcga
Nowadays three dimension (3D) architectural visualisation has become a powerful tool in the
conceptualisation, design and presentation of architectural products in the construction industry, providing
realistic interaction and walkthrough on engineering products. Traditional ways of implementing 3D
models involves the use of specialised 3D authoring tools along with skilled 3D designers with blueprints of
the model and this is a slow and laborious process. The aim of this paper is to automate this process by
simply analyzing the blueprint document and generating the 3D scene automatically. For this purpose we
have devised a 3-Phase recognition approach to pseudo 3D building generation from 2D floor plan and
developed a software accordingly.
Ensuring Distributed Accountability in the CloudSuraj Mehta
Ensuring distributed accountability for data sharing in the cloud is in short nothing
but a novel highly decentralized information accountability framework to keep track
of the actual usage of the users' data in the cloud. Cloud computing enables highly
ecient services that are easily consumed over the internet.
3D ICONS: Europeana goes 3D, Daniel Pletinckx, Visual Dimension Belgium3D ICONS Project
This presentation by Daniel Pletinckx of Visual Dimension and 3D ICONS describes the requirements for publishing 3D media in Europeana. It discusses the developments in 3D PDF, Pseudo3D (panoramas), Unity3d/Unreal, Remote rendering, Web GL and Web GL based streaming, and 3DHOP (the 3D Heritage Online Presenter) developed by ISTI CNR. It then discusses the user experience of the various technical solutions and presents examples.
3D-ICONS: Interactive storytelling through innovative interfaces, Carlotta C...3D ICONS Project
This presentation by Carlotta Capurro and Daniel Pletinckx, (Visual Dimension bvba) gives an introduction to the 3D-ICONS guidelines for creating 3D models of cultural objects. It introduces 3D capture techniques, post-processing of 3D content, 3D publishing methodology, metadata, licencing and IPR considerations, and includes a case study of the digitisation of Ename, Belgium. A 4D visualisation of the Ename abbey site has been created providing a framework for interactive storytelling about the evolution of the abbey through time.
Metadata for 3D models, presentation given by Sheena Bassett at the ArcheoLandscapes Conference in Romania in October 2014.
The presentation describes the aims of the 3D ICONS project, the uses of metadata, the CARARE metadata schema, paradata and the requirements for metadata in the 3D ICONS project and for Europeana.
Analysis of the 3D reconstruction methodologies used within the framework of ...3D ICONS Project
Analysis of the 3D reconstruction methodologies used within the framework of the 3D-ICONS project - presentation given by Livio De Luca of CNRS, France at the 3D ICONS workshop at the ISPRS Technical Commission V Symposium, which was held in Riva del Garda, Italy on 23-25 June 2014.
The presentation analyses the 3D reconstruction methodologies used within the framework of the 3D ICONS project.
The 'Rubble of the North' -a solution for modelling the irregular architectur...3D ICONS Project
The 'Rubble of the North' -a solution for modelling the irregular architecture of Ireland's historic monuments - a presentation given by Rob Shaw of the Discovery Programme, Ireland at the 3D ICONS workshop at the ISPRS Technical Commission V Symposium, which was held in Riva del Garda, Italy on 23-25 June 2014.
The presentation gives and overview of the digitisation, the challenges faced, solutions and deliverables.
Digitisation, processing and visualisation of monuments within the 3D-ICONS f...3D ICONS Project
Digitisation, processing and visualisation of monuments within the 3D-ICONS framework: The case of Athena Research Centre, Xanthi. Presentation given by Anestis Koutsoudis at the 3D ICONS workshop at ISPRS Technical Commission V Symposium, which was held in Riva del Garda, Italy on 23-25 June 2014.
The presentation describes the Byzantine churches and monastic monuments that were digitised by the Athena Research Centre and the processes that were used.
The last mile of 3DIcons: making available 3D contents and their metadata thr...3D ICONS Project
'The last mile of 3D ICONS: making available 3D contents and their metadata through Europeana' presentation given by Sara Gonizzi at the 3D ICONS workshop at the ISPRS Technical Commission V Symposium, which was held in Riva del Garda, Italy on 23-25 June 2014.
The presentation describes the process of digitising artefacts held at the Archaeological Museum of Milan in 3D, and then capturing the metadata and paradata for the content.
Combining the outcomes of CARARE and 3D-COFORM, Andrea D'Andrea3D ICONS Project
Presentation given by Andrea D'Andrea at the 3D ICONS workshop at VAST 2012 on "Combining the outcomes of CARARE and 3D-COFORM to capture in 3D, store, manage and retrieve the digital Monuments of Europe ".
Managing archaeological knowledge. The experience of CISA-UNO. Andrea D'Andrea3D ICONS Project
Presentation given by Andrea D'Andreat in Naples on 19th February 2014 describing the experience of CISA-UNO in 3D ICONS and other projects including ArcheoZone, ALUKA, INNOVA, EPOCH and DICOR. In Italian
Part 1 of the printed publication "3D-ICONS Guidelines and Case Studies" First published in November 2014.
Public fascination with the architectural and archaeological heritage is well known, it is proven to be one of the main reasons for tourism according to the UN World Tourism Organisation. Historic buildings and archaeological monuments form a significant component Europe’s cultural heritage; they are the physical testimonies of European history and of the di°erent events that led to the creation of the European landscape, as we know it today.
The documentation of built heritage increasingly avails of 3D scanning and other remote sensing technologies, which produces digital replicas in an accurate and fast way. Such digital models have a large range of uses, from the conservation and preservation of monuments to the communication of their cultural value to the public. They may also support in-depth analysis of their architectural and artistic features as well as allow the production of interpretive reconstructions of their past appearance.
The goal of the 3D-ICONS project, funded under the European Commission’s ICT Policy Support Programme which builds on the results of CARARE (www.carare.eu) and 3D-COFORM (www.3d-coform.eu), is to provide Europeana with 3D models of architectural and archaeological monuments of remarkable cultural importance. The project brings together 16 partners (see appendix 2) from across Europe (11 countries) with relevant expertise in 3D modelling and digitization. The main purpose of this project is to produce around 4000 accurate 3D models which have to be processed into a simplified form in order to be visualized on low end personal computers and on the web.
3D ICONS Guidelines and Case Studies, Anthony Corns, Discovery Programme3D ICONS Project
A presentation about the 3D ICONS Guidelines and Case Studies given by Anthony Corns of the Discovery Programme at the 3D ICONS workshop, Borsa Mediteranea in Pasetum.
3D ICONS has published Guidelines which cover documentation of the digitisation, modelling, online access pipeline for the creation of online 3D models of cultural heritage objects. The document includes 28 Case Studies - examples of 3D content creation by project partners across a range of monuments, architectural features and artefacts.
http://3dicons-project.eu
3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi ...3D ICONS Project
3D processing and metadata ingestion at POLIMI, Presentation given by Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli at the 3D ICONS workshop at the XVIII Borso Mediterranea del Turismo Archeologico conference in Paestrum.
The presentation describes the 3D digitisation carried out by Politecnico di Milano (POLIMI0 as part of the 3D ICONS project.
3D data acquisition and archaeological documentation, Alberto Sanchez, France...3D ICONS Project
3D data acquisition and archaeological documentation, presentation given by Alberto Sanchez, Francesco Gomez, Ana Martinez and Arturo Ruiz at the 3D ICONS workshop at the XVIII Borso Mediterranea del Turismo Archeologico conference in Paestrum.
The presentation describes the contribution of the University of Jaen Research Institute for Iberian Archaeology to the 3D ICONS project and the methodology used.
Sorin Hermon, 'Towards an integrated repository for research and management o...3D ICONS Project
Sorin Hermon, 'Towards an integrated repository for research and management of archaeological 3D assets', presentation given at the World Archaeology Congress, Jordan, January 2013.
Franco Niccolucci, 'The integration and management of archaeological datasets...3D ICONS Project
Franco Niccolucci, 'The integration and management of archaeological datasets: the Europeana projects CARARE and 3D ICONS', a position paper given at the World Archaeology Congress, Jordan, January 2013
'Cultural Heritage under Lenses: 3D Icons Project and the Romanian experience...3D ICONS Project
E. Oberländer-Târnoveanu, Corina Nicolae, Mihai Bozgan, Marius Amarie and Tudor Martin, 'Cultural Heritage under Lenses: 3D Icons Project and the Romanian experience ', presentation given at the Cultural Heritage Creative Tools And Archives workshop, National Museum of Denmark, Copenhagen, 26-27 June 2013.
Creating Virtual Reality for Cultural Heritage. 3D Icons Project in Romania3D ICONS Project
E. Oberländer-Târnoveanu, Corina Nicolae, Mihai Bozgan, Marius Amarie and Tudor Martin, 'Creating Virtual Reality for Cultural Heritage. 3D Icons Project in Romania ', presentation given at the Congress 3D-Documentation in Archaeology & Monument Preservation, held at LWL Industrial Museum, Dortmund (Germany), 16th-18th October 2013
Levelwise PageRank with Loop-Based Dead End Handling Strategy : SHORT REPORT ...Subhajit Sahu
Abstract — Levelwise PageRank is an alternative method of PageRank computation which decomposes the input graph into a directed acyclic block-graph of strongly connected components, and processes them in topological order, one level at a time. This enables calculation for ranks in a distributed fashion without per-iteration communication, unlike the standard method where all vertices are processed in each iteration. It however comes with a precondition of the absence of dead ends in the input graph. Here, the native non-distributed performance of Levelwise PageRank was compared against Monolithic PageRank on a CPU as well as a GPU. To ensure a fair comparison, Monolithic PageRank was also performed on a graph where vertices were split by components. Results indicate that Levelwise PageRank is about as fast as Monolithic PageRank on the CPU, but quite a bit slower on the GPU. Slowdown on the GPU is likely caused by a large submission of small workloads, and expected to be non-issue when the computation is performed on massive graphs.
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Data and AI
Round table discussion of vector databases, unstructured data, ai, big data, real-time, robots and Milvus.
A lively discussion with NJ Gen AI Meetup Lead, Prasad and Procure.FYI's Co-Found
Enhanced Enterprise Intelligence with your personal AI Data Copilot.pdfGetInData
Recently we have observed the rise of open-source Large Language Models (LLMs) that are community-driven or developed by the AI market leaders, such as Meta (Llama3), Databricks (DBRX) and Snowflake (Arctic). On the other hand, there is a growth in interest in specialized, carefully fine-tuned yet relatively small models that can efficiently assist programmers in day-to-day tasks. Finally, Retrieval-Augmented Generation (RAG) architectures have gained a lot of traction as the preferred approach for LLMs context and prompt augmentation for building conversational SQL data copilots, code copilots and chatbots.
In this presentation, we will show how we built upon these three concepts a robust Data Copilot that can help to democratize access to company data assets and boost performance of everyone working with data platforms.
Why do we need yet another (open-source ) Copilot?
How can we build one?
Architecture and evaluation
Unleashing the Power of Data_ Choosing a Trusted Analytics Platform.pdfEnterprise Wired
In this guide, we'll explore the key considerations and features to look for when choosing a Trusted analytics platform that meets your organization's needs and delivers actionable intelligence you can trust.
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Data and AI
Discussion on Vector Databases, Unstructured Data and AI
https://www.meetup.com/unstructured-data-meetup-new-york/
This meetup is for people working in unstructured data. Speakers will come present about related topics such as vector databases, LLMs, and managing data at scale. The intended audience of this group includes roles like machine learning engineers, data scientists, data engineers, software engineers, and PMs.This meetup was formerly Milvus Meetup, and is sponsored by Zilliz maintainers of Milvus.
Chatty Kathy - UNC Bootcamp Final Project Presentation - Final Version - 5.23...John Andrews
SlideShare Description for "Chatty Kathy - UNC Bootcamp Final Project Presentation"
Title: Chatty Kathy: Enhancing Physical Activity Among Older Adults
Description:
Discover how Chatty Kathy, an innovative project developed at the UNC Bootcamp, aims to tackle the challenge of low physical activity among older adults. Our AI-driven solution uses peer interaction to boost and sustain exercise levels, significantly improving health outcomes. This presentation covers our problem statement, the rationale behind Chatty Kathy, synthetic data and persona creation, model performance metrics, a visual demonstration of the project, and potential future developments. Join us for an insightful Q&A session to explore the potential of this groundbreaking project.
Project Team: Jay Requarth, Jana Avery, John Andrews, Dr. Dick Davis II, Nee Buntoum, Nam Yeongjin & Mat Nicholas
Adjusting primitives for graph : SHORT REPORT / NOTESSubhajit Sahu
Graph algorithms, like PageRank Compressed Sparse Row (CSR) is an adjacency-list based graph representation that is
Multiply with different modes (map)
1. Performance of sequential execution based vs OpenMP based vector multiply.
2. Comparing various launch configs for CUDA based vector multiply.
Sum with different storage types (reduce)
1. Performance of vector element sum using float vs bfloat16 as the storage type.
Sum with different modes (reduce)
1. Performance of sequential execution based vs OpenMP based vector element sum.
2. Performance of memcpy vs in-place based CUDA based vector element sum.
3. Comparing various launch configs for CUDA based vector element sum (memcpy).
4. Comparing various launch configs for CUDA based vector element sum (in-place).
Sum with in-place strategies of CUDA mode (reduce)
1. Comparing various launch configs for CUDA based vector element sum (in-place).
1. I C O N S
D4.3: Final Report on Post-processing
3D ICONS is funded by the European Commission’s
ICT Policy Support Programme
Author:
L. De Luca (CNRS)
3D Digitisation of Icons of European Architectural and Archaeological Heritage
2. Revision History
Rev. Date Author Org. Description
0.1 25/08/14 L. De Luca CNRS Draft
0,2 27/08/14 S. Bassett CISA Revised version
0.3 01/09/14 L. De Luca CNRS Revised version
Revision: [Final]
Author:
L. De Luca (CNRS)
Statement of originality:
This deliverable contains original unpublished work except where clearly indicated
otherwise. Acknowledgement of previously published material and of the work of
others has been made through appropriate citation, quotation or both.
3D-ICONS is a project funded under the European Commission’s ICT Policy Support
Programme, project no. 297194.
The views and opinions expressed in this presentation are the sole responsibility of the
authors and do not necessarily reflect the views of the European Commission.
3. iv
Contents
Executive Summary.............................................................................................................................................. 6
1. Introduction ....................................................................................................................................................... 7
1.2 Post-processing in the 3D-icons project .......................................................................................... 7
1.3 What is 3D post-processing? ................................................................................................................ 7
2. General overview of the post-processing activities carried out within the 3D-ICONS
project........................................................................................................................................................................ 9
2.1 Problems encountered and actions undertaken ........................................................................... 9
2.2 Progress on post-processing activities .......................................................................................... 10
3. Study of the methodological approaches for post-processing..................................................... 12
3.1 Introduction ............................................................................................................................................. 12
3.2 Classification of elementary “processing steps” ........................................................................ 13
3.2.1 Geometric reconstruction .......................................................................................................... 13
3.2.2 Visual enrichment ......................................................................................................................... 14
3.2.3 Model structuring .......................................................................................................................... 15
3.2.4 Hypothetical reconstruction ..................................................................................................... 17
3.2.5 Complementary 2D media (derived from the 3D model) .............................................. 18
3.3 Case studies on post-processing ...................................................................................................... 18
3.3.1 Cerveteri and Tarquinia Tombs (FBK) .................................................................................. 19
3.3.2 Centre Pompidou in Paris (CNRS) ........................................................................................... 19
3.3.3 Treasury of Marseille in Delphi (CNRS) ............................................................................... 19
3.3.4 Roman archaeological finds (POLIMI) .................................................................................. 19
3.4 Statistics on the employed approaches, techniques and file formats ................................ 20
4. 3.4.1 Geometric reconstruction .......................................................................................................... 20
3.4.2 Visual enrichment ......................................................................................................................... 21
3.4.3 Model structuring .......................................................................................................................... 21
3.4.4 Hypothetical reconstruction ..................................................................................................... 22
3.5 Typical post-processing pipelines. .................................................................................................. 23
4. Producing multiple digital assets starting from a 3D digitisation .............................................. 24
4.1 Criteria for multiplying the digital assets starting from a 3D digitisation campaign .. 25
4.2 A case study: the abbey of Saint-Michel de Cuxa ....................................................................... 26
4.3 Multiple representations, multiple resolutions ......................................................................... 29
5. Linking the 3D processing pipeline to the metadata creation ..................................................... 30
6. Conclusions ...................................................................................................................................................... 35
v
5. Executive Summary
Deliverable 4.3 is a final presentation of all the aspects concerning the post-processing of
the 3D digitisations that will be supplied to Europeana by the content providers. The
report describes the completion of the geometrical and graphical phase of the 3D
digitisation, with respect to the schedule and the fulfilment of the identified requirements,
also including some problems that were met and the solutions adopted.
It provides a global overview (summarising and incorporating the outcomes of D4.1 –
interim report) of the post-processing phase by analysing and classifying the most
commonly used approaches and techniques for the elaboration and the refinement of 3D
models. Finally, this report includes a discussion about some approaches for producing
several final outputs starting from a 3D digitization according to different representation
purposes as well as an explanation of the relation between the post-processing work and
the metadata creation.
Activities concerning WP 4 started at month 12 and ended at month 30 of the project. As
presented in section 2.3 at the month 30 of the project (July 2014), the completed 3D
digitisations (98,3% of the total planned) have been post-processed by producing 3230 3D
models ready to be converted in the final publication formats (WP5).
6
6. D4.3 Final
( n (-(c
ing
7
1. Introduction
1.2 Post-processing in the 3D-icons project
The 3D-ICONS project will provide to Europeana an important collection of reality-based
3D models (detailed and accurate geometric representations) for further uses such as
documentation, digital inventories, cultural dissemination, etc. This report concerns the
activities carried out within the framework of the task 4.1 :
• The refinement of the 3D digitization produced in WP3 (Digitization), including
geometric reconstruction, visual enrichment and model structuring;
• The making of all necessary graphical and content refinement improvements of the
model and the other data provided;
• The creation of some complementary 2D media (derived from the 3D model) such
as video tours, panoramic images, etc. for enriching the documentation of the
monument and its details.
After a brief introduction, this final report presents a general overview of the post-processing
activities (see section 2) discussing the actions undertaken in order to
coordinate the implementation as well as the monitoring. Section 3 focuses on a discussion
about technical and methodological issues (lessons learned) about the most used post-processing
approaches, while section 4 explain how the produced data sets (including 3D
digitations, post-processed 3D models and complementary 2D media) are used for
elaborating a collection of digital assets documenting an heritage asset according to several
criteria (representation purposes, temporal states, object scales, etc..) ready to be
converted in the final publication formats (WP5). Finally, section 5 discusses the
relationship between the post-processing workflow and the metadata creation.
1.3 What is 3D post-processing?
First of all, it is necessary to start the discussion with an explanation of what is meant by
3D post-processing. To provide a simple and clear example, one can first refer to the post-
7. processing of 2D images (see Figure 1 on the left hand side below). This process is
generally well known in the field of digitisation and generally involves applying some
filters to enhance the image or change its resolution. Unlike the 2D post-processing, 3D
post-processing (see Figure 1 on the right) is a much more complex process consisting of
several processing steps concerning the direct improvement of the acquired data (by laser
scanning, photogrammetry, etc. ..) as well as its transformation into visually enriched (and
in some cases semantically structured) geometric representations. As can be understood in
section 4, post-processing also allows the elaboration of multiple 3D models starting from
the gathered data according to various representations purposes, levels of resolution and
description strategies (segmentation of elements in parts and sub-parts). The results of the
post-processing phase are 3D geometric representations accompanied by complementary
2D media, which are the digital assets ready to be converted (or embedded) into the final
web publishing formats (WP5).
Figure 1. Difference between 2D and 3D post-processing
8
8. 2. General overview of the post-processing activities carried
out within the 3D-ICONS project
2.1 Problems encountered and actions undertaken
Initial internal discussions focused on finding a good way for articulating the post-processing
activities within the entire pipeline. This revealed an important overlap
between the two work packages concerning the digitization implementation: WP3 (data
acquisition) and WP4 (post-processing). In fact, several processing activities can be
considered to be part of the data acquisition (WP3) as well as part of the post-processing
(WP4) according to the used digitisation and processing approach. In some cases, the 3D
digitization solutions (integrating hardware and software) merge the data acquisition
(WP3), the geometric reconstruction (WP3/4) and the visual enrichment (WP4) into a
unique automatic process; in other cases, the data acquisition (WP3) is strongly
independent from the 3D modelling and structuring strategy (WP3/WP4), and so on.
For example, concerning the 3D modelling carried out by automatic geometry processing
techniques (e.g. the 3D digitisation of a sculpture), the post-processing is affected by factors
hindering 3D data collection such as transparency, reflectance, poor light conditions,
accessibility of the object and its position, geometric complexity of the object etc. In this
case, the post-processing consists of applying functions such as filling holes and gaps,
correction of corrupted or duplicated sides and vertices, refinement and cleaning
anomalies caused by defects of light conditions and texturing.
But a historic building or an archaeological site is generally composed by the articulation of
elements at various scales and levels of geometric complexity. As a consequence, the 3D
digitisation requires the integration of various 3D digitisation and modelling techniques.
This specific context makes it difficult to really segment some of the more complex
pipelines into well-identified processing steps. In fact, the overall pipeline generally
depends on the main purpose of the 3D digitisation, the specific strategy to produce the
general model, as well as the final visual and geometric result to be achieved.
In addition, some content providers in the project have significant experience with 3D
digitization and representation of heritage artefacts, sometime coupled with an in-depth
knowledge of tools, methods and approaches, which constitute the results of their scientific
activity. Furthermore, beyond the use of common and commercial solutions for handling
3D representations in various formats, within the context of this project, and in order to
obtain excellent results in terms of geometric and visual accuracy, some project partners
are experimenting with the integration of several tools and emerging technologies into
more complex 3D reconstruction approaches.
9
9. Due to the WP3/WP4 overlap problem as well as the high heterogeneity of the developed
approaches (sometime resulting from a strong integration of technical and methodological
aspects), first of all we carried out the study of a common basis to be used for classifying
the different operating modes, as well the data structures and the file formats used for
handling geometrical and visual content (see section 3). This classification has a twofold
objective: on one hand the aim of harmonizing the production of 3D models within the
framework of the project (as well as the description of the processing steps during the
technical metadata creation phase); on the other hand, the need to identify common
refinement pipelines according to the final publication formats (WP5).
Another important action undertaken concerns the definition of a strategy for considering
the post-processing phase as a key moment for producing 3D models according to various
description purposes. The technical review (month 24) recommended identifying a set of
clear guidelines for the identification of individual 3D objects to be delivered to Europeana.
From a technical point of view, this issue concerns the post-processing phase and
contributes to the critical mass of delivered objects (see table 1 the next section). A set of
criteria have been identified in order to isolate some individual (and interesting) assets
from the surrounding structure (e.g. architectural elements, sculpted details, furnishing,
etc…) as well as to produce several representations of the same heritage assets according
to specific purposes (see section 4).
2.2 Progress on post-processing activities
Activities concerning the WP4 started at the month 12 and ended at the month 30 of the
project. At the end of this WP (july 2014), the completed 3D digitisations (98,3% of the
total planned) have been post-processed by producing 3230 3D models ready to be
converted in the final publication formats (WP5). The following table shows in the first
column the number of planned 3D acquisitions, in the second one, the number of objects
for which partners already achieved the 3D digitisation (WP3) and in the third one, the
final number of 3D digital assets resulting from the post-processing (WP4).
10
Planned 3D
Acquisitions
WP3
Completion of
Digitization
WP4
Completion of
Modelling
ARCHEOTRANSFERT 207 207 140
CETI 18 18 48
CISA 128 117 98
CMC 20 20 21
10. CNR-ISTI 210 210 157
CNR-ITABC 155 138 109
CYI-STARC 71 70 70
DISC 117 117 53
FBK 57 57 64
KMKG 455 455 693
MAP-CNRS 349 331 521
MNIR 80 80 61
POLIMI 531 531 743
UJA-CAAI 586 586 402
VisDim 50 50 50
Total 3034 2987 3230
Table 1. Progress of the 3D digitisation (acquisition and post-processing) activities carried
out by the partners at the month 30 (July 2014).
One can consider this progress good, because the post processing activities concern the
complex phase of the elaboration of final 3D representations starting from the gathered
data. In fact, as explained in section 3, according to the morphological complexity of the
artefact, different geometric reconstruction and visual enrichment techniques are often
integrated. In the case of complex architectural buildings, even if the digitization phase is
often carried out in an automatic way (e.g. 3D laser scanning) an important activity of
interactive modelling and structuring (time expensive) is required.
By analysing the values reported into this table, one can notice an increment of the number
of 3D models from WP3 to WP4. In fact, the 2987 completed 3D digitisations (results of the
3D scanning) have been used to produce 3230 digital assets.
This “increment” is driven by a set of criteria presented in section 4 and depends on the
nature of the heritage asset and on the post-processing activities which include several
possible techniques for the 3D geometric reconstruction, the visual enrichment, the model
structuring (and in some case the model segmentation into individual elements), as well as
the production of alternative representations (e.g. hypothetical states, levels of details,
etc…). The multiplication of the 3D models is carried out thanks to contribution of the
scientific advising (specialists or experts that are in all institutions involved in 3D-ICONS).
Furthermore, some models have been multiplied, as it wasn't possible to publish them in
high resolution given the morphological complexity of the heritage asset.
11
11. For example, KMKG produced 693 3D models starting from 455 digitisations, MAP-CNRS
521 3D models from 331 digitisations and POLIMI 743 3D models starting from
531 digitisations. This shows a positive trend illustrating that the overall number of the
ingested models will increase in the next months, according to the final conversion towards
the publication formats (WP5).
3. Study of the methodological approaches for post-processing
This section incorporate the results presented in the D4.1 - interim report (see section 3.1
to 3.2), then presents the results of a survey (see section 3.3) carried out within the
partners of the project in order to provide some statistics concerning the most used
techniques and formats for the post-processing phase. The study of these methodological
approaches, accompanied by an in-depth analysis of four case studies is then used as
starting point for identifying two typical post-processing pipelines determining the final
outputs to be converted into 3D interactive media (within the framework of the WP5 –
publication).
3.1 Introduction
From a purely technological standpoint, during the generation of the 3D representation of
an artefact, the geometrical precision, the detailed representation of visual aspects, the 3D
real-time visualization are fundamental aspects. But, beyond the application of a technical
process, whether simple or integrated, the methodological dimensions plays an important
role in identifying the objectives of 3D representation that can be achieved starting from a
3D-digitization. Indeed, in several cases, the type of representation desired for the
communication purpose (or required for the analysis needs) determines the data
processing and post-processing strategy. In fact, the elaboration of 3D models of complex
heritage buildings in their current state, as well as the reconstruction of their hypothetical
past states, requires the definition of specific pipelines integrating surveying tools,
geometric modelling techniques and (in some cases) an overall model structuring strategy.
Four main approaches are representative of the diffuse practices among the 3D-ICONS
partners working on the digital documentation and visualization of heritage artefacts
Firstly, some approaches are inclined to represent the geometric accuracy of 3D
models: these are mainly based on methods of automatic meshing starting from a 3D laser
scanning acquisition. Secondly, other approaches are based on morphological
descriptions that are specific to particular kinds of analysis (e.g. temporal transformations,
architectural composition, etc.); they are characterized by data acquisition and data
processing strategies consistent with specific representation goals. Thirdly, some
techniques focus on reproducing the visual appearance of the surfaces forming the
object, by taking into account photographic information. Finally, other approaches
12
12. concentrate on the simultaneous representation of multiple factors at multiple scales:
for this goal, they use different technical procedures in a complementary way.
In order to define a set of common elements of an overall methodology (taking into account
the heterogeneity of the 3D reconstruction approaches used by the project content
providers), we firstly analysed several 3D reconstruction projects carried by content
provides in order to produce a classification of the main procedures. This first analysis,
carried out by studying and comparing a range of technical solutions, allowed the structure
of a set of typical 3D reconstruction pipelines (from the 3D surveying on-site to the final 3D
model used for various application contexts) to be identified.
In the following section, post-processing issues and solutions provided by partners are
used as examples for each of the post-processing steps where applicable.
3.2 Classification of elementary “processing steps”
For the purpose of identifying a set of elementary “processing steps” which can be
combined in several ways in order to compose 3D reconstruction pipelines, this analysis
takes into account the:
• Deployed digitization tools (3D scanners, digital cameras, etc.);
• Employed acquisition, geometric modelling and visual enrichment techniques;
• Source and final data formats.
Five general post-processing aspects (detailed in the following sections) have been
identified: geometric reconstruction, visual enrichment, model structuring, hypothetical
reconstruction and complementary 3D media (derived from the 3D model). This work is
mainly based on the experiences of CNRS-MAP, CMC and UJA-CAAI as well as on several
inputs coming from the WP2 (CNR-ISTI) and WP3 (POLIMI, FBK) leaders.
3.2.1 Geometric reconstruction
The geometric reconstruction is the essential processing step for the elaboration of a 3D
representation of an artefact (starting from the results of a digitization campaign). The
choice of the relevant technique (see Figure 2) for this step is generally based on the
evaluation of the morphological complexity of the object, its scale, as well as the purpose of
the final 3D representation (e.g. graphic documentation, metric analysis, dissemination,
etc.).
13
13. Figure 2. Examples of geometric reconstruction techniques (CNRS-MAP)
A simple criterion for choosing (and evaluating) a relevant 3D reconstruction technique is the
degree of consistency of the 3D model with the real object. The list of techniques below is ordered
from those, which ensure high geometric consistency with the real object to the techniques that
introduce increasing levels of approximations:
• Automatic meshing from a dense 3D point cloud;
• Interactive or semi-automatic reconstruction based on relevant profiles;
• Interactive or semi-automatic reconstruction based on primitives adjustment;
• Interactive reconstruction based on technical iconography (plans, cross-sections
14
and elevations);
• Interactive reconstruction based on artistic iconography (sketches, paintings, etc.)
3.2.2 Visual enrichment
With regard to the visual enhancement of the geometric 3D reconstructions, several computer
graphics techniques were systematically examined in order to assess their degree of relevance in
the specific context of the digitization of heritage artefacts. As this project aims to provide
detailed and geometrically accurate 3D digitisations of heritage artefacts, our analysis primarily
focuses on techniques that provide the simulation of visual characteristics in geometric
consistency with the real object (see Figure 3). Other techniques, mainly used for cultural
disseminations purposes, are taken into account.
14. 15
Figure 3. Example of visual enrichment
based on the projection of textures starting
from photographs finely oriented on the 3D
model (CNRS-MAP)
The list of visual enrichment techniques below are ordered from those that ensure a strong
geometric consistency with the real object to the techniques that introduce increasing
approximations:
• Texture extraction and projection starting from photographs finely oriented on the
3D model (e.g. image-based modelling, photogrammetry);
• Texturing by photographic samples of the real materials of the artefact;
• Texturing by generic shaders.
Concerning the archiving of the results of the visual enrichment processes (textures, shaders,
colours, etc.), it’s very difficult to identify some general approaches because, in most cases, these
aspects are directly related to the 3D modelling and rendering software used, and often directly
embedded (in proprietary format) into the general 3D scene.
• Storing textures at different levels of resolution;
• Storing the source images used to generate textures;
• Using standard formats for image-based textures;
• Using standard descriptions of shaders.
3.2.3 Model structuring
Depending on the scale and on the morphological complexity, a geometric 3D
reconstruction of an architectural object or an archaeological site generally leads to the
representation of a single (and complex) geometric mesh or a collection of geometric
entities organized according to several criteria. The model structuring strategy (see Figure
4) is generally carried out with the aim of harmonising the hierarchical relations, which can
express the architectural composition of a building (e.g. relations between entities and
15. layouts) and can also be used as a guideline for structuring the related metadata. In some
cases, it could be important to identify a scientific advisor ensuring the consistency of the
chosen segmentation (e.g. temporal layers) and nomenclature (e.g. specialised vocabulary).
Figure 4. Example of 3D model structuring (CNRS-MAP) : on the left, according to temporal
states; on the right, according to a morphological segmentation (architectural units).
According to the technique used and to the general purpose of the 3D representation, the
results of a geometric reconstruction can be structured in four ways:
• Single unstructured entity (e.g. dense point clouds, or detailed mesh);
• Decomposed in elementary entities (e.g. 3D models composed by few parts);
• Decomposed in elementary entities hierarchically organized ( e.g. 3D models
decomposed in several parts for expressing the architectural layouts);
• Decomposed in entities organized in classes (e.g. 3D models decomposed in several
parts for expressing the classification of materials, temporal states, etc.).
According to the chosen model structuring strategy, the final dataset structure (including
geometry and visual enrichment) can be composed in several ways.
16
3D geometry:
• Single structured 3D file (with one level of detail);
• Multiple independent 3D files (with one level of detail);
• Multiple independent 3D files (with multiple level of detail);
• Hierarchical partition-based multi-resolution data structure, in a single file.
16. 17
Textures:
• Embedded into the 3D geometry file;
• Stored as external 2D files.
3.2.4 Hypothetical reconstruction
The hypothetical reconstruction of the past state of an architectural object or
archaeological site is an issue primarily related to field of historical studies. Nevertheless,
some specific technical and methodological issues with 3D graphical representation of
disappeared (or partially disappeared) heritage buildings are often integrated in 3D
reconstruction approaches. While primarily related to the analysis of iconographic sources
and historical knowledge, the methodological approaches for the elaboration of
hypothetical reconstructions (see Figure 5) can be based on the integration of metric
representations (2D or 3D) of existing parts of the object, as well as on the reconstruction
of the objects shapes starting from non-metric graphic descriptions of the artefact.
According to the informative degree of the iconography available as a source, the
elaboration of the 3D representation of the hypothetical state of an artefact, may mainly be
carried out based upon :
• the 3D acquisition of existing (or existed) parts;
• previous 2D surveys of existing (or existed) parts;
• non-metric iconographic sources of the studied artefact;
• iconographic sources (metric and / or non-metric) related to similar artefacts;
These methods (which can also be used in a complementary way) do not necessarily
determine the procedures, but emphasize the importance of the scientific dimension, the
intellectual rigor and transparency in the development of a hypothetical reconstruction. In
this sense, in order to include the 3D reconstruction of hypothetical states of an artefact
into an effective context of production of historical knowledge, the following
recommendations should be taken into account:
• Identify the scientific advisor(s) which can guide and validate the 3D model during
its elaboration;
• Save information about iconographic sources and bibliographical references and
used in the elaboration of the 3D model;
• Identify and save information indicating the degree of uncertainty (information
gaps, doubts, etc.).
17. 18
Figure 5. Example of 3D hypothetical
reconstruction of a past state (CNRS-MAP)
3.2.5 Complementary 2D media (derived from the 3D model)
During the elaboration of the 3D representation of a heritage artefact, complementary 2D
are produced starting from the 3D model. This 2D media can be produced in different ways,
depending on the type of 3D source (point cloud, geometric model, visually-enriched 3D
model), as well as on the final visualization type (static, dynamic, interactive). As those 2D
documentary media are directly derived from the 3D model, with the aim of anticipating
the metadata creation, it could be important to underline the relationships which can be
established between images, videos, etc. and the 3D model. 2D complementary media can
be produced starting from (and still remain linked to) the 3D general model of the entire
architectural or archaeological artefact or to entities or sub-entities of the 3D general
model (see section 4).
3.3 Case studies on post-processing
Starting from the results of the aforementioned classification, a set of typical 3D post-processing
pipelines has been identified by basing on several aspects. First, the common
articulation of several elementary processing steps into a relevant process. Second, the
evaluation of the relevance of each elementary processing steps (as well as the
combination of some processing sequences) according to the object scale, its morphological
complexity and its final representation format. The following sections illustrate some
examples showing the relationship between the characters of the artefact (morphological
complexity, scale, typology, etc.) and the chosen post-processing pipeline (composed by
several elementary processing steps).
18. 19
3.3.1 Cerveteri and Tarquinia Tombs (FBK)
A - GEOMETRIC RECONSTRUCTION
• Automatic meshing from a dense 3D point cloud
B - MODEL STRUCTURING
• Unstructured, single mesh
C - VISUAL ENRICHMENT
• Texture extraction and projection starting from
photographs finely oriented on the 3D model
(photogrammetry)
D - DATASET STRUCTURE (Geometry)
• Single 3D file with 1 level of detail
E - DATASET STRUCTURE (Textures)
• Sored as external 2D file
3.3.2 Centre Pompidou in Paris (CNRS)
A - GEOMETRIC RECONSTRUCTION
• Interactive and semi-automatic reconstruction
based on relevant profiles
• Interactive reconstruction based on technical
iconography (plans, cross-sections and elevations)
B - MODEL STRUCTURING
• Decomposed in elementary entities hierarchically
organized (by following the architectural layout)
• Decomposed in entities organized in classes
(materials)
C - VISUAL ENRICHMENT
• Texturing by photographic samples of the real
materials of the artifact
• Texturing by generic shaders
D - DATASET STRUCTURE (Geometry)
• Multiple independent 3D files with multiple levels of
details
E - DATASET STRUCTURE (Textures)
• Embedded into the 3D geometry file (generic
shaders)
• External 2D images (external textures)
19. 19
3.3.3 Treasury of Marseille in Delphi (CNRS)
A - GEOMETRIC RECONSTRUCTION
• Automatic meshing from a dense 3D point cloud
(sculpted elements)
• Interactive and semi-automatic reconstruction
based on relevant profiles (existing architectural
elements)
• Interactive reconstruction based on ancient
iconography (hypothetical architectural elements)
B - MODEL STRUCTURING
• Decomposed in elementary entities, partially
hierarchically organized
C - VISUAL ENRICHMENT
• Texture extraction and projection starting from
photographs finely oriented on the 3D model
(Image-based-modeling)
• Texturing by generic shaders.
D - DATASET STRUCTURE (Geometry)
• Single 3D file structured with 1 level of detail
E - DATASET STRUCTURE (Textures)
• Embedded into the 3D geometry file
• Stored as external 2D files
F - HYPOTHETICAL RECONSTRUCTION
• Basing on the 3D acquisition of existing (or existed)
parts;
• Basing on non-metric iconographic sources of the
studied artifact;
• Basing on iconographic sources (metric and / or
non-metric) related to similar artifacts;
G - SCIENTIFIC (Historic) ADVISING
• Scientific Committee (nomenclature, temporal states
and hypothetical reconstruction)
• Bibliographic references (nomenclature and
hypothetical reconstruction)
3.3.4 Roman archaeological finds (POLIMI)
A - GEOMETRIC RECONSTRUCTION
• Automatic meshing from a dense 3D point cloud
B - MODEL STRUCTURING
• Unstructured, single mesh
C - VISUAL ENRICHMENT
• Texture extraction and projection starting from
photographs finely oriented on the 3D model
(photogrammetry)
D - DATASET STRUCTURE (Geometry)
• Single 3D file structured with 1 level of detail
E - DATASET STRUCTURE (Textures)
• Stored as external 2D images
G - SCIENTIFIC (Historic) ADVISING
• Scientific Committee
20. 3.4 Statistics on the employed approaches, techniques and file formats
This section presents the result of a survey carried out within the partners of the project in order to identify the most used
techniques and formats for the post-processing phase. The max values on the bar charts represent the most commonly used
solutions.
20
3.4.1 Geometric reconstruction
Techniques
Source 3D model format Final 3D model format (for 3D real-time rendering)
21. 21
3.4.2 Visual enrichment
Techniques
Textures - Source 2D images format Textures - Final 2D Images format (for 3D real-time rendering)
3.4.3 Model structuring
Methodology
23. 3.5 Typical post-processing pipelines.
According to the survey, two typical post-processing pipelines were identified determining
the final outputs to be converted into 3D interactive media (see WP5 – publication). These
two general pipelines depend on the specific nature of the initial 3D digitization, on the
artefact morphology (historic building, sculpted detail, furnishing, archaeological find, etc..)
as well as on the adopted model structuring (or segmentation) strategy (see next section).
23
PIPELINE _ TYPE 1
3D digitization and reconstruction: dense point cloud + regular mesh + texture mapping (or vertex
colouring)
Processing: automatic processing + optimization
Model structuring: unstructured
Final 3D representation: the final model is generally managed in Meshlab, Geomagic, ....
This kind of 3D representation can be converted in PLY, OBJ formats in order to be exploited in WebGL-based
3D viewers, such us NEXUS (for dense meshes and point clouds) or Potree (only for dense point clouds). A
low definition version of the 3D model can be embedded into a 3D pdf.
PIPELINE _ TYPE 2
3D digitization and reconstruction: Sparse point cloud (e.g. topographical surveying, technical drawings)+
structured geometry + texture mapping
Processing: semi-automatic and manual processing
Model structuring: generally structured (architectural layout, temporal states, etc.)
Final 3D representation: the final model is generally managed in 3Ds Max, Maya, Blender,…
This kind of 3D representation can be converted in Collada (.dae), OBJ, VRML formats in order to be
interactively visualized by several 3D real-time engines (such as Unity, 3Dvia, etc..). A low definition version
of the 3D model can be embedded into a 3D pdf.
The post-processing phase concerns essentially the elaboration of the 3D representation, at
the appropriate level of geometric and visual accuracy, ready to be converted into the final
publication format (WP5). Given the important geometric and visual detail obtained by the
3D digitization and post-processing techniques, the amount of polygons to be downloaded
and visualised remain impressive, especially within the context of the web publishing.
Without entering into issues concerning the publication (WP5), some techniques for
drastically optimising/decimating the 3D models have been evaluated in order to produce
very light 3D models embedding the geometric detail as a complementary texture (by using
texture backing techniques). Figure 6 shows the results of this supplementary processing
step applied to a column.
24. Figure 6. Results of a 3D model decimation/optimisation based on texture baking techniques:
on the right, the original 3D model composed by 600.000 polygons (file size 140Mb), in the
middle a 3D model (file size 2 Mb) composed by 5000 polygons and integrating an ambient
occlusion + normal + bump maps; on the left a 3D model (file size 1 Mb) composed by 5000
polygons an integrating only an ambient occlusion map.
The results of this processing step can usually be interpreted by the recent WebGL-based
3D engines, as well as by several commercial 3D engines and some partners are currently
testing the integration of these simplification techniques in their 3D modelling pipelines.
4. Producing multiple digital assets starting from a 3D
digitisation
A historic building or an archaeological site is a very complex object requiring a 3D
digitisation strategy according to its morphological, historical and semantic complexity.
Then the post-processing of the acquired 3D data can be considered as a key for
elaborating multiple digital assets according to specific purposes (education, preservation,
dissemination, etc.). In this case, the semantic description of a complex artefact (e.g. by
decomposing an historic building into a set of architectural units) the can be used for
isolating several digital assets from a general structure. For example, a pillar in an
architectural ensemble can be presented as one individual object given the particular
interest of its sculpted iconography or the richness of its details. In this case, in order to
provide a detailed 3D representation (also according to the limitations of a 3D real-time
24
25. web visualisation in terms of number of polygons), it should be isolated from the
surrounding structure and presented as an individual digital asset with its own metadata.
4.1 Criteria for multiplying the digital assets starting from a 3D digitisation
campaign
Accordingly to the recommendations formulated by the reviewers in the 2nd year meeting,
we defined a set of criteria for the identification of individual 3D digital assets.
• Breaking into different parts (e.g. pillar in an architectural ensemble, cloister-gallery
in a monastery, etc.) is possible when the model (or models) of the details (or single
parts of the monument) is (are) relevant from an artistic, architectural,
archaeological perspective.
• Providing different models of the same object is possible when :
o the object itself and alternative hypothesis on its virtual reconstruction;
o different chronologies or phases;
o the existing object (e.g. statues, small findings) before and after the
restoration (this is a case quite frequent, for statutes, vases that are showed
in the museums only after the restoration. One can remove parts and, if you
believe necessary, you could add virtually part(s) to complete the object. One
can also change the texture (e.g. images for Etruscan tombs with painted-walls)
if one wants to deliver models concerning the state of the monument
at the moment of its discovery and now: this could help potential users to
compare the different state of conservation of the monument and to measure
the decay in the course of time in terms of readability of the pictorial cycle,
colour, etc.
• Carrying out the model by means of different technologies in the perspective to
merge the different models: e.g. CAD model of a monument enriched with statues,
frescos or other architectural details stored in another location (e.g. Museum, etc.),
or architectural details, still in situ, of the CAD model acquired by laser scanner, SfM
or other related technologies.
• Delivering models for different users:
o High resolution models for researchers;
o Intermediate resolution models for educational purposes;
o Low resolution for general public.
25
26. 4.2 A case study: the abbey of Saint-Michel de Cuxa
The following schemas illustrate the structure of a final dataset produced by the 3D
digitisation and post-processing of the Abbey of Saint-Michel de Cuxa (south of France).
Hundred of 3D models and thousands of 2D images, plus several complementary media
such as renderings and videos, compose this dataset.
Figure 7. Dataset produced by the 3D digitisation phase. Abbey of Saint-Michel de Cuxa
The 3D digitisation of the Abbey of Saint-Michel de Cuxa is based on several surveying and
3D reconstruction techniques, coupled with a method for the semantic structuring of the
3D model. The first step consisted of a surveying campaign of the abbey, which has been
led with a ToF 3D scanner. The resulting point clouds have been consequently merged into
41 millions coordinates connected each other in the same reference system (see Figure 7:
blue frames). Besides the 3D laser scanning, a survey of more than 2000 photographs was
conducted to collect basic information on the conservation status of surfaces and to render
26
27. the visual appearance of volumes (see Figure 7: purple frames). On the basis of calibration,
orientation, and multi-stereo matching techniques having been tested and developed in the
MAP-CNRS laboratory, certain elements of details (such as the cloister columns) have been
reconstructed with a high level of metric and visual resolution (see Figure 7: purple
frames). With the goal of acquiring information on covers (roofs, etc.) and on the
surrounding of the building, a remote-controlled drone (UAV) carrying a digital reflex
camera was used. We acquired approximately 1000 photographs that have then been
processed to produce an orthophotography of the abbey, by means of a mosaic of high
definition rectified images (see Figure 7: yellow frames).
Figure 8. Dataset produced by the 3D post-processing phase. Abbey of Saint-Michel de Cuxa
3D point clouds acquired by laser scanner and oriented photographs (by photogrammetric
techniques) have been used as a metric and visual support for the stage of 3D geometric
reconstruction. This phase permitted us to represent the morphological complexity of the
building through interactive and semiautomatic modelling procedures (see Figure 8:
27
28. purple frames on the top). In order to represent the richness of the sculpted items
integrated to architectural elements, some procedures for automatic reconstruction from
dense point clouds have been adopted. By exploiting the projective relation established
between 3D point cloud and photographs, the created surfaces were then visually enriched
by textures extraction (see Figure 8: purple frames on the bottom). This allowed
representing the current state of the building with a good level of realism at several levels
of detail. As a result of a calculation of dense matching, the orthophotography obtained
from the UAV acquisition was also used for the generation of a digital terrain model that
has been integrated to the 3D reconstruction of the building (see Figure 8: purple frames
on the bottom). Starting from the gathered data on-site, an archaeological study on the
previous states on the building has been carried out with a scientific committee. Three 3D
models representing the cloister at different temporal states (10th, 11th and 12th century)
have been elaborated (see Figure 8: blue frames).
Figure 9. Complementary 2D media derived from the 3D model. Abbey of Saint-Michel de Cuxa
The combination of all these surveying techniques with complementary geometrical
reconstruction has finally permitted us to create comprehensive representations of the
abbey at different levels of details and at different (current and hypothetical) temporal
28
29. states with a geometric/visual equilibrium suitable for real time 3D visualisation. In
addition to the final 3D representations, several renderings and videos have been produced
in order to present the results of the 3D digitization as well as of the historic study carried
out during the project in a non-interactive (and easy-to-use) way (see Figure 9).
Figure 10. Entire dataset of the 3D digitisation of the Abbey of Saint-Michel de Cuxa
The entire 3D digitisation and post-processing workflow (see figure 10) produced 111 3D
models, 75 images collections (acquired on-site), plus 64 images and 5 videos derived
from the 3D models.
4.3 Multiple representations, multiple resolutions
As presented in the previous case study, from a purely technical point of view, the 3D
digitation and post-processing stages allow structuring several representations of the same
complex artefact. The difference among the multiple representations that can be obtained
(by segmentation/structuring methods) from a 3D digitisation campaign relates to the
29
30. quantity of geometrical information they can contain (also according to the 3D
visualisation limitations – specially in the web publishing context). However, the study of a
complex heritage artefact can use representation techniques exploiting various geometrical
bases. It is, therefore, important to introduce a distinction between the concepts of
resolution and representation: a representation results from the application of a
description technique that allows visualising an individual object (or aspect) according to a
geometrical base and to a resolution level (according to the observation/description
requirements). Then, in order to provide a collection of digital assets for several potential
users (researchers, educators, general public, etc.) each representation should reflect a
specific purpose of potential users interested in studying or discovering heritage artefacts.
The structure of the whole dataset produced by a 3D digitisation campaign can thus be
exploited as a general repository able to deliver digital assets according to specific
documentation and dissemination purposes.
5. Linking the 3D processing pipeline to the metadata creation
Within the framework of the WP4, post-processing activities run in parallel with the
metadata creation. Beside the metadata for Europeana, essentially oriented for resource
discovery, in a larger vision, which raises the digital preservation issues, this project is also
an important opportunity to create documentation on processing pipelines (including
technical and methodological aspects discussed in the section 3 of this report). In this
sense, the work presented in the previous sections could also be used as a grid for
capturing metadata during the post-processing pipeline.
The following figures illustrate the relation between a 3D digitisation process (including
the acquisition, the post-processing and the elaboration of complementary 2D media) and
the metadata creation according to the CARARE2 schema. Starting from the same physical
object (piece of furniture in the Petit Trianon, Versailles), three digital assets are produced
(see figure 11): the first one is a high definition 3D laser scanning of the object (results of
the acquisition phase), the second one is a visually enriched geometric representation
(results of the post-processing), the third one is a video animation (complementary 2D
media) derived from the second 3D model. For each digital asset (see figures 12 to 14) the
employed conceptual model allows describing the relationships between heritage assets,
activities, actors, digital resources.
30
31. Figure 11. Three digital assets derived from the 3D digitisation of a piece of furniture of the Petit Trianon in Versailles
31
32. Figure 12. Relation between the 3D digitisation phase and the metadata creation concerning a piece of furniture of the Petit
Trianon in Versailles
32
33. Figure 13. Relation between the post-processing phase and the metadata creation concerning a piece of furniture of the Petit
Trianon in Versailles
33
34. Figure 14. Relation between the elaboration of complementary 2D media phase and the metadata creation concerning a piece of
furniture of the Petit Trianon in Versailles
34
35. 35
6. Conclusions
Post-processing is a highly significant part of the 3D-ICONS pipeline in terms of the effort
required and the impact on the resulting 3D-models. The work of WP4 has resulted in:
• Classification of the elementary processing steps composing the typical post-processing
approaches;
• Identification of two typical pipelines with the aim to pre-orient the processing of
the data through a specific final output (for web publishing purposes);
• Determining criteria for multiplying digital assets starting from a 3D digitisation
campaign.
• Providing an initial grid for describing post-processing activities when creating
metadata.
Besides the presented methodological and technical issues (also including the solutions
applied), the management and the long term preservation of high quantities of data (raw
data, visually-enriched 3D geometric representations, complementary 2D media, metadata,
etc.) forms an important issue within the framework of the project. Solutions for setting-up
several shared data repositories are currently being tested and evaluated within the
project consortium.