L'intervento di Marco Di Benedetto in occasione del workshop "Ruju - Una piattaforma per la ricostruzione 3D", che si è svolto a Nuoro il 23 novembre 2015. L'evento è stato organizzato dal Progetto Cluster Elettronica di Sardegna Ricerche.

1. Nuoro, 23 Novembre 2015
Marco Di Benedetto, Ph.D., Transform and Lighting S.r.l. Co-Founder
Spazio per logo
ente relatore
Organizzato dal Cluster Elettronica e dalla Transform and Lighting S.r.l.
Ruju
Una piattaforma estendibile per la ricostruzione 3D
REPUBBLICA ITALIANA
La piattaforma Ruju per la ricostruzione 3D di scene
e oggetti reali a partire da foto o scansioni
Nuoro, 23 Novembre 2015

2. REPUBBLICA ITALIANA
Transform and Lighting S.r.l.
• Founded in 2013 in Pisa by Computer Graphics (CG) researchers (at CNR)
• Specialized in:
• 3D reconstruction
• real-time rendering
• parallel and out-of-core processing
• augmented reality
• interactive applications
• Proprietary software and consultancy
• TnL for short
2

3. REPUBBLICA ITALIANA
TnL – Main Customers
• Anyone in need for enriching their business by exploiting 3D scenes
• Navionics S.p.A.: reconstruction of Earth surface from non-homogeneous
terrestrial and marine data
• rdGizmo ForYou Ltd.: 3D printing software
• Anyone providing interactive and multi-platform software
• Quipu S.r.l.: cardiovascular analysis application
3

4. REPUBBLICA ITALIANA
Project Mission
4
Sardegna Ricerche, Bando di Gara Cod.155_14, 2014
«To develop a client-server, multiuser, and extendible software platform
to create 3D models from scans and images. »

5. REPUBBLICA ITALIANA
Why Ruju ?
5

6. REPUBBLICA ITALIANA
Two Perspectives
User
• Interested creating 3D content
from range maps or photographs
• Technical details should be as
transparent as possible
• The software must the the way
not the goal!
6
Developer
• Wants to create 3D reconstruction
software (customers / internal)
• Use end extend the API to address
custom needs
• The software must the the way
not the goal!
Intuitive & Easy to Use Small Building Blocks

7. REPUBBLICA ITALIANA
The User Perspective
• Ruju presents to the user as aWebAplication
• No installation, a web browser is the only requirement
• Just like any interactive website
• An intuitive GraphicalUser Interface (GUI) drives the user through the
logical data flow to create a 3D model
• Two Data Pipelines
• 3D Scan
• Photographs
7

8. REPUBBLICA ITALIANA
The 3D Scan Pipeline: Data
8

9. REPUBBLICA ITALIANA
The 3D Scan Pipeline: Goal
9

10. REPUBBLICA ITALIANA
The 3D Scan Pipeline
• Multiple Range Maps or Point Clouds are what you begin with
• From 3D Scanners (Triangulation,Time-of-Flight, Structured Light, ...)
• FromComputer Simulations
• From already existing data set
• Think of them as patches of a puzzle
• They must be aligned...
• ... then fused into a single 3D object (scene)
10

11. REPUBBLICA ITALIANA
The Image Pipeline
• Multiple Images (i.e., photographs) are what you begin with
• From medium / high resolution cameras
• From already existing photo sets
• Think of them as multiple views of a static scene
• Cameras must be registered...
• ... then pixels must be triangulated to create a 3D point cloud
11

12. REPUBBLICA ITALIANA
Ground Control Points
• Reconstruction from
image has 7 Degrees of
Freedom (DoF)
No Georeferenziation
 No Scale!
12

13. REPUBBLICA ITALIANA
Ground Control Points
13

14. REPUBBLICA ITALIANA
GPC: Georeferentiation
14

15. REPUBBLICA ITALIANA
UserWorkspace
Logical Data Flow
1. Login and create a project
2. Populate the project with files
(pc, rm, img)
3. Execute commands on items to
produce metadata
(e.g., alignment 3D transformations)
4. Use files and metadata to produce
one or more 3D reconstructions
(point clouds & triangle meshes)
15
Authentication
Projects
Commands Reconstruction
Files

16. REPUBBLICA ITALIANA
Meet Ruju
16

17. REPUBBLICA ITALIANA
Ruju for Users
• Allows to reconstruct from 3D scans and photographs
• Web Application
• Intuitive Interface
• Works on many Platforms
17

18. REPUBBLICA ITALIANA
Two Perspectives
User
• Interested creating 3D content
from range maps or photographs
• Technical details should be as
transparent as possible
• The software must the the way
not the goal!
18
Developer
• Wants to create 3D reconstruction
software (customers / internal)
• Use end extend the API to address
custom needs
• The software must the the way
not the goal!
Intuitive & Easy to Use Small Building Blocks

19. REPUBBLICA ITALIANA
The Developer Perspective
• Ruju exposes state-of-the-art algorithms for 3D reconstruction
• Very efficient implementations
• Multiplatform
• Esists as a core library and a constellation of tools built on top
• Open Standards
19

20. REPUBBLICA ITALIANA
The Platform
• A software pipeline for 3D reconstruction
• Point clouds (pc)
• Range maps (rm)
• Images (img)
• Client-Server architecture
• HTTP server with native tools
• HTML5 client as standard website
• ExtendibleAPI and GUI
• Custom and open source processing software
• Robust web frameworks and libraries
20

21. REPUBBLICA ITALIANA
Server (CI)
System Architecture
• Client-Server model
• Standard protocols and widely-used open source software
• Ad-hoc software for 3D data processing and reconstruction
• Server: handles users, storage, and operations
• HTTP server with dynamic page generation with Code Igniter (PHP)
• Authentication module
• DBMS (MySql) and file system as storage
• Executable binaries to perform commands (reconstructions etc.)
• Client: operative and monitoring interface
• Standard HTML5 web pages
 any modern browser will do the job
• UsesThree.js (WebGL) for 3D rendering
21
StorageAuth
Exe Commands
Client
HTML5
Three.js
(WebGL)
NETWORK

22. REPUBBLICA ITALIANA
Third-Party Software included in Ruju
• Server:Web
• Apache HTTP server, PHP, MySQL, CodeIgniter, ...
• Server: Processing
• Implemented in C++ (efficiency & sw availability)
• Qt, Point Cloud Library, Eigen, Sparse ICP, Assimp, ...
• Client
• Will run on all modern HTML5 Web Browsers
(Google Chrome, Mozilla Firefox, ...)
• Bootstrap,Three.js (WebGL), ...
22

23. REPUBBLICA ITALIANA
Web Application
• CodeIgniter with Model-View-Controller paradigm
• Model: data management (mosty db access)
• View: how the internal state of the application is presented (i.e. html)
• Controller: application state management and flow control
• RESTful API
• <base_url>/sayHello/marco/19
function sayHello($name, $age)
23

24. REPUBBLICA ITALIANA
Data
• Point clouds, range maps, and images on file system
• Large & highly involved in computations, avoid round trip from/to db
• Metadata (IDs, annotations, transformations, ...) on database
• Lightweight, easily accessed
• Backup solutions must take fs and db into account
24

25. REPUBBLICA ITALIANA
Commands and the Ruju API
• Commands are small executable binaries that invoke a single
function of the core ruju library
• Input & output parameters as JSON files
• Operate on data
• Single shot (stateless)
• A typical command API class has three methods
• Core library (types of arguments are internal)
• JSON marshalled (arguments as dictionaries of variants)
• Main (argc, argv, directly invokable by main())
25

26. REPUBBLICA ITALIANA
Ruju for Developers
• Allows to build web and standalone native applications
for 3D reconstruction
• State-of-the-Art Algorithms
• Documented Code
• Multiplatform
26

27. REPUBBLICA ITALIANA
Summary:WhatWe Have
Requests
• Client-Server Architecture
• Run onWeb Browsers
• Multiuser
• 3D Reconstruction from Scan Data
• Import/Export
• Alignment
• Fusion
• 3D Reconstruction from Photographs
• Import/Export
• Point Cloud Generation
• BasicGeometric Measurements
27
OurTechnicalOffer (additions)
• RealTimeViewer
• Advanced Measurements
• Minimum Requirements on Steroids
• Multiuser System
• Data Size
Cluster Needs (more additions)
• Asynchronous Jobs Queue
• Ground Control Points, Rescaling Utilities
Our Enhancements (even more additions)
• Visualization Enhancements
• Email Notification,Administration, ...

28. REPUBBLICA ITALIANA
Ruju :What Else?
• The platform represents the solid foundations a big system, with some stores built on top
• New features and utilities can be easily added
• Many features are domain specific
• Texture
• Photo Projection
• Color Correction
• More advanced geometric measurements
• Filtering (smoothing, noise removal, general cleaning, ...)
• ...
• MultiresolutionVisualization
• YOU tell whatYOU need!
• Ruju Forum!
28

29. REPUBBLICA ITALIANA
ThankYou!
29

30. REPUBBLICA ITALIANA
30
EOF