Mars Rover Image-based Localization using 3D Point Cloud Models for Mobile Field Reporting Applications Mani Golpavar-Fard, Assistant Professor, University of Illinois and Rayan Jreije, Manager 3D Software Development, CCC The method which is marker-less and infrastructure-independent, leverages simple digital images taken on a commodity smartphone or a tablet, identifies the location and orientation of the user in 3D, and provides access to project information and Building Information Model (BIM) without the need for storing any of the project information (including BIM) on the mobile device

1. Mars Rover Image-based Localization using
3D Point Cloud Models for Mobile Field
Reporting Applications
Stuart Young - Fiatech

2. MARS ROVER:
Image-based Localization via 3D Point Clouds
Project Managers
Rayan Jreije (CCC)
Mani Golparvar-Fard (University of Illinois)
Steering Committee Members
Todd Sutton (Zachry Construction)
Stuart Young (FIATECH)

3. • “On-demand” and “mobile” access to project information
has potential for improving decision-making at site.
• Challenges…. occasionally connected environments which
are a typical for most projects.
Business Problem

4. • Leveraging existing point clouds (image-based, laser
scanning, and RGBD)
Opportunity
Digital Images captured from
different locations and angles
Digital
Images
Photogrammetric
algorithms
generate 3D Point
clouds out of the
captured images.
3D Point
Cloud

5. 5
1. Imagine you are conducting a site walk through for
field reporting purposes
2. You can now use your smartphone to take a photo and
access the most updated project information

6. 6
Specification A-54 i
Finishing Works for Short Interior Wall
50% complete
 Electrical cable trays installed.
3. You can also query project information and perform
content-authoring (site reporting) in 3D

7. 7
1. Imagine you are conducting facility management
routine inspection or responding to a service request
2. You can now use your smartphone to take a photo and
access specifications or warranties from BIM

8. 8

9. Objectives
• Conduct a pilot project to apply and validate the underlying
visual-based localization technique (MARS Rover)

10.  Support field reporting
 Associate project information
 High precision association in 2-5 sec
 Cloud-based
 With and/or without BIM
Requirements

11. A. Tracked 3D Location & Orientation
B. Provide Query Capabilities
C. Access/Share
Key Components

12. Phase I. Setup
Phase II. Tech Dev
Phase III. Job Site Implement
Phase IV. Completion Report
Project Execution

16. Rayan Jreije
Manager 3D and Visual Project Controls - CCC
The MARS-ROVER drive:
an EPC contractors Use
case

17. Digitization of as built models: Laser
Scanning
• Laser scanning solutions
in EPC industry reached
“production plateau”
• Point clouds became the
new currency

18. The Digital Imaging Alternative
– Building 3D Point Clouds from images
(Photogrammetry)
– Massive Image collection (Drones)
– Drones to Roam areas with no GPS coverage like
tunnels and undergrounds (Image Based
Localization)
– Enhanced Drones capabilities for obstacle avoidance
(Robotics)

19. Project Managers
Rayan Jreije (CCC)
Mani Golparvar-Fard (University of Illinois)
Acknowledgement: Financial support from CCC (50%)
Currently recruiting members
Interested to join? email young@fiatech.org