Drone flight planning and processing

1. Drone Flight Planning:
Principles and Practices
(Part 2 of 2)
Dany Laksono
January 17th, 2019

2. Drone Flight Planning: Principles and Practice 2
• What could be produced from UAV Photos: A Review
• Principles of 3D Reconstruction
• How to: Positioning the camera
• What options (read: softwares) do we have?
• Limitations and other notes

3. What could be produced from UAV
Photos?
Orthophoto Mosaic Point Cloud/3D Models Digital Elevation (DEM)
Yuwono, 2018

4. Everyone* could fly a
drone
The question is:
how do you produce
meaningful results?
* With adequate trainings

5. We need to make sure that the images overlapped
with each other
Only areas with overlapping
images would produce 3D
(it’s called “stereoview”)
Through a process called
“photogrammetry” or “Structure
from Motion”, 3D geometry of
objects in overlapping area will
be produced

6. The What
So what is
Photogrammetry?
In short, photogrammetry is:
A field of engineering science and art which aim is to
produce 3D geometry from series of images
The only requirement is that each photo overlap each
other in a way such that:
a. Position and orientation of camera during
exposure is known
b. 3D geometry of objects is known
Topographic Photogrammetry: using planes, UAV, etc
Non-Topographic Photogrammetry: capturing
buildings, monuments, etc

7. Structure from Motion (SfM) Photogrammetry
Image from theiaSFM
1. Feature Detection
2. Feature Matching
3. Pose Estimation and
Adjustment
4. Dense Reconstruction
5. Meshing
6. Texture Mapping
SfM Algorithm Sequence:

8. Feature Detection and Matching
“Create a database of keypoints in one image and find the most probable
match in the other image”
Image @OpenCV

9. Sparse Reconstruction
Feature detection, matching and initial bundle adjustment resulted in
estimated camera position and 3D geometry of Objects
Known:
Estimation of object’s
3D geometry
and
Camera Position and
Orientation

10. Dense Reconstruction
Denser Point-cloud,
based on interpolated
(or extrapolated)
sparse point cloud
The color in Colorized
Point cloud are
obtained from image
RGB

11. Multiview Stereo: Meshing and Texturing
“Mesh” is
obtained from
further
interpolation of
dense point cloud
Texture are obtained by ‘draping’
photo into the 3D Model based on
model’s Normal Linehttp://www.gris.tu-darmstadt.de/projects/mvs-texturing/

12. To conclude: Deliverables
of SfM Software
Sorted by the order of
processing, the result of SfM
and Multiview Stereo are :
a) Sparse Point-Cloud
b) Dense Point Cloud
c) Colorized Dense Point
Cloud
d) Mesh Surface
e) Textured Surface

13. DEM or Digital
Elevation
Model are
obtained from
interpolated
Dense Cloud or
3D Mesh
Other Deliverables
For Mapping purpose, DEM and Orthophoto Mosaic might be
more desireable

14. DEM or Digital
Elevation
Model are
obtained from
interpolated
Dense Cloud or
3D Mesh
Other Deliverables
An Orthophoto Mosaic is an orthographic projection imagery
(“Top-Down looking” camera)
Orthophoto is
a map ready
production,
showing
elevation-
corrected
photo instead
of just Photo
Mosaic

15. Remember: Photogrammetry Produces DSM instead of DTM
General Rule:
“What the
camera could
see, could be
produced as
3D”
Image from charim.net

16. SfM opens up new possibilities for 3D reconstruction
http://www.cs.cornell.edu/~snavely/bundler/

17. SfM opens up new possibilities for 3D reconstruction

18. SfM opens up new possibilities for 3D reconstruction
City-Scale Reconstruction using Unordered Images

19. SfM opens up new possibilities for 3D reconstruction
Indoor Mapping
& Localization

20. SfM opens up new possibilities for 3D reconstruction
Cloud-based 3D Reconstruction

21. SfM opens up new possibilities for 3D reconstruction
Model 3D from handheld

22. SfM opens up new possibilities for 3D reconstruction
Model 3D from handheld
https://www.youtube.com/watch?v=36PFT6SkYMI

23. But it also has Limitations..

24. Since SfM relies on Feature
Detection, it is prone to error on
feature with High Similarity

25. Limitations of SfM-MVS
Group of images
capturing eastern wing
of UGM building
Group of images
capturing western wing
of UGM building
Connectivity Graph
showing clustered
images, where it
shouldn’t be connected
at all

26. Limitations of SfM-MVS: Non-Lambertian Surface, Weak
Surface, Thin Structure

27. In Agisoft, SfM
workflow is defined in
one single menu

28. Big Players in 3D Reconstruction
.. Alternative to Commercial Software
• WebODM (https://www.opendronemap.org/webodm/download/)
• OpenMVG (https://github.com/openMVG/openMVG/)
• COLMAP (https://colmap.github.io/)
• Visual SFM/Bundler (http://ccwu.me/vsfm/)
• Apero Micmac (http://logiciels.ign.fr/?Micmac)
• TheiaSFM (http://www.theia-sfm.org/)
• Kitware MapTK (https://github.com/Kitware/maptk)

29. Big Players in 3D Reconstruction
.. Alternative to Commercial Software
• Regard3D (http://www.regard3d.org/)
• AliceVision Meshroom (https://alicevision.github.io/)
• Python Photogrammetry Toolbox (http://184.106.205.13/arcteam/ppt.php)
• Sf3M (http://sf3mapp.csic.es/)
• SFM Toolkit (http://www.visual-experiments.com/demos/sfmtoolkit/)

30. FOSS SfM-MVS
openmvg.readthedocs.org
• Perangkat lunak SfM gratis
• Mengintegrasikan feature detection, feature matching, dan
bundle adjustment
• (optional) menggunakan PMVS/CMVS untuk dense
reconstruction serta SfM Georef untuk Georeferencing
VisualSfM

31. FOSS SfM-MVS
openmvg.readthedocs.org
VisualSfM

32. FOSS SfM-MVS
openmvg.readthedocs.org
OpenMVG
• Perangkat lunak SfM opensource berbasis Linux
• Menggunakan Algoritma Incremental SfM dan Global SfM
• Memiliki fungsi Georeference yang terintegrasi
• Tersedia GUI untuk Windows: Regard3D

33. FOSS SfM-MVS
openmvg.readthedocs.org
Regard3D (OpenMVG GUI for Windows)

34. FOSS SfM-MVS
openmvg.readthedocs.org
COLMAP
Reconstruction of central Rome using 21K photos produced by COLMAP’s SfM pipeline.
Dense reconstruction of several landmarks produced by COLMAP’s MVS pipeline.

35. FOSS SfM-MVS
openmvg.readthedocs.org
SURE Photogrammetry
Point Cloud, 3D Model & orthophoto

40. TERIMA KASIH