Live Drone Map UAV Mapping Solution

L o g o
라이브드론맵 (Live Drone Map)
– 실시간 드론 매핑 솔루션
May 20, 2017
서울시립대학교 공간정보공학과 센서및모델링연구실
도시과학연구원 대도시무인이동체연구센터
천장우, 함상우, 최경아, 이임평*
OSGeo한국어지부 드론세미나

L o g o
Contents
Intro. to UAV Mapping1
Live Drone Map2
4
2

L o g o
3
Intro. to UAV Mapping1

L o g o
What is a UAV?
 UAVs: Unmanned Aerial Vehicles
 “UAVs are to be understood as uninhabited and reusable
motorized aerial vehicles” (Blyenburg, 1999). These vehicles
are remotely controlled, semi-autonomous, autonomous, or
have a combination of these capabilities.
 Main communities
 Military, Artificial Intelligence, Computer Vision, Robotics,
Aeronautics, …
 Geomatics (Photogrammetry, Remote Sensing and Surveying)
4

L o g o
UAV Mapping / Photogrammetry
 UAV mapping / photogrammetry
 opens various new applications in the close range domain,
combining aerial and terrestrial photogrammetry, but also
introduces low-cost alternatives to the classical manned aerial
photogrammtery.
 In the context of mapping
 Geospatial data collection with high geometric and temporal
resolution
(large scale data)
 Generation of elevation models, orthophotos, maps, 3D
models etc.
5

L o g o
Motivation for the Use of UAVs
 Advantages of UAVs
 Use in high risk situations and inaccessible areas
 Data acquisition with high temporal and spatial resolution
 Autonomous and stabilized
 Low-cost
 Limitations in the use of UAVs
 Limitations of the payload
 Regulations and insurance
 Use of Low-cost Sensors
6

L o g o
The accuracy of measurement methods in relation to the object/area size. Modified from
Fig. 1.4 in Luhmann, et al., 2006.
UAV Mapping vs. Others
7

L o g o
UAV Market Status and Prospects
 Commercial drone market is growing due to
increasing demand of private sector
 World commercial drone market size: Predicted to
grow up to 4 times larger (2016-2025)
 Leisure: 2.2 Billion$ → 3.9 Billion$ - Large but saturated
 Public: 36 Million$ → 464 Million$ – Small but growing fast
 Commercial: 390 Million$ → 6.5 Billion$ – Biggest soon
8

L o g o
UAV Platform_DJI
 DJI Drones
 DJI’s products are intended for amateur as well as professional
use
 Phantom series are the most popular product, and since
launch, have evolved to integrated flight programming with a
camera, WiFi connectivity, and the pilot’s mobile device.
9

L o g o
UAV Platform_Falcon8
 Developed by Intel company
 Automatic sensor data verification
 3 flight modes for any circumstances
 GPS mode, altitude mode, manual mode
 Weight & Payload : 2.3kg / 0.8kg
 Flight time : 16 min
10

L o g o
UAV Platform_Indago
 Developed by Lockheed martin company
 Flight time : 45 min (Maximum)
 Sensor : Optical and infrared ray camera
 Apply for Lifesave Project; find people who are
dementia patient
11

L o g o
UAV Platform_Skyranger
 Developed by Aeryon company
 Sustain extreme weather condition
 Tolerate temperatures from 22 below zero to 122
degrees Faherenheit
 Maneuver with wind gusts up to 55 miles per hour
 Sensor HD camera(with infrared filters in low-light)
12

L o g o
UAV Platform_Sensefly
 Developed by Parrot company
 Cover up to 12km2 in a single automated flight
 Weight : 500g
 Sensors : image sensor, u-BLOX GPS chip, attitude
sensor, radio transmitter, autopilot circuit board
 Flight time : 30 min
13

L o g o
Commercial Software
Software PhotoScan PhotoModeler Pix4D EnsoMOSAIC
Platform Linux, OS X,
Microsoft
Windows
Microsoft
Windows
Microsoft
Windows,
MacOS, Cloud
Microsoft
Windows
Automatic Yes Yes Yes Yes
Scalability Multiple
images
Multiple
images
Multiple
images
Multiple
images
Data source images images images images
Price $179~3499 $1145 $2000 $900
Online Service - - - -
14

L o g o
Opensource Software
Software MicMac OpenDroneM
ap
openMVG DroneMapper
Platform Linux, OS X,
Microsoft
Windows
Linux, OS X,
Microsoft
Windows
Linux, OS X,
Microsoft
Windows
Web-Based
Automatic Semi-
automatic
Yes Yes Yes
Scalability Multiple
images
Multiple
images
Multiple
images
Multiple
images
Data source images images images images
Price free free free free
Online Service Yes - - -
15

L o g o
Typical Products from UAV Mapping
16

L o g o
DEM (Digital Elevation Model)
17
 DEM(Digital Elevation Model)
 is a digital 3D model created from
terrain elevation data.
 can be represented as a grid or TIN
 is often used as a generic term for
DSMs and DTMs.
 DSM(Digital Surface Model)
 represents the earth's surface and
includes all objects on it.
 DTM(Digital Terrain Model)
 represents the bare ground surface
without any objects like plants and
buildings.
GRID
TIN

L o g o
Orthoimage
18
 Aerial images geometrically corrected (orthorectified)
such that the scale is uniform
 adjusted for relief displacement, lens distortion, and camera
tilt.
 overlapped with maps and used to measure true distances.
 commonly used in GIS as a "map" background image.
(Row, Column, Color)
(Northing, Easting, Color)
+ Height

L o g o
3D Model
 Represents terrain surfaces, sites, buildings, vegetation,
infrastructure and landscape elements as well as
related objects belonging to interested of areas.
19
 Supports presentation,
exploration, analysis, and
management tasks in a large
number of different
application domains.
 Allows "for visually
integrating heterogeneous
geoinformation within a
single framework”.

L o g o
Data Processing
20
Geo-
referncing
Dense
Matching
Meshing Texturing
Textured
3D
Model
EOPs
IOPs
Point
Clouds
Griding &
Interpolation
DSM
Rectification
Ortho-
image

L o g o
21
Live Drone Map2
UAV Based Rapid Mapping & Sharing Solution

L o g o
Background
 UN has performed field operations such as
 Peace keeping to monitor peace agreements
 SAR(Search and Rescue) and restoring in disaster areas
 Needs a GIS customized to a specific operation
 The DPKO (Department of Peacekeeping Operation) develops
and implements field activities strategy based on the GIS.
 The GIS should be established rapidly and applied to field
operations immediately.
 The UNGSC (UN Global Service Centre) provides the mission
specific and adaptive GIS to the DPKO.
 They used to employ existing sensory data such as satellite
images.
22

L o g o
Objectives
 Usually, these existing images are out-of-date and
retain low resolution.
 The mission areas may not have the existing data and
acquiring data newly takes time (4-6 weeks).
 UAV mapping systems are very expeditious means to
acquire high resolution data on the mission areas.
 We propose an automatic mapping system based on a
UAV, which can operate in a fully automatic way from
the data acquisition to the data processing.
23

L o g o
Role of UAV mapping system
 The system is to provide the updated geo-spatial data
of a target area in a rapid and automatic way using a
UAV based multi-sensor system.
 Input: the region of interests and the output options (types)
with the resolutions (1 cm~1 m)
 Output: geo-spatial data such as orthoimage, DSM with a
user-specified resolution
24

L o g o
Key Advantages – Rapid & Automatic
 A user just specifies the region of interests on a map
and the output options (types) with the required
resolutions.
 The final product will be generated within less than
several hours (TBD) after the data acquisition.
 All the remaining processes from the data acquisition
through data processing to final product generation
can be performed automatically.
 The user does NOT require expert knowledge about
UAV flight planning and operation or
photogrammetric data processing.
25

L o g o
Concept of Operations
2. Data Acquisition
5. Geo-data Generation
3. Data Transfer
26
1. Preparation
4. Georeferencing6. Delivery

L o g o
Implementation Strategy
 Plug-and-Play
 We will develop a fully integrated multi-sensor payload
system by minimizing its dependency on a UAV platform.
 The system does not heavily depends on a certain UAV
platform.
 If certain requirements (max. payload weight) are met, we can
easily plug the payload system into the UAV and play its roles.
 Hierarchical Modular Design
 The system will be configured with a hierarchical modular
design approach.
 Each module can be easily replaced for upgrade or
customization.
27

L o g o
Implementation Strategy
 Low Cost / Open Source
 Affordable commercial or low cost Open Source HW and SW
can be employed.
 Hardware :
• UAV platforms: 3DR, DJI, DIY Kits, etc.
• Sensor control & integration: Arduino, Raspberry Pi
 Software : Customized from Open-Source
• Flight control & planning
• Photogrammetric Data processing: Photoscan / OpenDroneMap
28

L o g o
UN Geo-Cloud Arch.- Data Collection
29
Proposed by Prof. K. Lee, Oct/01/2015
Secure and High Performance Storage
Data Collector
Applications
Metadata Imagery Data
Geo-Portal
Server
Vector Data Feature Data
1. UAV Collector
3. Map
Generalization
Field Collector
(Mil. Observer)
Trajectory
Collector
Geo-MM
Collector
2. Crowdsourcing Collector
Processing
Server
Feature
ServerImage
Server
GeoMM
Server
4. 3D
Collector

L o g o
Configuration of UAV Collector
Flight Planning Data Quality Check
Georeferencing Orthorectification
30
UAV Mapping System
UAV Platform
SensorsSensor Support

L o g o
Data Acquisition System
31

L o g o
Platforms
32
UAV(TAM GD 14000) UAV(D5) UAV (Q-TILT)
Model
A: Current Version
B: Upgrade by 2017
Spec.
Flight time
~60 minutes
(Average 35 minutes)
Flight time ~30min Flight time
A: 30 minutes
B: >60 minutes
Maximum
Payload
6,000g
Maximum
Payload
10kg Maximum Payload
A: 1,000g
B: > 2,000g
Speed
Rate of climb : 7.5 m/s
Cruising speed : 12.0
m/s
Speed 18m/s Speed
Rate of climb : 7.0
m/s
Cruising speed :
25.0 m/s
Operating
radius
1,000 m Wind limit 15m/s
Operating
radius
A: 5,000 m
B: >15,000 m
Weight 8,650g Weight 35kg Weight
A: 12,000 g
B: 20,000 g
Dimensions
140cm X 140cm X
70cm
Battery Size 44,000mAh / 44V Dimensions
140cm X 140cm X
70cm
Temperatur
e
-10°C ~ 50°C Temperature -10°C ~ 50°C

L o g o
Q-Tilt (Fixed Wing & VTOL)
33

L o g o
Sensors
34
Digital Camera (& Lens) GPS/INS Lidar
Model
Sony A7 II SONY Sonnar T*
FE 35mm F2.8 ZA
APX-15
(Applanix single board GNSS-Inertial solution)
Velodyne
Spec.
Resolutions 6000x4000
- SBAS
- Support PPS time synchronization
16 channel
Pixel size 5.97um Operating Temp 40° ~ 75° Measurement
Range : ~100m
ISO 100~25600 Dimensions 67 x 60 x 15mm
Maximum Shutter
speed
1/8000sec Weight 60g ±3cm Accuracy
Weight(Camera) 599g Accuracy SPS DGPS RTK4
Post-
Processe
d5 FOV : ±15
Degree (Vertical)
360 Degree
(Horizontal)
Focal length 35mm Position(m) 1.5-3.0 0.5-2.0 0.02-0.05 0.02-0.05
Size 61.5x36.5mm
Velocity(m/s) 0.05 0.05 0.02 0.015
Roll&Pitch(deg) 0.04 0.03 0.03 0.025
Wavelenth :
903nm
Weight(Lens) 120g True Heading3(deg) 0.30 0.28 0.18 0.080
Single and Dual
Returns

L o g o
Sensor Control & Data Transmission
 To control sensors, collect the sensory data (images)
with time tags and transmit them to the ground.
 Based on open source HW such as Latte (single board
computer) & long-range WiFi
 Time accuracy: < 10ms
35
Latte (single board computer)
Long-range WiFi

L o g o
Data Processing SW
36
 Produce individual orthoimages and mosaics of
mission areas automatically and rapidly
PA Data
Data
Processing
SW
Individually
Geo-rectified
Images
Images
Ref.
Data
Mosaic
Orthoimage
DSM
Rapidly &
Automatically
UAV Pos/Att from GPS/INS
from Cameras
Existing Geo-data
(GCPs, DEMs, Ortho-images)
Real-time Data Processing
in several seconds after each image acquisition
Rapid Post-Processing
in several hours after a flight mission

L o g o
SW at GitHub
 Real-time Data Processing
 A subset for Live Drone Map
 Download sensory data
 Process them to generate individual orthoimage
 Upload the orthoimage to a cloud server (Mago3D)
 https://github.com/flyhamsw/UN_symposium_v1.2
 Rapid Post-processing
 Georeferencing SW
 https://github.com/flyhamsw/Georeferencing
 Ortho-mosaic Generation SW
 https://github.com/flyhamsw/Orthophoto-generation
37

L o g o
Real-time Data Processing
38

L o g o
Rapid Post-Processing
39

L o g o
Live Drone Map
40
 UAV Based Real-time Mapping & Sharing Solution
 We combine the real-time mapping solution (UAV
collector) with a cloud based geo-data sharing
solution (Mago3D).
 Users in distant areas with Internet can access the
image maps updated in real-time using UAV systems
during its flight.
 The image maps are visualized in 2D/3D with existing
geo-data without any plug-in software through a
standard web-browser on a desktop computer or even
mobile devices.
 Smartphone images can be uploaded and shared.

L o g o
Visualization in Live Drone Map
41
3D on a Desktop Computer 2D on a Smartphone

L o g o
Applications of Live Drone Map
42
2. Real-time Data Collection 4. All-source Situation Room
1. Disaster
Occurance
UAV + Sensor payload(camera+GPS/INS)
Real-time transmission
of images and Pos./Att.
Real-time transmission
of imagery maps
3. Real-time Mapping

L o g o
1st Demonstration in Korea
43
 In Seoul on Nov. 9, 2016.
 3rd International Symposium
 Partnership for Technology in Peacekeeping
 Information and Communications Technology Division
 United Nations Department of Field Support
 Indoor demonstration
 inside Seoul City Hall

L o g o
1st Demonstration in Korea
44

L o g o
2nd Demonstration in Korea
45
 In Seoul on Feb. 21, 2017.

L o g o
2nd Demonstration in Korea
 Digital Times
 https://www.youtube.com/watch?v=yTrdPNbn_LY&feature=you
tu.be
 YTN news
 http://m.ytn.co.kr/news_view.php?key=201702212159482174&s
_mcd=0102
46

L o g o
 Integrated with MCC on Apr. 27, 2017.
3rd Demonstration at UN GSC in Italy
47
Mission Area
(GIS team/patrol)
MCC
Mission HQ
(Commander)
San Pancrazio Airfield
UN GCS, Brindisi
New York 40 km
> 10 hour via flight

L o g o
3rd Demonstration: Configuration
48
Reception
& Archiving
Processing
Acquisition &
Transmission
Geo-rectified
Image
Sensory
Data
UAV Collector
Updating
(Geo-portal)
Delivery /
Visualization
(Users)
Mago3D
Smartphone
Crowd sourcing
MCC

L o g o
3rd Demonstration : UAV Air Field
49

L o g o
50

L o g o
51

L o g o
3rd Demonstration : MCC
52

L o g o
3rd Demonstration : Day 1 (Apr. 20)
53
 시험, 시연, 교육, 워크숍 등 세부 일정 협의
 무인기 비행허가 및 장비 배송 확인
 시연 시나리오 확인
 라이브 드론맵 개발과정, 무인기 매핑 워크숍 일정 확인
 Air Field로 이동하여 실험 진행
 지상 SW 실험 : 기존 데이터를 저장소에 전송하여 영상처리를
진행하고 지오포털 상에 가시화 되는 것을 확인
 무인기 비행을 제외하고 실내에서 센서 탑재체의 데이터가 WiFi
통신을 통하여 저장소로 전송되는 것 확인
 문제점 확인
 이탈리아 현지의 통신 상태가 불안정하여 핸드폰 테더링을 통해
LTE 통신 이용

L o g o
54

L o g o
55
 지상 SW 점검
 테더링 스마트폰을 이용하여 네트워크 설정
 GSD를 변경하며 업로드 시간 체크
 센서 탑재체 실외시험 진행
 긴 멀티텝을 사용하여 실외에서 센서 데이터가 저장소로 전송되
는 것을 확인
 MCC와의 협의
 MCC : 통신과 전력이 고립된 지역에서도 운영이 가능하도록 설
계된 컨테이너 박스 형태의 시스템
 MCC 내부에 있는 컴퓨터를 이용하여 클라우드 서버 접속 후 갱
신되는 지도 확인
 GIS 부서 견학

L o g o
56

L o g o
57
 무인기 비행 테스트 진행(Apr. 24)
 이착륙 및 1차 수동 비행 시험
 Waypoint 경로 설정 후 2차 자율 비행 시험
 탑재체 장착 후 3차 자율 비행 시험
 기존 데이터를 저장소에 전송하여 영상처리를 진행하고 지오포
털 상에 가시화 되는 것을 확인
 워크숍 진행
 사진측량 및 무인기 매핑 사례 소개
 라이브 드론맵의 개발과정 및 구성과 동작원리

L o g o
58

L o g o
59
 1차 통합 비행 시험
 AR Works에서 보유한 무인기 2대 비행시험
• 이탈리아 현지 바람이 너무 강하게 불어 크기가 작은 무인기를 이용
하여 시연하기로 결정
 Apx data의 문제점 확인 : 보드 문제
 MCC와의 통신 확인
 UN 실무자 견학
 무인기 매핑 실습 진행
 WebODM 설치 및 실행과정 설명
 Client 서버 상에 설치된 WebODM에 데이터를 전송하여 영상
처리를 진행하고 결과(3D point cloud, 정사영상) 확인

L o g o
60

L o g o
61
 지상 SW와 MCC 통합 시험
 MCC 인터넷망(위성 기반)의 속도를 고려하여 2D 환경에서 4초
마다 영상을 갱신하기로 결정
 센서 탑재체 확인 및 문제점 체크
 무인기 정류장치 대신 외장 배터리를 이용하여 전원 공급
 통합 비행 시험
 MCC 팀과 연계하여 2차 통합 비행 시험

L o g o
62

L o g o

L o g o
64
 San Pancrazio Airfield 현장에서
무인기 이륙
 영상 촬영 후 지상으로 전송
 지상 SW에서 지오레퍼런싱 수행 후
클라우드 서버로 전송
 MCC에서 위성 인터넷으로 클라우드
서버에 접속, Mago3D를 통해 실시간
으로 가시화

L o g o

L o g o
66

L o g o
67

L o g o
Conclusions
 Live Drone Map can acquire multi-sensory data and
produce high resolution geospatial information in real-
time and visualize them rapidly.
 With Live Drone Map, we can quickly acquire high
resolution image maps of mission areas with no maps
or old maps only.
 The accuracy of the image maps are about 1~5 m
depending on the flight altitude without any GCP.
 These latest maps will be useful for many important
tasks in UN field operations, such as monitoring
disputed areas and restoring disaster areas.
68

L o g o
Future Work
 We need to apply our Live Drone Map to real mission
areas such as Lebanon and improve it for more
practical use.
 We are currently using WiFi and LTE networks, but we
will employ RF links and mobile networks of the
mission areas.
 We are currently processing data on the computers in
mission areas, but we will also perform the data
processing on a cloud server.
 We will test with cheaper drones such as DJI ones,
which are widely used in the field.
69

Live Drone Map UAV Mapping Solution

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Live Drone Map UAV Mapping Solution

Similar to Live Drone Map UAV Mapping Solution (20)

Recently uploaded

Recently uploaded (20)

Live Drone Map UAV Mapping Solution