Resource-aware and incremental mosaics of wide areas from small-scale UAVs. we study the problem of placing mobile sensors to get high coverage, Based on Voronoi diagrams. BASIC PROTOCOLS,VIRTUAL MOVEMENT PROTOCOLS

1. Resource Aware and
Incremental mosaics of wide
areas from small scale
UAVs.
By
Bhaskar Reddy gurram(1774985)
Murali gullapelli(1741737)

2. Plan of attack
1. Abstract
2. Introduction
3. Related work
4. Overview
5. Efficient overview image generation
6. Conclusion

3. Abstract:
Small-scale Unmanned aerial vehicles (UAVs) are an emerging research area
and have been recently demonstrated in many application such as
● Disaster response management
● Construction site monitoring
● Wide area surveillance.
In this work we present a system composed of multiple networked UAVs for
autonomously monitoring a wide area.
Each uav is able to follow waypoints and capturing high resolution images.
We implement an Incremental Approach for generating an orthographic mosaics.

4. Abstract cont…
● Captured images are are pre-processed on-board,annotated with other
sensor data and Transferred by a Prioritized transmission scheme.
● Ultimate goal of our approach is to generate an mosaics as fast as possible
and to improve its quality over time.
● We evaluate our incremental mosaicking in the strongly resource limited UAV
network composed of up to three concurrently flying UAVs.
● Our results are compared to state -of-the art mosaicing methods and show a
unique performance in our dedicated application scenarios.

5. Introduction:
● In our main use case of emergency and disaster response the overall goal is
to build different mosaics from large unknown areas quickly.
● We employ multiple small-scale quad unmanned aerial vehicles(UAVs) that
are able to vertically take -off and land(vtol)concurrently improve the time of
coverage.
● Such easy to operate autonomous aerial sensing platforms since human
resources are limited in emergency situations.
● To successfully transmit ,mosaics and merge high resolution images ,we
propose an incremental image processing and mosaicking where we are able
increase the mosaic quality over time.

6. UAV System:
● UAV platform have been introduced by companies such as Ascending
Technologies or Micro drones.
● They offer battery powered devices with different control and sensing
capabilities
● With a take off weight between one and five kilograms,the UAV still carry
high resolution cameras and operate for 12-45 minutes.
● Our UAVs incorporate two processing units with various sensors ,such as
inertial measurements unit or global navigation systems,and at least one
camera.

7. Contribution:
Contribution of work covers the,
● Resource aware.
● Incremental improvement
of an overview mosaic from high resolution images transmitted via an aerial
network from small scale UAVs.
This approach fills the gap between panorama mosaics & expensive full 3D
constructions by delivering result with increasing quality in a very short time.
So flight routes are optimized to reduce redundant data,means overlap between
images.

8. Related work:
● Aerial photogrammetry technologies expanded to higher altitudes such as
satellites that provide already high resolution photos of earth surface.
● Unfortunately in NO-REAL-TIME.
● Typically, individual images are taken from a single airplane & processed
offline after landing.
● But aerial camera networks and online mosaicking rarely investigated before.
● So present evolution reaches from single cameras flying at high altitude to
network cameras also deployed at low altitudes.

9. Related work cont..
● And also live data streaming but UAV camera require more complex and
active communication links to transmit sensed data.
● Recently multiple vehicles are combined to increase their resource efficiency
and operating range.
● We are addressing here combination of efficient data transmission and
mosaicking from low altitudes.

10. Deployed UAV system:
● Our system deals with the response management after severe disasters
where mosaics should be generated as fast as possible from images
captured by small-scale ,low altitude UAVs.
Fig 1:Key processing steps and their challenges

11. Designed system:3 main components
1. Highly mobile small scale UAVs equipped with various sensors and
processing units.
2. Wireless communication network for control and data transmission.
3. Ground station where routes are planned data is collected,processed and
visualized.

12. Fig 2: our systems with small scale UAVs and communication network,ground station

13. ● Light grey arrow in the background
sketches the data flow of our
process through the blocks marked
in green which are covered by this
work on the application layer.
● On the left side multiple user
interfaces are shown:
1. One interface allows the
operators to input their
requirements.
2. Another interface is available
for observing the results.

14. Mission:
● The main requirements for a mission are a bundle of available resources.
● I.e multiple participating heterogeneous UAVs the area to cover.
● The user specifies time to complete the mission and temporal and spatial
target resolution.
● From this inputs predefined routes are generated & included as mission plan
into the mission.
● Already generated route plans with dedicated picture points,which are 3D
locations in the world coordinate system,where images shall be taken.
● The planned routes are sent to the UAVs via the communication network.

15. Depicts two scenarios for
our mosaicking system:
1st scenario:
● A firefighter practise scenario
we planned 5 consecutive
missions by a single uav to
update the final overview
image by more recent data.
● Flight time of each route was
about 620s over the area
about 12,000m^2

16. 2nd scenario:
● Three UAVs complete one mission
concurrently.the whole area of
more than 45,000m^2 is covered
in less than 500s.
● Since a maximum flight time of
any our UAV system is one
mission is not allowed to exceed
840s.
● After completing a flight each UAV
autonomously lands and stay
active to complete unfinished
tasks.

17. Networking:
● In wide area scenario we cannot rely on existing communication infrastructure
with sufficient bandwidth.
● The typical period between capturing images is about 10-15s on an individual
UAV.
● But the amount of raw data exceeds more than 10MB per image & we have
multiple UAVs operation concurrently.
Effective strategy needs to be realized to process and transmit the captured
data.

18. We utilize a wireless LAn in infrastructure based on IEEE 802.11 a at 5GHz with 3
antennas attached in multiple input & multiple output mode(MIMO).
Antenna setup is optimized to emphasize on best connectivity ,even during motion
& tilt of the UAV.
Extensive tests have shown:
● Stable communication within proposed scenario,but with limited rates.
● There is the maximum achievable data rate is 54 mbit/s in close regions.

19. Imaging:
● On our small-scale UAVs we utilize high resolution cameras for RGB still
images, such as light weight off-shelf digital consumer cameras with
resolution upto 12 megapixel & remote control capabilities.
● Cameras are mounted on our adaptive camera mount which is stabilized and
adjust its view angle.
● Camera is directly connected via USB to onboard processing unit.
● The image processing on-board the UAV is implemented using shell & python
scripts,kakadu JPEG2000 library.

20. Efficient overview image generation:
● For efficient & resource aware mosaicking we pre-process and annotate
images with metadata altered on the UAV.
● Transmission is based on data prioritization to transfer important data first in
limited bandwidth scenario.
● Already processed results previous mosaics are integrated to reduce over all
processing effort only to fresh data.
● Fresh image data is defined to be either images of uncovered areas(not
transmitted before)
● High resolution representation of already covered area.

21. Incremental Approach:
Basically 3 individual steps in our incremental mosaicking are considered for
quality improvement.
1. A metadata based.
2. Feature based mosaicking.
3. Structure based.

22. 1.Metadata based
Generated by simple image placement using only metadata.
Such as
Uav position( )
Orientation( )
Intrinsic camera parameter( )

23. Feature based:
● Feature based mosaicking is executed with the feature extraction and
registration of images by similarity transformations if an immediate
improvement is required by the application.
● We are reducing the overlapping images areas already during the planning to
reduce redundant data during transmission.
● This reduces the quality but efficient as first part of the image data based
mosaicking.

24. Structure based
● Structure based mosaicking is the last leap of improvement when employing
the structure analysis of the scene.
● A sparse 3D model is constructed from the extracted key-point and we
estimate a common projection plane within this 3D structure.
● -Image transformation.
● -geometric projection.
● -Image data based on mosaicking.

25. Fig 4: the block chart shows incremental approach at the base station and other blocks on each UAV.

26. Block diagram description:
● Basic processing blocks and layers of our mosaicking are presented.
● On the left the internal sensing units on the UAV platform encode data &
transmit to base station.
● Data is collected and organized with already processed data in the buffer.
● In the computation layer the different processing stages are covered while
presentation layer executes the image transmission and provide different
quality levels of mosaics.
● Apparently this outputs depend on achieved computation results.

27. Prioritized image transfer:
● Throughout the mission multiple UAVs capture more and more images(10-
15s period).
● But the amount of raw data exceeds more than 10MB per image which is
challenging to transmit.
● The limited resource on the UAV require an efficient strategy to process &
transmit captured data.
● UAV images or fragments images are prioritized before transmission
according to their forecast benefit to the whole overview mosaic.

28. Prioritization cont..
● Basically , data from newly explored areas, which can be a complete images
or a rectangular fragment of one image has the highest priority and should be
transferred immediately.
Fig 5: data separation two overlapped images.

29. Prioritized transformation:
● Highest priority is given to new data
with even low resolution.
● Primary aim is to cover the full area
with in time.
● The complete images are
scheduled in higher resolutions
later for transmission.

30. Application layer scheduling:
● Basically images are split into fragments & prioritized according to their
resolution and qualities.
● On each uav the images are progressively encoded by JPEG2000 buffered.
● Initial priority depend on resolution layer encoding.
● The highest priority is assigned to image fragments of the lowest resolution &
uncovered regions.
● Next highest priority is given to next higher resolution(which is low priority).

31. Performance of Incremental Approach:
Fig:6 Quality improvement over time by our approach.

32. Conclusion:
● In this work we presented an incremental mosaicking approach of aerial
images from multiple small-scale UAVs flying at low altitudes.
● Our approach was driven for dedicated applications such as disaster
response management over wide area.
● Focused on by prioritized data transmission and incremental processing.
● We successfully generate a growing mosaic by utilizing limited resources
efficiency.