Using artificial intelligence algorithms and machine learning techniques, ToPa 3D was able to identify the center point of each nursery plant pot in the ground for precision trimming operations. This saved Woodburn Nursery time and money to ensure their operation maintained maximum efficiency
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
ToPa 3D Orthomosaic Technical Report for Woodburn Nursery Machine Learning Project
1. 1
ToPa 3D Orthomosaic Technical Report
Orthomosaic and Machine Learning
Technical Report
Woodburn Nursery 2021
Project code: T21-AGNW-001
Prepared by: Heather Sauerland | Geospatial Technologist
Admin:
heather@topa3d.com
ToPa 3D, Inc.
Paul Tice | CEO
paul@topa3d.com
Date submitted:
To:
October 29, 2021
Ag Geospatial NW, LLC
REPORT SENSITIVITY
Intended for journal publication NO
Results are incomplete YES
Commercial/Marketing/IP concerns NONE
3. 3
ToPa 3D Orthomosaic Technical Report
Collection Type
The goal of this project was to determine the best combination of drone equipment, flight height, ground control point
configuration, and viability of using AI object detection to locate pot centers.
ToPa 3D mapped the approximately 19 acres of the Woodburn Nursery and surveyed in ground control targets as needed to
produce an orthomosaic map with the intention of 1 cm ground sampling distance (GSD) or less from sUAV imagery before
GCP correction.
As this was an R&D project, the project site was mapped with three drones with varying altitudes, capturing 80% nadir photo
overlap to determine the best equipment for future replication and the ability to scale. All mapping missions were created and
flown with the DJI GSPro application.
The drones used for this project were:
1. DJI Phantom 4 Pro, 20 MP camera, mechanical shutter, Focal Length-24mm/35mm equivalent
2. DJI Mavic Pro 2, 20 MP camera, digital shutter, Focal Length-28mm
3. DJI Inspire 2, X7 24mm camera
4. DJI Inspire 2, X7 35mm camera
4. 4
ToPa 3D Orthomosaic Technical Report
Methodology
The intended plan was to map the project site from 300’ down to 100’ at 50’ increments to determine the most efficient way to
collect high-resolution imagery. Due to long flight times and/or weather conditions, several of these altitudes were canceled. It was
also determined that any planned flight with a GSD over 1 cm before GCP correction was also canceled. This is due to the
expected relative horizontal accuracy after GCP were applied to be double the GSD. The required corrected control GCP
accuracy for the project was 3cm.
All flights were flown using the DJI GSPro mapping app with 80% side and front photo overlap, which is recommended for
agriculture. https://support.pix4d.com/hc/en-us/articles/203756125-How-to-verify-that-there-is-enough-overlap-between-the-
images (Pix4D Mapping Software Documentation)
All flights were processed with Pix4D using photogrammetric practices to create a high resolution orthomosaic. See Figure 1 for
Woodburn Nursery Orthomosaic in Appendix B.
Using the Pix4D Quality Report, the most accurate flights by sUAV (drone) and most accurate control point configuration was
determined. Only the most accurate flights in combination with the most accurate control point configuration were used in the
object detection platform. With all drones, the second control point configuration was determined to be the most accurate when
considering both mean error of control points and checkpoints for confirmation. Please see Figure 2 for the Phantom 4 125’ Flight
Pix4D Quality Report. See Figures 3-5 for control point configurations in the Appendix B.
A machine learning AI software was used to identify the center of open pots and the center of planted pots. This
required the technician to use drone imagery to identify a subset of objects (open pots, plants with no visible pot, and
plants with some visible pots). Using this training data, machine learning software identified 9,425 individual plants
and/or pots in the dataset.
To determine the validity of the machine learning identified objects, this data was compared to surveyed pots and
plants. The difference between these two datasets was then measured in a GIS software. See Figure 6 for Center
Points Map in Appendix B.
Pots that did not have a plant (open pots) were found to be the most accurate to the surveyed subset. These open
pots had an average distance offset of 22.23 mm, with the largest offset of 142 mm and the lowest at 1 mm. See the
Woodburn Sitecheck Open Pots spreadsheet for measurements, Figure 1 in Appendix A.
Pots that contained a plant were more difficult for the objection detection software to find a true center point of the pot.
This is due to uneven growing of the plant or possible lean of plant. While machine learning can determine the center
of the visible plant, this may not equate well to the center of the pot. Pots with plants had an average offset distance of
60.67 mm, with the largest offset at 138 mm and the lowest at 7 mm. See the Woodburn Sitecheck Planted Pots
spreadsheet for measurements, Figure 2 in Appendix A.
5. 5
ToPa 3D Orthomosaic Technical Report
Expected Accuracy of Data
1. GSD
a. Flights were performed with an expected GSD of 1 cm before GCP correction, which results in a horizontal GSD
of approximately 2-3 cm after correction.
2. Accuracy of machine learning identified objects
a. Open Pots
i. The average distance between object detected centers of open pots was 22.23 mm.
ii. Largest offset was 142 mm.
iii. Smallest offset was 1 mm.
b. Planted Pots
i. The average distance between object detected centers of pots with plants was 60.67 mm.
ii. The largest offset was 138 mm.
iii. The smallest offset was 7 mm.
3. Surveyed control points, check points, and selected pot centers were provided by Ag Geospatial NW, LLC
11. 11
ToPa 3D Orthomosaic Technical Report
Appendix B.
Reports
Figure 1. Woodburn Nursery Orthomosaic, Phantom 4, 125’
12. 12
ToPa 3D Orthomosaic Technical Report
Figure 2. Phantom 4 Pro, 125’ Flight, Pix4D Quality Report
Quality Report
Generated with Pix4Dmapper version 4.6.4
Important: Click on the different icons for:
Help to analyze the results in the Quality Report
Additional information about the sections
Click here for additional tips to analyze the Quality Report
Summary
Project Woodburn_Nursery_Phantom_125'
Processed 2021-10-24 08:26:40
Camera Model Name(s) FC6310_8.8_5472x3648 (dc701ccfd379a5bc4f213b10686a7943) (RGB)
Average Ground Sampling Distance (GSD) 0.94 cm / 0.37 in
Area Covered 0.107 km2
/ 10.6606 ha / 0.04 sq. mi. / 26.3566 acres
Quality Check
Images median of 57987 keypoints per image
Dataset 926 out of 926 images calibrated (100%), all images enabled
Camera Optimization 0% relative difference between initial and optimized internal camera parameters
Matching median of 28741.9 matches per calibrated image
Georeferencing yes, 8 GCPs (8 3D), mean RMS error = 0.008 ft
13. 13
ToPa 3D Orthomosaic Technical Report
Calibration Details
Number of Calibrated Images 926 out of 926
Number of Geolocated Images 926 out of 926
Absolute camera position and orientation uncertainties
X [ft] Y [ft] Z [ft]
Omega
[degree]
Phi
[degree]
Kappa
[degree]
Camera Displacement
X [ft]
Camera Displacement
Y [ft]
Camera Displacement
Z [ft]
Mean 0.054 0.395 0.013 0.205 0.027 0.006 0.005 0.003 0.200
Sigma 0.032 0.131 0.003 0.068 0.016 0.003 0.001 0.001 0.066
Bundle Block Adjustment Details
Number of 2D Keypoint Observations for Bundle Block Adjustment 25725164
Number of 3D Points for Bundle Block Adjustment 6585326
Mean Reprojection Error [pixels] 0.184
Internal Camera Parameters
FC6310_8.8_5472x3648 (dc701ccfd379a5bc4f213b10686a7943) (RGB). Sensor Dimensions: 12.833 [mm] x
8.556 [mm]
EXIF ID: FC6310_8.8_5472x3648
Focal
Length
Principal
Point x
Principal
Point y
R1 R2 R3 T1 T2
Initial Values
3668.760 [pixel]
8.604 [mm]
2736.000 [pixel]
6.417 [mm]
1824.000 [pixel]
4.278 [mm]
0.003 -0.008 0.008 -0.000 0.000
Optimized Values
3668.763 [pixel]
8.604 [mm]
2740.909 [pixel]
6.428 [mm]
1824.403 [pixel]
4.279 [mm]
-0.012 0.003 0.006 -0.001 -0.002
Uncertainties (Sigma)
0.227 [pixel]
0.001 [mm]
0.208 [pixel]
0.000 [mm]
0.218 [pixel]
0.001 [mm]
0.000 0.000 0.000 0.000 0.000
CorrelatedIndependentFC0xC0yR1R2R3T1T2
The correlation between camera internal parameters determined by the bundle adjustment. White
indicates a full correlation between the parameters, ie. any change in one can be fully compensated
by the other. Black indicates that the parameter is completely independent, and is not affected by
other parameters.
The number of Automatic Tie Points (ATPs) per pixel, averaged over all images of the camera model,
is color coded between black and white. White indicates that, on average, more than 16 ATPs have
been extracted at the pixel location. Black indicates that, on average, 0 ATPs have been extracted at
the pixel location. Click on the image to the see the average direction and magnitude of the re-
projection error for each pixel. Note that the vectors are scaled for better visualization. The scale bar
indicates the magnitude of 1 pixel error.
14. 14
ToPa 3D Orthomosaic Technical Report
2D Keypoints Table
Number of 2D Keypoints per Image Number of Matched 2D Keypoints per Image
Median 57987 28742
Min 26257 8672
Max 79315 48032
Mean 57805 27781
3D Points from 2D Keypoint Matches
Number of 3D Points Observed
In 2 Images 3137419
In 3 Images 1113384
In 4 Images 639853
In 5 Images 421373
In 6 Images 300725
In 7 Images 226616
In 8 Images 176789
In 9 Images 137355
In 10 Images 108565
In 11 Images 84595
In 12 Images 65648
In 13 Images 50308
In 14 Images 38324
In 15 Images 26642
In 16 Images 19286
In 17 Images 15186
In 18 Images 10743
In 19 Images 6097
In 20 Images 3503
In 21 Images 1848
In 22 Images 810
In 23 Images 229
In 24 Images 24
In 25 Images 3
In 26 Images 1
15. 15
ToPa 3D Orthomosaic Technical Report
Geolocation Details
Ground Control Points
GCP Name Accuracy XY/Z [ft] Error X [ft] Error Y [ft] Error Z [ft] Projection Error [pixel] Verified/Marked
1000 (3D) 0.020/ 0.020 0.016 0.001 0.023 0.493 5 / 5
1003 (3D) 0.020/ 0.020 0.002 0.002 0.011 0.279 5 / 5
1005 (3D) 0.020/ 0.020 0.000 -0.001 -0.017 0.356 5 / 5
1009 (3D) 0.020/ 0.020 -0.005 -0.001 0.005 0.354 5 / 5
1011 (3D) 0.020/ 0.020 0.004 0.001 -0.007 0.362 5 / 5
1013 (3D) 0.020/ 0.020 -0.011 -0.001 -0.010 0.512 5 / 5
1015 (3D) 0.020/ 0.020 0.000 0.003 0.026 0.219 5 / 5
1017 (3D) 0.020/ 0.020 0.001 -0.005 -0.020 0.354 5 / 5
Mean [ft] 0.000828 -0.000171 0.001211
Sigma [ft] 0.007063 0.002506 0.016446
RMS Error [ft] 0.007111 0.002512 0.016491
0 out of 13 check points have been labeled as inaccurate.
Check Point Name Accuracy XY/Z [ft] Error X [ft] Error Y [ft] Error Z [ft] Projection Error [pixel] Verified/Marked
1001 0.0272 0.0907 0.1152 0.2812 5 / 5
1002 0.0655 0.0921 0.0451 0.2523 5 / 5
1004 -0.0266 -0.0380 0.0424 0.2698 5 / 5
1006 -0.0449 -0.0187 0.0864 0.1742 5 / 5
1007 -0.0311 0.0278 0.0753 0.3408 5 / 5
1008 -0.0479 -0.0026 -0.0509 0.4121 5 / 5
1010 -0.0224 -0.0028 0.0053 0.3896 5 / 5
1012 0.0185 0.0510 0.1450 0.1260 5 / 5
1014 -0.0223 0.0356 0.1030 0.3593 5 / 5
1016 0.0364 0.0380 0.0514 0.9147 5 / 5
1018 0.0326 0.0379 0.1091 0.4784 5 / 5
1019 0.0476 0.0274 0.0216 0.5751 5 / 5
1020 -0.0235 -0.0212 0.1129 0.4137 5 / 5
Mean [ft] 0.000703 0.024398 0.066293
Sigma [ft] 0.036744 0.038730 0.051984
RMS Error [ft] 0.036750 0.045774 0.084245
Localization accuracy per GCP and mean errors in the three coordinate directions. The last column counts the number of calibrated images where the GCP has
been automatically verified vs. manually marked.
16. 16
ToPa 3D Orthomosaic Technical Report
Absolute Geolocation Variance
Min Error [ft] Max Error [ft] Geolocation Error X [%] Geolocation Error Y [%] Geolocation Error Z [%]
- -49.21 0.00 0.00 0.00
-49.21 -39.37 0.00 0.00 0.00
-39.37 -29.53 0.00 0.00 0.00
-29.53 -19.69 0.00 0.00 0.00
-19.69 -9.84 0.00 9.83 9.94
-9.84 0.00 49.57 44.06 48.81
0.00 9.84 50.43 43.84 20.52
9.84 19.69 0.00 2.27 20.73
19.69 29.53 0.00 0.00 0.00
29.53 39.37 0.00 0.00 0.00
39.37 49.21 0.00 0.00 0.00
49.21 - 0.00 0.00 0.00
Mean [ft] -4.170204 -7.087969 51.775953
Sigma [ft] 1.268811 6.528687 8.323862
RMS Error [ft] 4.358954 9.636548 52.440785
Min Error and Max Error represent geolocation error intervals between -1.5 and 1.5 times the maximum accuracy of all the images. Columns X, Y, Z show the
percentage of images with geolocation errors within the predefined error intervals. The geolocation error is the difference between the initial and computed
image positions. Note that the image geolocation errors do not correspond to the accuracy of the observed 3D points.
Geolocation Bias X Y Z
Translation [ft] -4.170204 -7.087969 51.775953
Bias between image initial and computed geolocation given in output coordinate system.
Relative Geolocation Variance
Relative Geolocation Error Images X [%] Images Y [%] Images Z [%]
[-1.00, 1.00] 100.00 100.00 100.00
[-2.00, 2.00] 100.00 100.00 100.00
[-3.00, 3.00] 100.00 100.00 100.00
Mean of Geolocation Accuracy [ft] 16.404199 16.404199 32.808399
Sigma of Geolocation Accuracy [ft] 0.000004 0.000004 0.000007
Images X, Y, Z represent the percentage of images with a relative geolocation error in X, Y, Z.
17. 17
ToPa 3D Orthomosaic Technical Report
Geolocation Orientational Variance RMS [degree]
Omega 0.591
Phi 0.663
Kappa 8.462
Geolocation RMS error of the orientation angles given by the difference between the initial and computed image orientation angles.
Figure 6: Camera movement estimated by the rolling shutter camera model. The green line follows the computed image positions. The blue dots represent the
camera position at the start of the exposure. The blue lines represent the camera motion during the rolling shutter readout, re-scaled by a project dependant
scaling factor for better visibility.
Median Camera Speed 12.4118 [ft/s]
Median Camera Displacement During Sensor Readout) 0.4279 [ft]
Median Rolling Shutter Readout Time 34.8234 [ms]
Initial Processing Details
System Information
Hardware
CPU: Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
RAM: 64GB
GPU: AMD Radeon Pro 5600M (Driver: 26.20.15032.1001)
Operating System Windows 10 Pro, 64-bit
Coordinate Systems
Image Coordinate System WGS 84 (EGM 96 Geoid)
Ground Control Point (GCP) Coordinate System OCRS_Salem_NAD_1983_2011_TM_Ft_Intl (EGM 96 Geoid)
Output Coordinate System OCRS_Salem_NAD_1983_2011_TM_Ft_Intl (EGM 96 Geoid)
Processing Options
Detected Template No Template Available
Keypoints Image Scale Full, Image Scale: 1
Advanced: Matching Image Pairs Aerial Grid or Corridor
Advanced: Matching Strategy Use Geometrically Verified Matching: yes
Advanced: Keypoint Extraction Targeted Number of Keypoints: Automatic
18. 18
ToPa 3D Orthomosaic Technical Report
Advanced: Calibration
Calibration Method: Standard
Internal Parameters Optimization: All prior
External Parameters Optimization: All
Rematch: Auto, no
Point Cloud Densification details
Processing Options
Image Scale multiscale, 1/2 (Half image size, Default)
Point Density Optimal
Minimum Number of Matches 3
3D Textured Mesh Generation yes
3D Textured Mesh Settings:
Resolution: Medium Resolution (default)
Color Balancing: no
LOD Generated: no
Advanced: 3D Textured Mesh Settings Sample Density Divider: 1
Advanced: Image Groups group1
Advanced: Use Processing Area yes
Advanced: Use Annotations yes
Time for Point Cloud Densification 10h:15m:52s
Time for Point Cloud Classification 34m:50s
Time for 3D Textured Mesh Generation 18m:02s
Results
Number of Generated Tiles 8
Number of 3D Densified Points 105766203
Average Density (per ft3
) 103.65
DSM, Orthomosaic and Index Details
Processing Options
DSM and Orthomosaic Resolution 1 x GSD (0.936 [cm/pixel])
DSM Filters
Noise Filtering: yes
Surface Smoothing: yes, Type: Sharp
19. 19
ToPa 3D Orthomosaic Technical Report
Raster DSM
Generated: yes
Method: Inverse Distance Weighting
Merge Tiles: yes
Orthomosaic
Generated: yes
Merge Tiles: yes
GeoTIFF Without Transparency: no
Google Maps Tiles and KML: yes
Grid DSM Generated: yes, Spacing [cm]: 100
Raster DTM
Generated: yes
Merge Tiles: yes
DTM Resolution 5 x GSD (0.936 [cm/pixel])
Time for DSM Generation 44m:47s
Time for Orthomosaic Generation 01h:20m:27s
Time for DTM Generation 22m:11s
Time for Contour Lines Generation 00s
Time for Reflectance Map Generation 00s
Time for Index Map Generation 00s
23. 26
ToPa 3D Orthomosaic Technical Report
(38.10m) Overlap Scrapped
100'
(30.48m)
80%
Overlap
Flight Time > 1 Hour =
Scrapped
N/A N/A
100'
(30.48m)
80%
Overlap
Flight Time > 1 Hour =
Scrapped
N/A N/A
100'
(30.48m)
80%
Overlap
Flight Time > 1 Hour =
Scrapped
N/A N/A
24. 27
ToPa 3D Orthomosaic Technical Report
Additional Information and Recommendations
Several qualities were considered to determine the best equipment for future replication:
1. Ability to reach required GSD before and after GCP correction
2. Length of flight time
3. Cost/ease of initial investment into equipment
All drones and camera combinations were able to capture at the project’s required GSD. Some drones required extended
flight times to reach the 1cm GSD before GCP correction. Flights that were more than 1 hour were removed from the dataset
which did not provide any noticeable GSD improvement. See Figure 7 for Drone Flight Information in Appendix B.
It is our recommendation after considering the qualities listed above, that the Phantom Pro 4 be used for future flights. This
drone is a lower cost drone and is familiar to most users. It also had the fastest flight time for the desired GSD before and
after GCP correction.
The Mavic 2 Pro is a viable option due to its popularity in the market, small profile, and quality photos, however it was slower
than the Phantom for comparable GSD.
While the Inspire 2 can take high quality photos, the increased cost, lower portability, and less familiarity to the casual drone
pilot makes it a less desirable option.
The machine learning AI algorithms worked more accurately on open pots. Some pots with plants had visible rims of pots in the
orthomosaic. It was thought that this might be enough for the machine learning to identify the pot and infer that rest of the
circumference, therefore identifying a more accurate center point than that of the plant. This was tested with AI algorithms and
was found to not have a significant increase in accuracy. It is therefore recommended that this process be performed on open
pots in the future.
Delivered center point locations are in WGS 84.
25. 28
ToPa 3D Orthomosaic Technical Report
Glossary and Acronyms
Below is a Abbreviations and Acronyms list.
GSD Ground sampling distance
Photogrammetry The use of overlapping photography in surveying and mapping to measure
distancesbetween objects.
sUAV Small Unmanned Aerial Vehicle
GCP Ground control point
AI/Object Detection The use of computers and algorithms to automatically identify an object. This process includes a human
element of teaching the computer on a subset of data and then using that trained computer to detect
objects in a larger dataset.
AGL Above ground level- Height of flight above the ground
-END of REPORT-