Estimation of Crop Acreage From Satellite Imagery -- EENG 510 -- Daniel Kuntz

Daniel Kuntz EENG 510
Estimation of Crop Acreage from Satellite Imagery Dec. 8, 2014
Table of Contents
1.0 Introduction............................................................................................................................................3
1.1 Assumptions.......................................................................................................................................3
1.2 Data Used for This Project................................................................................................................3
2.0 Review of Previous Work.......................................................................................................................3
2.1 High Temporal Resolution Methods..................................................................................................4
2.2 High Spatial Resolution Methods......................................................................................................4
2.3 Discussion of Previous Work.............................................................................................................5
3.0 Methodology..........................................................................................................................................5
3.1 Field Detection..................................................................................................................................5
3.2 Segmentation of Fields......................................................................................................................5
3.3 Classification of Crop Fields.............................................................................................................6
3.4 Software Implementation..................................................................................................................6
4.0 Results....................................................................................................................................................6
4.1 Detection of Fields.............................................................................................................................6
4.2 Segmentation.....................................................................................................................................8
4.3 Classification Results.......................................................................................................................11
5.0 Discussion............................................................................................................................................13
5.1 Feasibility........................................................................................................................................13
5.2 Future Improvements.......................................................................................................................13
6.0 Conclusion............................................................................................................................................13
7.0 Bibliography.........................................................................................................................................14
8.0 Appendix..............................................................................................................................................15
8.1 Software and Libraries Used in This Project...................................................................................15
8.2 Main Code.......................................................................................................................................15
8.2.1 Detection and Segmentation Code...........................................................................................15
8.2.2 Data Extraction Code...............................................................................................................17
8.2.3 Prediction and Evaluation Code...............................................................................................19
8.3 Utility Libraries...............................................................................................................................21
8.3.1 Landsat Data Utilities..............................................................................................................21
8.3.2 Coordinate Calculations...........................................................................................................24
8.3.3 Hough Line Drawing Code......................................................................................................26
8.3.4 Temporal Manipulation Utilities..............................................................................................27
8.3.5 CropScape Tools......................................................................................................................28
1 / 29

Figures
Figure 1: Temporal NVDI Average of the SLV............................................................................................7
Figure 2: Overall Field Segmentation Results.............................................................................................8
Figure 3: Good Segmentation Results........................................................................................................10
Figure 4: Bad Segmentation Results..........................................................................................................10
Figure 5: Fields Used for Classification.....................................................................................................11
Figure 6: 2011 CropScape Classifications.................................................................................................12
Tables
Table 1: Landsat 4/5 Reflectance Bands [1].................................................................................................3
Table 2: SVM Settings.................................................................................................................................6
Table 3: Field Segmentation Results............................................................................................................9
2 / 29

1.0 Introduction
The aim of this project is to explore remote sensing techniques for prediction of agricultural yield.
Predictions of this kind serve important functions in land management and market estimation and are
becoming ever more relevant for improving the efficiency of modern agriculture. In particular, this
project will use satellite imagery of the San Luis Valley (SLV) of Colorado, to try and determine the
location of fields and what crops are contained in those fields.
1.1 Assumptions
There are certain assumptions implicit from using the SLV agricultural region:
1. The fields are irrigated by center pivot and are uniform circles with a radius of ¼ mile.
2. The fields also lie mostly on a perpendicular road grid.
3. The climate of the SLV is that of a high alpine desert, and as such, there is not a lot of non-
agricultural vegetation around fields that might get confused for crops.
1.2 Data Used for This Project
Data for this project was obtained from the USGS Earth Explorer website which archives publicly
available satellite data. Specifically, images from 2000-2012 from Landsat 4 and 5 were used. For these
images, each scene consisted of the 7 different bands outlined in Table 1. Each of these bands has a
resolution of 30 meters except band 6 which has a resolution of 120 meters.
Band Wavelength (μm) Description
1 0.45 – 0.52 Blue
2 0.52 – 0.60 Green
3 0.63 – 0.69 Red
4 0.76 – 0.90 Near Infrared
5 1.55 – 1.75 Shortwave Infrared 1
6 10.40 – 12.50 Thermal Infrared
7 2.08 – 2.35 Shortwave Infrared 2
Table 1: Landsat 4/5 Reflectance Bands [1]
2.0 Review of Previous Work
Much of the previous work in related to this project takes one of two different approaches. The first and
most common is to look at high temporal resolution satellite imagery such as MODIS (which has a
spatial resolution of ~1 km, teporal resolution of ~1 day) and then matching the reflectance to a known
temporal reflectance signature of the crops, one such method is described by Bolton et al. [2] The second
3 / 29

and less common method is to use high spatial resolution imagery such as Landsat (temporal resolution
~15 days) or IKONOS and to use a more sophisticated classification technique, one such method is
described by H. Xie et al. [3]. Sections 2.1, 2.2 detail these methods as described by their respective
papers.
2.1 High Temporal Resolution Methods
The method described by Bolton et al. [2]:
1. Divide regions of interest into semi-arid and non-semi-arid.
2. Determine the “green-up” date by finding the onset date of a increase in the Enhanced Vegetation
Index (EVI) at each pixel.
3. Collect time series values for EVI, EVI2, Normalized Difference Vegetation Index (NDVI) and
Normalized Difference Water Index (NDWI) for 120 days after the green-up.
4. Determine pixels that are dominated by a specific crop type (>50%) by matching it to a pre-
determined crop signature and use that number as a weighted average.
5. Create a linear regression model to matches the calculated weighted averages to the actual yield
for years where data had already been compiled.
This paper reported errors between 9-15% depending on the type of crop, but does not divide out regions
in the study further than by whether it is arid or semi-arid.
2.2 High Spatial Resolution Methods
The method described by Xie et al. [3]:
1. Manually segment the crop fields using GIS software.
2. Determine the mean reflectance in each of the eight bands for each field region to get a
“reflectance vector” with dimensionality equal to the number of reflectance bands.
3. The angular difference (1) is then compared to various reference vectors to determine what
classification to give each field. This is referred to as the “Spectral Angle Mapper” (SAM)
classifier.
θ=cos
−1
(⟨ ⃗m,⃗r ⟩
‖⃗m‖‖⃗r‖), where: ⃗m = The measured reflectance vector
⃗r = The reference reflectance vector
(1)
4. If the classifier matches it to a know spectrum, then it is classified as an “active crop”, otherwise
it is classified as “fallow land”
However, the results of this report were described as a comparison to another method, and hence
reliability with respect to a ground truth was not established. Also, the reference reflectance vectors had
to be determined beforehand in order to compare the crops to each. This involves further study of crop
phrenology and does not rely on any kind of machine learning methods.
4 / 29

2.3 Discussion of Previous Work
The two papers described above both have serious limitations. The technique described by Bolton et al.
can not be used to predict localized output and hence is not useful for land management techniques.
Where as the technique described by Xie et al is labor intensive, and was only used to classify if a field
was active or fallow, which is severely limiting because different crops require different land
management and affect the food commodities markets differently. The USGS CropScape platform
(which was used as a ground truth for this project) is accurate (reported ~80% accuracy [6]) but there are
no technical documents available showing exactly what is done to obtain results.
3.0 Methodology
For this project similar technique as the one described by Xie et al. was implemented, but with a few
differences. Fields are to be automatically segmented using techniques learned in class and as described
by Yan et al. [4]. Also, a more sophisticated classifier, Support Vector Machines (SVM), will be applied
to the dataset rather than SAM.
3.1 Field Detection
Fields are detected by using a temporal average of the Normalized Difference Vegetation Index (NVDI)
over the course of years 2005 to 2012 as described by Yan et al. [4]. This has the benefit of “stabilizing”
the pixels detected by the NVDI over the course of a few years and produced much better results than
just using one image.
NVDI=
rIR−rRED
rIR+rRED
, where: rx = reflectance in band x (2)
NVDIAVG=
1
n
∑
i=1
n
NVDIi , where:
NVDIi = NVDI in each scene
n = number of scenes
(3)
This gives a good indication of where each field is.
3.2 Segmentation of Fields
Fields were segmented using a hough line transform to find the roads that run in-between all of the
fields. This was effective since the fields generally lie between a perpendicular road grid. Once the lines
are found they are drawn back on to the map to segment out the fields. This procedure was done on
image blocks of 100, 300 then 500 pixels as this provided the best effect.
5 / 29

3.3 Classification of Crop Fields
Classification was accomplished via Support Vector Machine (SVM) method, settings used are listed in
Table 2. Data used for classification is listed below:
• Month
• Day
• Band 1-7 Mean of Reflectance of each field
• Band 1-7 Variance of Reflectance of each field
Setting Value Description
Kernel rbf Gaussian Radial Basis Function
C 1.0 Penalty coefficient for Errors
Table 2: SVM Settings
3.4 Software Implementation
Please see Appendices 8.1-8.3 for software implementation of the methodologies listed above.
4.0 Results
4.1 Detection of Fields
Figure 1 on the next page shows the results of the temporal NVDI averaging to extract areas that have a
high probability of being a field.
6 / 29

As is demonstrated by Figure 1, there is a clear distinction between the fields and non-agricultural areas.
Areas that are not fields are mostly pixels with values less than 2, making thresholding very easy. Once
thresholded, fields can be segmented and filtered to determine areas that are individual fields.
There are a few areas that are being picked up that are not fields, specifically in the lower left corner
where there is a green belt surrounding the Rio Grande river. Also, there are areas in the mountains that
are being picked up as fields but can mostly be removed by filtering by size and shape. Also, these may
be possible to eliminate as fields by classification in future improvements to the algorithm.
7 / 29
Figure 1: Temporal NVDI Average of the SLV

4.2 Segmentation
The overall segmentation results are shown in Figure 2.
It is apparent that most fields are being captured, with the exception of a few, also, there are some false
positives with a high NVDI and similar area but which are not fields. One way to analyze the
performance of the algorithm is to find fields that have not been segmented properly. This is done with
the following criteria:
8 / 29
Figure 2: Overall Field Segmentation Results

C = Number of fields with area > 700 (Multi-field blobs)
A = Total area of fields with area > 700
K =
A
620
(Number of fields represented by the blobs)
P =
K
Nfields +(K−C)
(Percentage of under-segmented Fields)
(4)
Where “P” is the overall percentage of fields that have been under-segmented. The results of this
analysis are shown in Table 3. We see that the percentage is quite high at ~11%. Suggesting that a better
methodology for segmentation would be highly beneficial (this is covered in section 5.2).
N_fields C A K P
1272 72 92.167 149 11.05
Table 3: Field Segmentation Results
Upon further analysis it is easy to see areas where the Hough Transform algorithm performs well and
where it performs poorly. Figure 3 shows a typical good result, and Figure 4 shows a typical poor results.
9 / 29
Figure 3: Good Segmentation Results

Field segmentation by this technique shows that the results are worst when there are fields which do not
align with the other fields. When there are a sparse number of fields in the area and where there are
irregularly shaped fields.
4.3 Classification Results
Classification was difficult because of lack of time and computing resources. A very inefficient method
for extracting field reflectance was implemented and truth values had to be added manually via
spreadsheet. Because of this, only 97 fields were used as a test case. These fields are shown in Figure 5,
CropScape classifications for the year 2011 are shown in Figure 6 for reference. All satellite images for
years 2009 – 2012 with low cloud cover were used.
10 / 29
Figure 4: Bad Segmentation Results

11 / 29
Figure 5: Fields Used for Classification
Figure 6: 2011 CropScape Classifications

Because of the limited amount of fields, and slightly inaccurate ground truth data, combined satellite
images through all years only provided ~ 1300 data points for explanatory vectors of 16 data points, this
is not ideal as usually x10 the data-points to the dimensions of your explanatory vector is a rule of thumb
[7].
To get a feel for how well the crops are predicted the crop type the data was divided randomly into
training and testing sets with training sets comprising of 80% of the data and testing sets comprising of
20% of the data. Then the prediction routine was trained and evaluated against the 20% training data.
This was done 100 times and the result was averaged. The output of this program is shown below:
After 100 trials, correct outcome stats:
Mean: 0.48190476190476184
Var: 0.0009165784832451503
This shows that about 48% accuracy was achieved in predicting field results.
5.0 Discussion
5.1 Feasibility
If the methodology is refined, it might prove the feasibility of using remote sensing in such a way. Other
variations such as per-pixel classification also need to compared and contrasted against as well as other
classification techniques and parameters to determine, which, if any perform the best.
5.2 Future Improvements
There are many areas for improvement:
1. In the arena of field segmentation, a more general “watershed” segmentation technique can be
used that is not limited to round fields. Also, Once the Hough Transform is performed, it can be
done at different resolutions to better connect areas that are segmented at lower resolutions.
2. In the arena of classification a much bigger data set would improve results.
3. Getting ground truth data from a more accurate source than CropScape would help classification
results, this could be obtained through field surveys or talking to farmers.
4. Using classification as a means of removing non-agricultural areas, thus a more aggressive NVDI
threshold can be used and false positive can be thrown out on a basis of having non-agricultural
vegetation.
6.0 Conclusion
This project shows a preliminary exploration of the remote sensing agricultural classification problem.
12 / 29

As it stands, many refinement need to be made before this methodology would be worthy of commercial
use, but it does show feasibility. There are also many improvements that could make performance and
accuracy of the implementation better, which means that techniques like this can be used to suitably
predict the use of land for agricultural and land management purposes.
13 / 29

7.0 Bibliography
[1] “Frequently Asked Questions about the Landsat Missions,”
http://landsat.usgs.gov/band_designations_landsat_satellites.php, Accessed Dec. 5, 2014
[2] Douglas K. Bolton, Mark A. Friedl, Forecasting crop yield using remotely sensed vegetation indices
and crop phenology metrics, Agricultural and Forest Meteorology, Volume 173, 15 May 2013, Pages 74-
84, ISSN 0168-1923, http://dx.doi.org/10.1016/j.agrformet.2013.01.007.
[3] Xie, H. H., Tian, Y. Q., Granillo, J. A., & Keller, G. R. (2007). Suitable remote sensing method and
data for mapping and measuring active crop fields. International Journal Of Remote Sensing, 28(2), 395-
411. doi:10.1080/01431160600702673
[4] L. Yan, D.P. Roy, Automated crop field extraction from multi-temporal Web Enabled Landsat Data,
Remote Sensing of Environment, Volume 144, 25 March 2014, Pages 42-64, ISSN 0034-4257,
http://dx.doi.org/10.1016/j.rse.2014.01.006.
[5] “How to Convert From UTM to LatLng in Python or Javascript,” Stack Overflow,
http://stackoverflow.com/questions/343865/how-to-convert-from-utm-to-latlng-in-python-or-javascript,
Answer provided by user: Staale
[6] “CropScape General Information,”
http://www.nass.usda.gov/research/Cropland/sarsfaqs2.html#Section1_12.0, Accessed Dec. 8, 2014
[7] Flavia Zarcula, Researcher and Statistician for OMNI Institute, Denver, CO. said “You generally
need about 10 times as many data-points as you have dimensions in your explanatory vector for most
predictive models,” personal conversation, October, 2014
14 / 29

8.0 Appendix
8.1 Software and Libraries Used in This Project
Name Description
Python Programming language
Numpy Linear algebra library for Python
Scikit-Image Image manipulation library for Python
Scikit-Learn Machine learning library for Python
LibreOffice Calc Open source spreadsheet software
8.2 Main Code
8.2.1 Detection and Segmentation Code
"""
Script that finds a crop field mask from satellite data
"""
from numpy import array, logical_not, logical_and, int_, uint16, pi, where
import matplotlib.pyplot as plt
from skimage.io import imsave, use_plugin
from skimage.transform import hough_line, hough_line_peaks
from skimage.morphology import binary_erosion, binary_closing, binary_opening,
rectangle, remove_small_objects
from skimage.measure import label, regionprops
from landsatutil.temporal import collect_bands, compress_temporal_image
from landsatutil.segmentation import draw_hough_line
# Set plugin
use_plugin('freeimage')
# UTM coordinates zone 13
# Very small area
#nw_corner = array([396210, 4175310])
#se_corner = array([404460, 4167150])
# Small Area
#nw_corner = array([390000, 4188090])
#se_corner = array([423900, 4164000])
# Large Area
nw_corner = array([387494, 4218065])
se_corner = array([440000, 4160000])
15 / 29

# Calculate file names
fname_post = '_{0}_{1}_{2}_{3}.'.format(nw_corner[0], nw_corner[1], se_corner[0],
se_corner[1])
fname_template = 'tmp/{0}' + fname_post + '{1}'
# Get temporal bands and compress them into a field mask
print('Collecting Bands')
temporal_band_4 = collect_bands(
4,
nw_corner,
se_corner,
list(range(2005, 2012)),
'Bulk Order 397884/L4-5 TM'
)
temporal_band_3 = collect_bands(
3,
nw_corner,
se_corner,
list(range(2005, 2012)),
'Bulk Order 397884/L4-5 TM'
)
temporal_nvdi = (temporal_band_4 - temporal_band_3) / (temporal_band_4 +
temporal_band_3)
field_mask = compress_temporal_image(temporal_nvdi)
imsave(fname_template.format('temporal_nvdi', 'png'), field_mask)
# Detect Fields
print('Detecting Fields')
field_mask = field_mask >= 10
# Find the area containing the fields
field_area = binary_closing(binary_erosion(field_mask, rectangle(5, 5)),
rectangle(50, 50))
between_fields = logical_and(field_area, logical_not(field_mask))
# Find the roads and separate the fields
# Separate out into smaller blocks
print('Separating Fields in image')
for stride in [100, 200, 400]: # pixels
num_row_strides = int_(between_fields.shape[0]/stride)
num_col_strides = int_(between_fields.shape[1]/stride)
r_stride = int_(between_fields.shape[0]/num_row_strides)
c_stride = int_(between_fields.shape[1]/num_col_strides)
for r in range(num_row_strides+1):
for c in range(num_col_strides+1):
h, theta, d = hough_line(between_fields[r*r_stride:(r+1)*r_stride,
c*c_stride:(c+1)*c_stride])
threshold = 0 #0.0005*max(h)
h, theta, d = hough_line_peaks(h, theta, d, min_distance=20,
threshold=threshold)
for n in range(len(theta)):
if abs(theta[n]) < 0.1 or abs(theta[n]) > ((pi/2) - 0.1):
draw_hough_line(field_mask[r*r_stride:(r+1)*r_stride,
16 / 29

c*c_stride:(c+1)*c_stride], d[n], theta[n])
# do a few small openings
field_mask = binary_opening(field_mask, rectangle(1, 5))
field_mask = binary_opening(field_mask, rectangle(5, 1))
imsave(fname_template.format('segmented_fields', 'png'), field_mask)
# Label fields
field_mask = label(field_mask, 4, 0) + 1
remove_small_objects(field_mask, 100, 1, True)
field_props = regionprops(field_mask)
# Write field_props to csv file
out_file = open(fname_template.format('field_props', 'csv'), 'w')
print('label,area,center_row,center_col,nw_col,nw_row,se_col,se_row', file=out_file)
for prop in field_props:
if (prop.area > 100) and (prop.area < 700):
print(','.join([str(x) for x in [
prop.label,
prop.area,
prop.centroid[0],
prop.centroid[1],
prop.bbox[0],
prop.bbox[1],
prop.bbox[2],
prop.bbox[3],
]]), file=out_file)
else:
field_mask[where(field_mask == prop.label)] = 0
# Visualize Data
plt.figure()
plt.imshow(field_mask)
'''
for prop in field_props:
plt.annotate(str(prop.label), xy=array([prop.centroid[1], prop.centroid[0]]))
'''
plt.show()
# Save the field mask
save_file_name = fname_template.format('field_mask', 'png')
imsave(save_file_name, uint16(field_mask))
8.2.2 Data Extraction Code
"""
Collates crop data and creates a descriptive csv file
"""
from numpy import array, where, mean, var
from skimage.io import imread, use_plugin
17 / 29

from os import listdir
from pandas import read_csv
import matplotlib.pyplot as plt
from landsatutil.scene import LandsatScene
# Change skimage plugin
use_plugin('freeimage')
# Very small area
nw_corner = array([396210, 4175310])
se_corner = array([404460, 4167150])
se_corner[1])
# Open the field mask
fname_field_mask = fname_template.format('field_mask', 'png')
field_mask = imread(fname_field_mask)
# Open the field properties file
fname_field_props = fname_template.format('field_props', 'csv')
field_props = read_csv(fname_field_props)
# Create an output file
result = open(fname_template.format('field_data', 'csv'), 'w')
print(','.join(['label', 'year', 'month', 'day', 'hour',
'b1_ref', 'b1_var',
'b2_ref', 'b2_var',
'b3_ref', 'b3_var',
'b4_ref', 'b4_var',
'b5_ref', 'b5_var',
'b6_ref', 'b6_var',
'b7_ref', 'b7_var'
]), file=result)
# Open a landsat scene for the specified years and write data to csv
# Convert year list items to stings
year_list = ['2008', '2009', '2010', '2011']
# Get list of archives in directory
archive_list = listdir('tmp/')
# Get all files with the year specified
archive_list = [file for file in archive_list if file[9:13] in year_list]
for line in field_props.iterrows():
props = line[1]
field_label = int(props['label'])
print('Collecting Data for Field: {0:d}'.format(field_label))
field_indices = where(field_mask == field_label)
nw_corner_field = array([nw_corner[0] + 30*(props['nw_row']),
nw_corner[1] - 30*(props['nw_col'])])
18 / 29

se_corner_field = array([nw_corner[0] + 30*(props['se_row']),
nw_corner[1] - 30*(props['se_col'])])
for archive_name in archive_list:
fscene = LandsatScene(archive_name)
year = fscene.year
month = fscene.month
day = fscene.day
hour = fscene.hour
reflectance = []
for band in range(1, 8):
field_reflectance = fscene.get_band_subimage(band, nw_corner_field,
se_corner_field)
single_field_mask = field_mask[props['nw_col']:props['se_col'],
props['nw_row']:props['se_row']]/field_label
field_reflectance2 = field_reflectance * single_field_mask
reflectance.append(mean(field_reflectance2[where(field_reflectance2 !=
0)]))
reflectance.append(var(field_reflectance2[where(field_reflectance2 !=
0)]))
# Plots
#f, ax = plt.subplots(1, 3)
#ax[0].imshow(field_reflectance)
#ax[1].imshow(single_field_mask)
#ax[2].imshow(field_reflectance2)
#plt.show()
data_list = [field_label, year, month, day, hour] + reflectance
data_list = [str(x) for x in data_list]
print(','.join(data_list), file=result)
8.2.3 Prediction and Evaluation Code
from numpy import array, zeros
from numpy import mean, var
from random import sample
from pandas import read_csv
from sklearn.svm import SVC
# Very small area
nw_corner = array([396210, 4175310])
se_corner = array([404460, 4167150])
se_corner[1])
# Organize data
explanatory_variables = [
'month',
'b1_ref', 'b1_var',
19 / 29

'b2_ref', 'b2_var',
'b3_ref', 'b3_var',
'b4_ref', 'b4_var',
'b5_ref', 'b5_var',
'b6_ref', 'b6_var',
'b7_ref', 'b7_var',
]
outcome_variable = 'CropTruth'
data_set = read_csv(fname_template.format('field_data_truth', 'csv'))
data_matrix = data_set[explanatory_variables].as_matrix()
outcome_truth = data_set[outcome_variable].as_matrix()
# Select random indices for each trial
n_trials = 100
train_fraction = 0.8
trial_outcomes = zeros(n_trials)
data_length = data_set.shape[0]
num_samples = int(data_length * train_fraction)
num_tests = data_length - num_samples
all_indices = range(data_length)
# Evaluate try a few time to see how well the crops can be predicted.
for trial in range(n_trials):
# Divide data array into training set and
training_indices = sample(all_indices, num_samples)
training_indices.sort()
test_indices = [x for x in all_indices if x not in training_indices]
test_indices.sort()
# Create the training data set
sampled_training_data = data_matrix[training_indices]
sampled_training_outcomes = outcome_truth[training_indices]
sampled_test_data = data_matrix[test_indices]
sampled_test_outcomes = outcome_truth[test_indices]
# Predict outcomes and save values
model = SVC()
model.fit(sampled_training_data, sampled_training_outcomes)
modeled_outcome = model.predict(sampled_test_data)
#print(modeled_outcome)
# Figure out how well the model has done
percent_correct = sum(modeled_outcome == sampled_test_outcomes) / num_tests
trial_outcomes[trial] = percent_correct
print('After {0} trials, correct outcome stats:ntMean: {1}ntVar: {2}'.format(
n_trials, mean(trial_outcomes), var(trial_outcomes)
))
20 / 29

8.3 Utility Libraries
8.3.1 Landsat Data Utilities
"""
Landsat scene extraction data abstraction utilities.
"""
# Imports
import tarfile
from numpy import array, abs, float_
from numpy import int32 as int_
from skimage.io import imread
from os.path import isdir, basename, join
# Constants
TMP_DIR = 'tmp'
class LandsatScene(object):
def __init__(self, archive_path):
"""
Create a LandsatScene object from a Landsat data product archive
:param archive_path: path to the archive
:return: LandsatScene object
"""
# Check if the archive has already been extracted
self.archive_name = join(TMP_DIR, basename(archive_path).split('.')[0])
if not isdir(self.archive_name):
archive = tarfile.open(archive_path)
archive.extractall(self.archive_name)
archive.close()
# Open the archive
file_list = listdir(self.archive_name)
try:
self.metadata_file = [name for name in file_list if 'MTL.txt' in name]
[0]
except IndexError:
print('Could not find metadata file in archive')
# Read the metadata file
self._read_metadata()
# Get the scene id
self.scene_id = self.metadata['METADATA_FILE_INFO/LANDSAT_SCENE_ID']
# Get image size info pixel
self.pixel_size =
21 / 29

float(self.metadata['PROJECTION_PARAMETERS/GRID_CELL_SIZE_REFLECTIVE'])
# Get the number of pixels in the images
self.image_size = array([
int_(self.metadata['PRODUCT_METADATA/REFLECTIVE_SAMPLES']),
int_(self.metadata['PRODUCT_METADATA/REFLECTIVE_LINES']),
], dtype=int_)
# Get UTM (per unit pixel) coordinate extents (4x2 matrix with columns NW,
NE, SW, SE and rows X, Y)
self.coords = array([
[
float(self.metadata['PRODUCT_METADATA/CORNER_UL_PROJECTION_X_PRODUCT']),
float(self.metadata['PRODUCT_METADATA/CORNER_UL_PROJECTION_Y_PRODUCT']),
],
[
float(self.metadata['PRODUCT_METADATA/CORNER_UR_PROJECTION_X_PRODUCT']),
float(self.metadata['PRODUCT_METADATA/CORNER_UR_PROJECTION_Y_PRODUCT']),
],
[
float(self.metadata['PRODUCT_METADATA/CORNER_LL_PROJECTION_X_PRODUCT']),
float(self.metadata['PRODUCT_METADATA/CORNER_LL_PROJECTION_Y_PRODUCT']),
],
[
float(self.metadata['PRODUCT_METADATA/CORNER_LR_PROJECTION_X_PRODUCT']),
float(self.metadata['PRODUCT_METADATA/CORNER_LR_PROJECTION_Y_PRODUCT']),
],
])/self.pixel_size
# Get the radiance correction functions
n = 1
self.band_correction = [lambda x: 0]
while True:
# Get the key names
data_mult_key =
'RADIOMETRIC_RESCALING/RADIANCE_MULT_BAND_{0:d}'.format(n)
data_add_key = 'RADIOMETRIC_RESCALING/RADIANCE_ADD_BAND_{0:d}'.format(n)
# Make sure that the key name exists
if data_mult_key not in self.metadata.keys():
break
print(data_mult_key)
# create coefficients for linear equation m*x + b
m = float(self.metadata[data_mult_key])
b = float(self.metadata[data_add_key])
self.band_correction.append(lambda x: m*x + b)
22 / 29

n += 1
# Get the date of the scene
date_str = self.metadata['PRODUCT_METADATA/DATE_ACQUIRED']
date = [int_(x) for x in date_str.split('-')]
self.year = date[0]
self.month = date[1]
self.day = date[2]
# Get the time of the scene
time_str = self.metadata['PRODUCT_METADATA/SCENE_CENTER_TIME'].split('.')[0]
time = [int_(x) for x in time_str.split(':')]
self.hour = time[0]
self.minute = time[1]
self.second = time[2]
def _read_metadata(self):
"""
Extract contents of metadata file into a dictionary
"""
# Create Empty Dictionary
self.metadata = {}
# Set group to None
group = 'NONE'
# Open the data file
md_file = open(join(self.archive_name, self.metadata_file), 'r')
# Extract data from each line
for line in md_file:
line_items = line.split('=')
if len(line_items) != 2:
continue
param = line_items[0].strip(' "n')
value = line_items[1].strip(' "n')
if param == 'GROUP':
group = value
elif param == 'END_GROUP':
pass
else:
self.metadata['{0}/{1}'.format(group, param)] = value
def coords_to_pixel(self, coords):
"""
Function for interpolating the pixel location of the given UTM coordinates
:param coords: array with [x, y] coordinates (meters)
:return: pixel coordinates as an array [r, c]
"""
# Divide Through by Pixel Size and subtract offset
23 / 29

# negative values in the y must be flipped as image and utm y coordinates
# are in opposite directions
pixel_coords = abs(coords/self.pixel_size - self.coords[0, :])
return int_(pixel_coords)
def get_band_subimage(self, band, nw_coords, se_coords, convert=True):
"""
Gets a sub-image from the specified band from the north west coordinates
to the south east coordinates.
:param band: Band of interest (1: blue, 2: green, etc)
:param nw_coords: North west coordinates in UTM (meters)
:param se_coords: South east coordinates in UTM (meters)
:param convert: If true, converts the image to radiance
:return: Scene subimage at at the desired coordinates as numpy float array
"""
# Read the file
image_file = '{0}/{1}/{1}_B{2:d}.TIF'.format(TMP_DIR, self.scene_id, band)
band_image = float_(imread(image_file))
# Get pixel coordinates
nw_pixel = self.coords_to_pixel(nw_coords)
se_pixel = self.coords_to_pixel(se_coords)
# Truncate
subimage = band_image[nw_pixel[1]:se_pixel[1], nw_pixel[0]:se_pixel[0]]
# Convert to radiance if needed
if convert:
return self.band_correction[band](subimage)
else:
return subimage
8.3.2 Coordinate Calculations
Code used from [5]
"""
Simple calculations for converting latitude and longitude to meters
"""
# Imports
from math import pi, cos, sin, pow, tan
R_EARTH = (6.371*10e6)/2.0
def dist_lat(lat_diff):
"""
Calculates the distance in meters along a latitude difference
:param lat_diff: latitude difference in degrees
:return: distance in meters
24 / 29

"""
return lat_diff*(pi/180.0)*R_EARTH
def dist_lon(lon_diff, lat):
"""
Calculates the distance in meters along a longitude difference at a given
latitude
:param lon_diff: longitude difference in degrees
:param lat: latitude to calculate distance in degrees
:return: distance along the longitude distance in meters
"""
radius = cos(lat*(pi/180.0))*R_EARTH
return lon_diff*(pi/180.0)*radius
def utm_to_latlon(zone, easting, northing, n_hemisphere=True):
"""
Converts UTM coordinates (meters) to latitude and longitude
:param zone: UTM zone
:param easting: UTM easting parameter
:param northing: UTM northing parameter
:param n_hemisphere:
:return: decimal (latitude, longitude)
"""
if not n_hemisphere:
northing = 10000000 - northing
a = 6378137
e = 0.081819191
e1sq = 0.006739497
k0 = 0.9996
arc = northing / k0
mu = arc / (a * (1 - pow(e, 2) / 4.0 - 3 * pow(e, 4) / 64.0 - 5 * pow(e, 6) /
256.0))
ei = (1 - pow((1 - e * e), (1 / 2.0))) / (1 + pow((1 - e * e), (1 / 2.0)))
ca = 3 * ei / 2 - 27 * pow(ei, 3) / 32.0
cb = 21 * pow(ei, 2) / 16 - 55 * pow(ei, 4) / 32
cc = 151 * pow(ei, 3) / 96
cd = 1097 * pow(ei, 4) / 512
phi1 = mu + ca * sin(2 * mu) + cb * sin(4 * mu) + cc * sin(6 * mu) + cd * sin(8
* mu)
n0 = a / pow((1 - pow((e * sin(phi1)), 2)), (1 / 2.0))
r0 = a * (1 - e * e) / pow((1 - pow((e * sin(phi1)), 2)), (3 / 2.0))
25 / 29

fact1 = n0 * tan(phi1) / r0
_a1 = 500000 - easting
dd0 = _a1 / (n0 * k0)
fact2 = dd0 * dd0 / 2
t0 = pow(tan(phi1), 2)
Q0 = e1sq * pow(cos(phi1), 2)
fact3 = (5 + 3 * t0 + 10 * Q0 - 4 * Q0 * Q0 - 9 * e1sq) * pow(dd0, 4) / 24
fact4 = (61 + 90 * t0 + 298 * Q0 + 45 * t0 * t0 - 252 * e1sq - 3 * Q0 * Q0) *
pow(dd0, 6) / 720
lof1 = _a1 / (n0 * k0)
lof2 = (1 + 2 * t0 + Q0) * pow(dd0, 3) / 6.0
lof3 = (5 - 2 * Q0 + 28 * t0 - 3 * pow(Q0, 2) + 8 * e1sq + 24 * pow(t0, 2)) *
pow(dd0, 5) / 120
_a2 = (lof1 - lof2 + lof3) / cos(phi1)
_a3 = _a2 * 180 / pi
latitude = 180 * (phi1 - fact1 * (fact2 + fact3 + fact4)) / pi
if not n_hemisphere:
latitude = -latitude
longitude = ((zone > 0) and (6 * zone - 183.0) or 3.0) - _a3
return latitude, longitude
8.3.3 Hough Line Drawing Code
"""
Utilities for segmenting features from landsat data
"""
from numpy import cos, sin, pi, int_
from skimage.draw import line
def draw_hough_line(image, dist, theta, color=0):
"""
Draws a line described by the hough transform to an image
:param image: Image to draw on
:param dist: Hough transform distance
:param theta: Hough transform angle
:param color: intensity to draw line
"""
rows, cols = image.shape
if abs(theta) < pi/4:
# Find the x (col) intercepts
x0 = int_(dist/cos(theta))
26 / 29

x1 = int_(x0 - rows * sin(theta))
intercepts = (0, x0, rows, x1)
else:
# Find the y (row) intercepts
y0 = int_(dist/sin(theta))
y1 = int_(y0 + cols * cos(theta))
intercepts = (y0, 0, y1, cols)
r, c = line(*intercepts)
# Check to make sure each point stays in the image bounds and draw it
for n in range(r.size):
if r[n] >= 0 and c[n] >= 0:
if r[n] < rows and c[n] < cols:
image[r[n], c[n]] = color
8.3.4 Temporal Manipulation Utilities
"""
Tools for temporal data extraction
"""
# Imports
from os.path import join
from numpy import min, max, average, zeros
from skimage.util import img_as_uint
from .scene import LandsatScene
def collect_bands(band, nw_coords, se_coords, year_list, directory):
"""
Collects sub-images of each band for each year and puts them
into a 3-dimensional array with the 3rd dimension being time
:param band: band of interest
:param nw_coords: UTM coordinates (meters) of the north west corner of interest
:param se_coords; UTM coordinates (meters) of the south east corner of interest
:param year_list: list of years to collect
:param directory: directory to search for available datasets
:return: A three dimensional numpy array with the images stacked in dimension 3
"""
# Convert year list items to stings
year_list = [str(x) for x in year_list]
# Get list of archives in directory
archive_list = listdir(directory)
# Get all files with the year specified
archive_list = [file for file in archive_list if file[9:13] in year_list]
# Create list to hold subimages before fusion
subimage_list = []
27 / 29

for archive in archive_list:
scene = LandsatScene(join(directory, archive))
subimage_list.append(scene.get_band_subimage(band, nw_coords, se_coords))
# Make sure that all images have the same shape (they should) but
# truncate extra values anyway (just in-case)
y_extent = min([image.shape[0] for image in subimage_list])
x_extent = min([image.shape[1] for image in subimage_list])
# Average on the fly
temporal_image = zeros((y_extent, x_extent))
for n, image in enumerate(subimage_list):
temporal_image = ((temporal_image * n) + image[0:y_extent, 0:x_extent])/(n +
1)
return temporal_image
def compress_temporal_image(temporal_image):
"""
Averages frames in the image and then normalizes them to values between [0,
2^16]
:param temporal_image: Image to compress
:return: 2D numpy image array of type ubyte
"""
# Normalize the image to values between [-1 and 1]
image_center = average([min(temporal_image), max(temporal_image)])
image_shift = temporal_image - image_center
image_normalized = image_shift/max([-min(image_shift), max(image_shift)])
# Change image to a ubyte and return
return img_as_uint(image_normalized)
8.3.5 CropScape Tools
from .earth_calc import utm_to_latlon
def get_crop_data(filename, shape, zone, nw_corner, se_corner, year,
n_hemisphere=True):
"""
Downloads cropscape data to an image file
:param filename: Name of the image file to save to (will add .png extension)
:param shape: pixel shape of the image
:param zone: UTM coordinate zone
:param nw_corner: northwest corner in UTM coordinates (easting, northing)
(meters)
:param se_corner: southeast corner in UTM coordinates (easting, northing)
(meters)
28 / 29

:param year: year to extract data for
:return: image of crop data
"""
nw_lat, nw_lon = utm_to_latlon(zone, nw_corner[0], nw_corner[1],
n_hemisphere=n_hemisphere)
se_lat, se_lon = utm_to_latlon(zone, se_corner[0], se_corner[1],
n_hemisphere=n_hemisphere)
if '.png' not in filename:
filename += '.png'
request = 'http://129.174.131.7/cgi/wms_cdlall.cgi?
SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=cdl_{0:d}&STYLES=&SRS=EPSG:4326&BBOX
={1:f},{2:f},{3:f},{4:f}&WIDTH={5:d}&HEIGHT={6:d}&FORMAT=image/png'.format(
year, nw_lon, se_lat, se_lon, nw_lat, shape[0], shape[1]
)
urlretrieve(request, filename)
crop_data = imread(filename)
return crop_data
29 / 29

Estimation of Crop Acreage From Satellite Imagery -- EENG 510 -- Daniel Kuntz

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Estimation of Crop Acreage From Satellite Imagery -- EENG 510 -- Daniel Kuntz

Similar to Estimation of Crop Acreage From Satellite Imagery -- EENG 510 -- Daniel Kuntz (20)

Estimation of Crop Acreage From Satellite Imagery -- EENG 510 -- Daniel Kuntz