This document discusses using hyperspectral imaging and machine learning to classify crop types and growth stages from their spectral signatures in order to help address global food security issues. It presents research using a dataset containing hyperspectral images and labels of crop type and stage. Various machine learning models are tested for crop and stage classification, including multiclass, multilabel, chained, and multitask classifiers. The multitask classifier that jointly predicts crop type and stage achieved the best results, though upsampling data reduced its performance. Further improvements could come from analyzing larger geographic datasets and combining strengths of different model types.

1. CROP PHENOTYPING
THROUGH SPECTRAL
CLASSIFICATION
General Assembly Data Science Immersive
Marta Ghiglioni

2. BACKGROUND
+ Food security is one of the
big challenges policy makers
worldwide are facing.
+ To feed 7.4+ billion people
sustainably we need high
quality data to inform policy
makers to drive ultra precision
farming at global scale.

3. HYPERSPECTRAL IMAGING

4. HYPERSPECTRAL IMAGING
Becoming a more popular data
source because:
+ Scale access through low orbit
satellite imaging
+ Investigating ways to
automatize the insights
extraction

5. PROBLEM STATEMENT
+ Is it possible to identify
crop types and their
development stage through
their spectral signature?
+ How can we use these
information for food security?

6. GHISA
Global
Hyperspectral
Imaging
Spectroscopy of
Agricultural-Crops & Vegetation

7. CROP SPECTRUM

8. DATA SOURCES
FEATURES
+ Hyperspectral images
EO-1, Hyperion hyperspectral data
+ 242 bands, each of 10 nm bandwidth in
the 400-2500 nm range
LABELS
+ Crop type:
USDA Cropland Data Layer
+ Stage type:
Center for Sustainability and
the Global Environment
(SAGE)

9. PREVIOUS WORKS
Scientific research on the data set, aimed
of identify the crop types and stage of
growth using the full images, and
modeling by region.
Accuracy reached was 93 %

10. PRELIMINARY ANALYSIS
+ After reading research papers I
decided to attempt to classify crop
type and stage using only their
spectral signature.
+ I also decided to focus my research on
the first portion of the spectrum
(bands 427 – 923). Indeed, these
bands are more suitable for a larger
scale of applications, easier to collect
and have a higher resolution.

11. DATASET
Emerge_VEarly
Early_Mid
Late
Critical
Mature_Senesc
0
100
200
300
400
500
600
700
800
900
Corn Cotton Rice Soybean Winter
Wheat
Categories Distribution
Emerge_VEarly Early_Mid Late Critical Mature_Senesc
CLASS UNBALANCE
+ SMOTE

12. PREPROCESSING
BLIND SOURCE SEPARATION
+ Independent Component Analysis
DIMENSIONALITY REDUCTION
+ Principal Component Analysis
+ Linear Discriminant Analysis

13. MODELS
ARCHITECTURES
+ Multiclass output is a single correct prediction
+ Multilabel labels are not mutually exclusive
+ Multitask the same model generates two or more outputs
ALGORITHMS
+ Multi-layer Perceptron Classifier
+ Convolutional Neural Network
+ Chained Classifier (Gradient Boost)

14. Predictions
Target
Multiclass Classifier
(Crop + Stage)
[0]
[…]
[23]
Input
X437
…
…
…
…
X923
[0, … 1, 0]
…
…
…
…
[0, 0, … 1]
Length = 23
Dense
Output
Activation = SoftMax
Loss = Binary Cross Entropy
Input Layer
SCORES
Balanced Accuracy score 79 %
Accuracy score 80 %

15. Multiclass Classifier

16. Predictions
Target
Multilabel Classifier
Crop & Stage
[0]
[…]
[10]
[1, … 1, 0]
…
…
…
…
[0, 0, … 1]
length = 10
1dConvolutional
+
1dMaxpooling
Dense
Flatten
Input
(50, 1)
X437
…
…
X923
Output
Activation = Sigmoid
Loss = Categorical Cross Entropy
SCORES
Balanced Accuracy score 81.2 %
Accuracy score 83.1 %

17. Multilabel Classifier
Most common unidentified
label combinations
More wrongful, nonsystematic predictions omitted in the slide
True labels Predicted labels
True labels Predicted labels

18. Predictions
Target
Multiclass Chained Classifier
Crop
Crop Class
[0]
[…]
[4]
Crop
Classifier
Stage
Classifier
Stage
Stage Class
[0]
[…]
[4]
Input
X437
…
…
…
…
X923
SCORES
Crop Accuracy score 76.06 %
Stage Accuracy score 79.1 %

19. Multiclass Chained Classifier

20. Predictions
Target
Multitask Classifier
Crop Class
[0, 1, 0, 0]
[…]
[1, 0, 0, 0]
Stage Class
[0, 0, 1, 0]
[…]
[1, 0, 0, 0]
Input
(50, 1)
X437
…
…
X923
Crop
Crop Stage
[1, …, 0] [0, … 1]
…
…
…
…
[1, …, 0] [0, … 1]
Length = 5 * 2
Stage

21. Multitask Classifier
+ Leveraging on stage as a tool to
predict crops with more accuracy.
+ Upsample data highly reduces the
performance of this model

22. Multitask Classifier
+ Leveraging on stage as a tool to
predict crops with more accuracy.
+ Up sample data highly reduces the
performance of this model

23. CONCLUSION
+ Multitask is a promising
architecture
+ ICA improves the results on fully
connected neural networks, but it
impacts negatively convolutional
networks.
+ Spectral data can be incredibly
insightful even just using the first
50 bands.

24. NEXT STEPS
+ Pull larger data sample,
train for results across
geographic scope (Central
Asia dataset available)
+ Pipeline of models,
combining strength of
different methods