Monitoring Biomass
Dynamics at Scale:
Emerging Trends and
Recent Successes
Presented at the
CIMMYT Workshop on Remote Sensing
for Agriculture

Budhendra Bhaduri
Corporate Research Fellow
R. Vatsavai, A. Cheriyadat, E. Bright

December 15, 2013
Mexico City, Mexico
Acknowledgement
People who do the real work
–
–
–
–
–
–
–
–
–

Eddie Bright
Raju Vatsavai
Anil Cheriyadat
Amy Rose
Marie Urban
Steve Fernandez
Mark Tuttle
Devin White
… and many others

People who make it possible
– Our sponsors
– …
Managed by UT-Battelle
for the Department of Energy
Managed by UT-Battelle
for the Department of Energy
Spatial refinement of LandScan Global

Managed by UT-Battelle
for the Department of Energy
“Developed Land Cover” Examples

Managed by UT-Battelle
for the Department of Energy
Addis Ababa, Ethiopia



2 Xeon Quad core 2.4GHz
CPUs + 4 Tesla GPUs +
48GB



Image analyzed (0.6m)
 40,000x40,000 pixels
(800 sq. km)
 RGB bands



Overall accuracy 93%
 Settlement class 89%
 Non-settlement class
94%



Total processing time

Managed by UT-Battelle
for the Department of Energy

 27 seconds
Scalable probabilistic approach
Bitemporal change
– Point based – at individual pixel
(or small neighborhood)
– Mostly univariate
– Multivariate techniques produce
multi-band change maps
– Mostly the output is continuous
(requires thresholding)

New probabilistic approach
– Model that data as probability
distribution
– Estimate the overlap between
two grids (distributions)
– Computationally efficient and
scalable
Managed by UT-Battelle
for the Department of Energy

t1

Highly overlapping (no-change) to
No overlap (change)

t2
Kacha Garhi Camp, Pakistan
Established 1980 for Afghan Refugees
QuickBird (2004 and 2009, 4B, 2.4m)

Managed by UT-Battelle
for the Department of Energy
Comparison of Performance

Difference

Probabilistic
Managed by UT-Battelle
for the Department of Energy

Ratio
SAR Change Detection
SAR Imagery during Ike – noise, spatial resolution (1.56m vs. 12.5m)

8/31/08
1.56m

9/13/08
12.5m

Flooded regions

Managed by UT-Battelle
for the Department of Energy
New Probabilistic Change Detection
Predicted changes have good correlation with ground-truth
Detected changes – Flooded regions

Managed by UT-Battelle
for the Department of Energy

NGA Flood Overlaid – Shows Good Correlation
Managed by UT-Battelle
for the Department of Energy
Managed by UT-Battelle
for the Department of Energy

14
Online Detection of Anomaly, Change and
Change Point from Space-Time Data

Potere, D., Feierabend, N., Bright, E., Strahler, A. “Walmart from Space: A New Source for
Land Cover Change Validation” Photogrametric Engineering and Remote Sensing. Vol 74. July
2008.

Managed by UT-Battelle
for the Department of Energy
June 20,2007
July 19,2007

Fargo,ND
Fargo,ND

Sunflower
Sunflower

Managed by UT-Battelle
for the Department of Energy

Corn
Corn

Soybeans
Soybeans
Geocomputation based strategy
Design and develop a robust and scalable spatiotemporal data mining framework
utilizing high resolution spatial and temporal data streams (MODIS and AWiFS)
Preprocessing

Change
detection

• Reprojection
• Atmospheric
corrections
• Time series
filtering

• Time series
prediction
• Unsupervised
multidimensional
geospatial image
clustering

Change
characterization
• Classification
• Phenology-based
• Crop Type-based

Peak

Google
Earth
Length of growing
season

Greenup
Onset

Managed by UT-Battelle
for the Department of Energy

Dormancy
Onset

Key features of crop phenology

NASA
World
Wind

Other
thin
clients
Gaussian Process Model
MODIS NDVI Time Series from Iowa
– 6 years (2001 – 2006)
– 23 observations per year

Trained for first 5 years and monitored last
year

Accuracy was 88% on a validation set
consisting of 97 labeled time series with 13
No Change
true changes
Variance
Predicted
Observed

Varun Chandola, Ranga Raju Vatsavai: Scalable Time Series
Change Detection for Biomass Monitoring Using Gaussian
Managed by UT-Battelle
Process. NASA CIDU 2010: 69-82 (One of the best
for the Department of Energy
papers, invited to SADM Journal).

Change
Wide area biomass monitoring in near
real time is becoming a reality
MODIS Tile (4800x4800 pixels)
– 23,040,000 time series
– 10 trillion at Global scale (432
land tiles)

FROST: An SGI Altrix ICE
8200 Cluster at ORNL
– 128 compute nodes each with 16
virtual cores and 24 GB of RAM

Multicore (multithreaded) and
Distributed (message passing)
computing strategy

Managed by UT-Battelle
for the Department of Energy

Serial

• 41,105 seconds (11.4
hours)

Threads (16)

• 5,872 seconds (1.6 hours)

MPI (96
nodes)

• 604 seconds (10 minutes)

MPI + Threads
• 34 seconds
(1536 cores)
Managed by UT-Battelle
for the Department of Energy
2005

2004

2003

2002

Managed by UT-Battelle
for the Department of Energy

2001

2000

25

100

NDVI

175

Apple Valley, CA – Wal-Mart Distribution Center
100

175

2003

2002

2001

2000

2004

2005

2003

2004

2005

Year

2002

2001

25

Managed by UT-Battelle
for the Department of Energy

2000

75

NDVI

150

225

2005

2004

2003

2002

2001

2000

25

100 175 250

Three Wal-Mart NDVI Time Series

Monitoring Biomass Dynamics at Scale: Emerging Trends and Recent Successes

  • 1.
    Monitoring Biomass Dynamics atScale: Emerging Trends and Recent Successes Presented at the CIMMYT Workshop on Remote Sensing for Agriculture Budhendra Bhaduri Corporate Research Fellow R. Vatsavai, A. Cheriyadat, E. Bright December 15, 2013 Mexico City, Mexico
  • 2.
    Acknowledgement People who dothe real work – – – – – – – – – Eddie Bright Raju Vatsavai Anil Cheriyadat Amy Rose Marie Urban Steve Fernandez Mark Tuttle Devin White … and many others People who make it possible – Our sponsors – … Managed by UT-Battelle for the Department of Energy
  • 3.
    Managed by UT-Battelle forthe Department of Energy
  • 4.
    Spatial refinement ofLandScan Global Managed by UT-Battelle for the Department of Energy
  • 5.
    “Developed Land Cover”Examples Managed by UT-Battelle for the Department of Energy
  • 6.
    Addis Ababa, Ethiopia  2Xeon Quad core 2.4GHz CPUs + 4 Tesla GPUs + 48GB  Image analyzed (0.6m)  40,000x40,000 pixels (800 sq. km)  RGB bands  Overall accuracy 93%  Settlement class 89%  Non-settlement class 94%  Total processing time Managed by UT-Battelle for the Department of Energy  27 seconds
  • 7.
    Scalable probabilistic approach Bitemporalchange – Point based – at individual pixel (or small neighborhood) – Mostly univariate – Multivariate techniques produce multi-band change maps – Mostly the output is continuous (requires thresholding) New probabilistic approach – Model that data as probability distribution – Estimate the overlap between two grids (distributions) – Computationally efficient and scalable Managed by UT-Battelle for the Department of Energy t1 Highly overlapping (no-change) to No overlap (change) t2
  • 8.
    Kacha Garhi Camp,Pakistan Established 1980 for Afghan Refugees QuickBird (2004 and 2009, 4B, 2.4m) Managed by UT-Battelle for the Department of Energy
  • 9.
    Comparison of Performance Difference Probabilistic Managedby UT-Battelle for the Department of Energy Ratio
  • 10.
    SAR Change Detection SARImagery during Ike – noise, spatial resolution (1.56m vs. 12.5m) 8/31/08 1.56m 9/13/08 12.5m Flooded regions Managed by UT-Battelle for the Department of Energy
  • 11.
    New Probabilistic ChangeDetection Predicted changes have good correlation with ground-truth Detected changes – Flooded regions Managed by UT-Battelle for the Department of Energy NGA Flood Overlaid – Shows Good Correlation
  • 12.
    Managed by UT-Battelle forthe Department of Energy
  • 13.
    Managed by UT-Battelle forthe Department of Energy 14
  • 14.
    Online Detection ofAnomaly, Change and Change Point from Space-Time Data Potere, D., Feierabend, N., Bright, E., Strahler, A. “Walmart from Space: A New Source for Land Cover Change Validation” Photogrametric Engineering and Remote Sensing. Vol 74. July 2008. Managed by UT-Battelle for the Department of Energy
  • 15.
    June 20,2007 July 19,2007 Fargo,ND Fargo,ND Sunflower Sunflower Managedby UT-Battelle for the Department of Energy Corn Corn Soybeans Soybeans
  • 16.
    Geocomputation based strategy Designand develop a robust and scalable spatiotemporal data mining framework utilizing high resolution spatial and temporal data streams (MODIS and AWiFS) Preprocessing Change detection • Reprojection • Atmospheric corrections • Time series filtering • Time series prediction • Unsupervised multidimensional geospatial image clustering Change characterization • Classification • Phenology-based • Crop Type-based Peak Google Earth Length of growing season Greenup Onset Managed by UT-Battelle for the Department of Energy Dormancy Onset Key features of crop phenology NASA World Wind Other thin clients
  • 17.
    Gaussian Process Model MODISNDVI Time Series from Iowa – 6 years (2001 – 2006) – 23 observations per year Trained for first 5 years and monitored last year Accuracy was 88% on a validation set consisting of 97 labeled time series with 13 No Change true changes Variance Predicted Observed Varun Chandola, Ranga Raju Vatsavai: Scalable Time Series Change Detection for Biomass Monitoring Using Gaussian Managed by UT-Battelle Process. NASA CIDU 2010: 69-82 (One of the best for the Department of Energy papers, invited to SADM Journal). Change
  • 18.
    Wide area biomassmonitoring in near real time is becoming a reality MODIS Tile (4800x4800 pixels) – 23,040,000 time series – 10 trillion at Global scale (432 land tiles) FROST: An SGI Altrix ICE 8200 Cluster at ORNL – 128 compute nodes each with 16 virtual cores and 24 GB of RAM Multicore (multithreaded) and Distributed (message passing) computing strategy Managed by UT-Battelle for the Department of Energy Serial • 41,105 seconds (11.4 hours) Threads (16) • 5,872 seconds (1.6 hours) MPI (96 nodes) • 604 seconds (10 minutes) MPI + Threads • 34 seconds (1536 cores)
  • 19.
    Managed by UT-Battelle forthe Department of Energy
  • 20.
    2005 2004 2003 2002 Managed by UT-Battelle forthe Department of Energy 2001 2000 25 100 NDVI 175 Apple Valley, CA – Wal-Mart Distribution Center
  • 21.
    100 175 2003 2002 2001 2000 2004 2005 2003 2004 2005 Year 2002 2001 25 Managed by UT-Battelle forthe Department of Energy 2000 75 NDVI 150 225 2005 2004 2003 2002 2001 2000 25 100 175 250 Three Wal-Mart NDVI Time Series

Editor's Notes

  • #7 MODIS 1 KM Landsat 30m SMT-16 meters
  • #9 Basic idea – abstract grids as a statistical distribution, then compare two grids (meaning their statistical distributions to detect changes) – as we are comparing distributions (a) – miss-registrations have less impact, (2) we can compare images of different resolutions.
  • #10 Different types of changes are highlighted – (a) growth, (b) removal
  • #11 Pixel based methods like difference and ratio produces highly noisy images and lot of changes are not real changes.
  • #12 Both noise and speckle noise will influence pixel based methods.
  • #13 NGA Flood Contours: Couldn’t find how they are generated, I will check again and update you soon.Highlight – good (spatial) correlation between predicted changes and NGA flood contour map