DEEP LEARNING IN
AGRICULTURE
Erik Andrejko	

Head, Data Science —The Climate Corporation	

SiliconValley Machine Learning Meetup	

Mar 28 2014
OUTLINE
• The Climate Corporation	

• The Agricultural Challenge	

• The Role of Deep Learning
THE CLIMATE 

CORPORATION
OUR MISSION
To help all the world’s
people and businesses
manage and adapt to
climate change…	

starting with agriculture
PROTECT & IMPROVE
MP
TWI
GROWERAPPLICATIONS
Yield Potential
Profitability
Yield Expectation
TWI
Total Weather Insurance: parametric
supplemental crop insurance product
MP
Government subsidized loss-adjusted
insurance program, integrated risk
management
GROWER APPLICATIONS
Collection of grower management advisors
using agronomic and climatological models
PROTECT
IMPROVE
The Climate Corporation aims to help farmers around the world 

protect and improve their farming operations & profitability.
THE AGRICULTURAL
CHALLENGE
WORLD POPULATION
Alexandratos N, Bruinsma J. 2012.World agriculture towards 2030/2050, the 2012 revision. ESA Working Paper No.
12-03, June 2012. Rome: Food and Agriculture Organization of the United Nations (FAO)
AGRICULTURAL DEMAND
POPULATION
MORE PEOPLE
CONSUMPTIONHIGHER DEMAND / CAPITA
NORMAN BORLAUG
IS CREDITED WITH SAVING
BILLION LIVES1
GREEN REVOLUTION
YIELD INCREASES
Ray DK, Mueller ND,West PC, Foley JA. 2013.YieldTrends Are Insufficient to Double Global Crop Production by 2050. PLoS ONE 8(6), doi:10.1371/journal.pone.0066428.
crop yield must
I N C R E A S E
60% to meet
demand
b y 2 0 5 0Ray DK, Mueller ND,West PC, Foley JA. 2013.YieldTrends Are Insufficient to Double Global Crop Production by 2050. PLoS ONE 8(6), doi:10.1371/journal.pone.0066428.
*
EXAMPLE: BEEF
2,500M 700M
Source: FAO and USDA (assuming 2kg of cereal of 500g of beef)
Current worldwide cereal
production (tonnes)
Corn demand to support
current US per-capita beef
consumption for a
population of 7B (tonnes)
IT’S POSSIBLE
I now say that the
world has the
technology… to feed
on a sustainable basis 

a population of 

10 billion people. 	

”
“
Normal Borlaug
DATA SCIENCE MEETS
AGRICULTURE
NEXT REVOLUTION ?
INTENSIFY
Apply breeding, fertilization 

to increase yields.
OPTIMIZE
Apply data science to optimize
management.
GREEN REVOLUTION
GREEN DATA REVOLUTION
1960 –
2010 –
BIOTECH
Marker assisted selection.
BIOTECH REVOLUTION
1980 –
DATA SCIENCE
Computer Science
Domain Science Statistics
What is important?
How can it be built?
How can predictions be made?
SCIENTIFIC DATA SCIENCE
use software engineering to
enable domain science
maximizing use of data
MACHINE LEARNING
GENETICS PRACTICES ENVIRONMENT YIELD
T	

R	

A	

I	

N
T	

E	

S	

T
predictive
model
problem: curse of dimensionality
YIELD OPTIMIZATION
OPTIMIZED YIELD
Yield optimized for
environment by optimization of
genetics and management using
predictive model.
YIELD
Yield optimized for
environment by optimization 

of genetics and management
traditional practices.
EASY! RIGHT?
unfortunately, no
CHALLENGES
Spatio-­‐Temporal	
  Data
Sparse	
  Data
Latent	
  Features
Curse	
  of	
  Dimensionality
DATA CHALLENGES LEARNING CHALLENGES
Missing	
  Data Multi-­‐task	
  Learning
Noisy	
  Data
DATA POTENTIAL
YIELD MONITOR DATA
14B OBSERVATIONS
REMOTE SENSING DATA
260B OBSERVATIONS
WEATHER DATA
20B OBSERVATIONS
one season, one crop, one country
FEATURE ENGINEERING
Zea mays (corn)
Genetics,	
  Environment,	
  Practices
Soil	
  Processes
Nutrient	
  Processes
Crop	
  Processes
Yield
LATENT SPACE
LATENT FEATURE SPACE
Environment, Genetics and Practices
Physical Processes
Yield Outcome
}{Engineered
Features
Learned
Features
CAUSAL DESCRIPTION
THE ROLE OF DEEP
LEARNING
FEATURE LEARNING
genetics, environment and practices
soil processes
nutrient processes
crop processes
yield
Hierarchical
Dimensionality
Reduction
Deep Neural
Network
yield
genetics, environment and practices
hiddenlayers
physicalmodels
SPATIAL DATA
Lee, Honglak, et al. "Convolutional deep belief networks for scalable unsupervised learning of hierarchical
representations." Proceedings of the 26th Annual International Conference on Machine Learning.ACM, 2009.
CONVOLUTIONAL DBN
Hierarchical representation of spatial data	

!
High-dimensional, scalable visible layer	

!
Unsupervised hierarchical learning
MULTI-TASK LEARNING
DEEP NEURAL NETWORK
Hidden layers (latent features) shared
across tasks	

!
Multi-task informs latent features
deep neural network in multi-task setting
y
genetics, environment and practices
HiddenLayers
w
Deng, Li, Geoffrey Hinton, and Brian Kingsbury. "New types of deep neural network learning for speech
recognition and related applications:An overview." Acoustics, Speech and Signal Processing (ICASSP), 2013
IEEE International Conference on. IEEE, 2013.
MISSING DATA
Hinton, Geoffrey E., Simon Osindero, andYee-WhyeTeh. "A fast learning algorithm for deep belief nets."
Neural computation 18.7 (2006): 1527-1554.
DEEP BELIEF NETWORK
Greedy layer-wise training algorithm	

!
Robust to noisy inputs	

!
Generative process (MRF)
Alternating Gibbs sampling in deep belief network
additional applications of deep-learning in agriculture
OTHER APPLICATIONS
Crop	
  identification
Disease	
  detection
Practice	
  classifications
Remote	
  sensing
Image	
  segmentation	
  /	
  
clustering
Nutrient	
  deficiency	
  
detection
Cloud	
  detection Environment	
  classification
PROTECT & IMPROVE
REDUCE RISK INCREASE YIELDS
Goal: optimize global food production
POSSIBILITIES
QUESTIONS ?
erik@climate.com

Deep Learning In Agriculture

  • 1.
    DEEP LEARNING IN AGRICULTURE ErikAndrejko Head, Data Science —The Climate Corporation SiliconValley Machine Learning Meetup Mar 28 2014
  • 2.
    OUTLINE • The ClimateCorporation • The Agricultural Challenge • The Role of Deep Learning
  • 3.
  • 4.
    OUR MISSION To helpall the world’s people and businesses manage and adapt to climate change… starting with agriculture
  • 5.
    PROTECT & IMPROVE MP TWI GROWERAPPLICATIONS YieldPotential Profitability Yield Expectation TWI Total Weather Insurance: parametric supplemental crop insurance product MP Government subsidized loss-adjusted insurance program, integrated risk management GROWER APPLICATIONS Collection of grower management advisors using agronomic and climatological models PROTECT IMPROVE The Climate Corporation aims to help farmers around the world 
 protect and improve their farming operations & profitability.
  • 7.
  • 8.
    WORLD POPULATION Alexandratos N,Bruinsma J. 2012.World agriculture towards 2030/2050, the 2012 revision. ESA Working Paper No. 12-03, June 2012. Rome: Food and Agriculture Organization of the United Nations (FAO)
  • 9.
  • 10.
    NORMAN BORLAUG IS CREDITEDWITH SAVING BILLION LIVES1 GREEN REVOLUTION
  • 11.
    YIELD INCREASES Ray DK,Mueller ND,West PC, Foley JA. 2013.YieldTrends Are Insufficient to Double Global Crop Production by 2050. PLoS ONE 8(6), doi:10.1371/journal.pone.0066428.
  • 12.
    crop yield must IN C R E A S E 60% to meet demand b y 2 0 5 0Ray DK, Mueller ND,West PC, Foley JA. 2013.YieldTrends Are Insufficient to Double Global Crop Production by 2050. PLoS ONE 8(6), doi:10.1371/journal.pone.0066428. *
  • 13.
    EXAMPLE: BEEF 2,500M 700M Source:FAO and USDA (assuming 2kg of cereal of 500g of beef) Current worldwide cereal production (tonnes) Corn demand to support current US per-capita beef consumption for a population of 7B (tonnes)
  • 14.
    IT’S POSSIBLE I nowsay that the world has the technology… to feed on a sustainable basis 
 a population of 
 10 billion people. ” “ Normal Borlaug
  • 15.
  • 16.
    NEXT REVOLUTION ? INTENSIFY Applybreeding, fertilization 
 to increase yields. OPTIMIZE Apply data science to optimize management. GREEN REVOLUTION GREEN DATA REVOLUTION 1960 – 2010 – BIOTECH Marker assisted selection. BIOTECH REVOLUTION 1980 –
  • 17.
    DATA SCIENCE Computer Science DomainScience Statistics What is important? How can it be built? How can predictions be made? SCIENTIFIC DATA SCIENCE use software engineering to enable domain science maximizing use of data
  • 18.
    MACHINE LEARNING GENETICS PRACTICESENVIRONMENT YIELD T R A I N T E S T predictive model problem: curse of dimensionality
  • 19.
    YIELD OPTIMIZATION OPTIMIZED YIELD Yieldoptimized for environment by optimization of genetics and management using predictive model. YIELD Yield optimized for environment by optimization 
 of genetics and management traditional practices.
  • 20.
  • 21.
    CHALLENGES Spatio-­‐Temporal  Data Sparse  Data Latent  Features Curse  of  Dimensionality DATA CHALLENGES LEARNING CHALLENGES Missing  Data Multi-­‐task  Learning Noisy  Data
  • 22.
    DATA POTENTIAL YIELD MONITORDATA 14B OBSERVATIONS REMOTE SENSING DATA 260B OBSERVATIONS WEATHER DATA 20B OBSERVATIONS one season, one crop, one country
  • 23.
    FEATURE ENGINEERING Zea mays(corn) Genetics,  Environment,  Practices Soil  Processes Nutrient  Processes Crop  Processes Yield LATENT SPACE
  • 24.
    LATENT FEATURE SPACE Environment,Genetics and Practices Physical Processes Yield Outcome }{Engineered Features Learned Features
  • 25.
  • 26.
    THE ROLE OFDEEP LEARNING
  • 27.
    FEATURE LEARNING genetics, environmentand practices soil processes nutrient processes crop processes yield Hierarchical Dimensionality Reduction Deep Neural Network yield genetics, environment and practices hiddenlayers physicalmodels
  • 28.
    SPATIAL DATA Lee, Honglak,et al. "Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations." Proceedings of the 26th Annual International Conference on Machine Learning.ACM, 2009. CONVOLUTIONAL DBN Hierarchical representation of spatial data ! High-dimensional, scalable visible layer ! Unsupervised hierarchical learning
  • 29.
    MULTI-TASK LEARNING DEEP NEURALNETWORK Hidden layers (latent features) shared across tasks ! Multi-task informs latent features deep neural network in multi-task setting y genetics, environment and practices HiddenLayers w Deng, Li, Geoffrey Hinton, and Brian Kingsbury. "New types of deep neural network learning for speech recognition and related applications:An overview." Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on. IEEE, 2013.
  • 30.
    MISSING DATA Hinton, GeoffreyE., Simon Osindero, andYee-WhyeTeh. "A fast learning algorithm for deep belief nets." Neural computation 18.7 (2006): 1527-1554. DEEP BELIEF NETWORK Greedy layer-wise training algorithm ! Robust to noisy inputs ! Generative process (MRF) Alternating Gibbs sampling in deep belief network
  • 31.
    additional applications ofdeep-learning in agriculture OTHER APPLICATIONS Crop  identification Disease  detection Practice  classifications Remote  sensing Image  segmentation  /   clustering Nutrient  deficiency   detection Cloud  detection Environment  classification
  • 32.
    PROTECT & IMPROVE REDUCERISK INCREASE YIELDS Goal: optimize global food production POSSIBILITIES
  • 33.