When modelling soil-landscape relationships we generally test a handful of statistical learners. Having limited data, we use cross-validation to select the best performing learner. In this study we evaluate the benefits of combining learners for soil prediction using stacked generalization. It consists of calibrating multiple learners and submitting them to 10-fold cross-validation. Cross-validation predictions are used as covariates in an interceptless linear regression of the target variable. Constrained to be non-negative, the estimated regression coefficients are the stacking weights expressing the importance of each learner. When making predictions, each learner is used in turn and the weights used to optimally combine multiple predictions into a single prediction. The data was downloaded from the national database maintained by Embrapa. The target variable was the soil iron content (g/kg). Covariates (p = 7) were constructed using soil profile data. The n = 22,981 records remaining after some data cleaning were split into calibration (n_cal = 16,086) and validation (n_val = 6,895) sets. Six learners were used: linear regression with stepwise selection (lm), multivariate adaptive regression splines (mars), regression random forest (rf), single-hidden-layer neural network (nnet), weighted k-nearest neighbor regression (knn) and support vector machine with polynomial kernel (svm). rf and knn severely overfitted the data, while lm, mars and svm were the most stable learners. The first two yielded the lowest absolute and squared errors (rmse < 45 g/kg) and explained more of the variance (ave ~ 0.6). mars, nnet and lm were the least biased learners (me ~ -0.1 g/kg), while svm was the most biased (me = -5.14 g/kg). lm explained the smallest amount of variance (ave = 0.49). rf received the largest stacking weight (w = 0.55), knn and svm received moderate weights (w ~ 0.2) and nnet and mars received the smallest weights (w < 0.1) -- lm was dropped from the stack (w = 0). Combining learners lowered all absolute and squared errors (rmse = 43.23 g/kg), yielded a considerably small bias (me = 0.53 g/kg), and explained the same amount of variance explained by rf (ave = 0.61). Staking learners was more beneficial than using the single best performing learner because it reduced generalization errors. The magnitude of the benefits seems to depend upon the diversity of learners (over and underfitting, biased and nonbiased). Besides, by using least squares regression to compute stacking weights we can estimate the prediction error variance of any combination of learners.

1. Stacked generalization of statistical
learners – a case study with soil iron
content in Brazil
Pedometrics 2017, Wageningen, NL
Thursday 29 June
Parallel session on Machine learning for soil mapping (5H)
Chaired by Laura Poggio
* alessandrosamuelrosa@gmail.com
A. (Alessandro) Samuel-Rosa* & R. S. D. Dalmolin

2. 25 years ago
Machine learning for soil mapping (5H)
Stacked generalization of statistical learners – a case study with soil iron in Brazil

3. 25 years ago
Machine learning for soil mapping (5H)
Stacked generalization of statistical learners – a case study with soil iron in Brazil

4. Machine learning for soil mapping (5H)
Stacked generalization of statistical learners – a case study with soil iron in Brazil
Nowadays
Model-based Gaussian and robust geostatistics (georob)
Andreas Papritz (May 9, 2017)

5. Nowadays
Machine learning for soil mapping (5H)
Stacked generalization of statistical learners – a case study with soil iron in Brazil
Legacy soil data
Suboptimal
geographic/feature
coverage
Extrapolation/reference
area method

6. 25 years ago
Machine learning for soil mapping (5H)
Stacked generalization of statistical learners – a case study with soil iron in Brazil

7. Stacked generalization (regression)
Machine learning for soil mapping (5H)
Stacked generalization of statistical learners – a case study with soil iron in Brazil
Goals
1) Combine statistical learners, and
2) Improve generalization accuracy
Learning error Generalization error
?

8. Stacked generalization (regression)
Machine learning for soil mapping (5H)
Stacked generalization of statistical learners – a case study with soil iron in Brazil
Strategy
1) Metamodel with cross-validation predictions as
covariates
2) Constrained metamodel coefficients to drop
redundant covariates/models

9. Case study
Machine learning for soil mapping (5H)
Stacked generalization of statistical learners – a case study with soil iron in Brazil
● Brazilian soil
database
● 5770 soil profiles
(22,981 records)
● 70% learning / 30%
evaluation
● iron ~ depth + taxon +
colour + parent +
carbon + clay + ph
1. Linear regression with
stepwise selection
2. Multivariate adaptive
regression splines
3. Regression random forest
4. Single-hidden-layer neural
network
5. Weighted k-nearest
neighbor regression
6. Support vector machine
with polynomial kernel
10-fold cross-validation
Metamodel

10. (Meta)Model evaluation
Machine learning for soil mapping (5H)
Stacked generalization of statistical learners – a case study with soil iron in Brazil
Learner ME MSE MAE RMSE AVE
rf 0.56 2369.58 27.82 48.68 0.55 0.5378
svm -5.24 2600.78 28.69 51.00 0.51 0.2773
kknn -0.19 2427.48 28.75 49.27 0.54 0.0796
mars -0.19 2577.03 29.84 50.76 0.51 0.0752
nnet 0.17 2721.52 31.76 52.17 0.49 0.0403
lm -0.31 2875.50 32.70 53.62 0.46 0.0000
Metamodel -0.53 2349.93 27.63 48.48 0.56 1.0101

11. Evaluation
Machine learning for soil mapping (5H)
Stacked generalization of statistical learners – a case study with soil iron in Brazil

12. Evaluation
Machine learning for soil mapping (5H)
Stacked generalization of statistical learners – a case study with soil iron in Brazil

13. Conclusions
Machine learning for soil mapping (5H)
Stacked generalization of statistical learners – a case study with soil iron in Brazil
● Stack of learners is superior
– Always?
● Easy to compute prediction error variance
– Standard regression/classification formulas
● Environmental interpretation
– Danger zone!?
● Cannot make miracles
– Data quality/quantity, diversity of learners

14. Stacked generalization of statistical
learners – a case study with soil iron
content in Brazil
A. (Alessandro) Samuel-Rosa* & R. S. D. Dalmolin
* alessandrosamuelrosa@gmail.com
This project was developed under the auspices of the Postgraduate Program in Soil
Science of the Federal University of Santa Maria as part of the National Postdoctoral
Program (PNPD) of the Coordination for Advancement for High Level Personnel
(CAPES) of the Ministry of Education of Brazil.