http://www.fao.org/globalsoilpartnership This presentation was made during the International Workshop "Soil Spectroscopy: the present and future of Soil Monitoring" that took place at FAO HQ Rome, Italy from 4-6 December 2013. This presentation was made by Bo Stenberg & Johanna Wetterlind, and it has the objective to present Can small NIR calibrations at the farm scale be an alternative to large calibrations for farm soil mapping of soil type? ©FAO: http://www.fao.org

1. Bo Stenberg and Johanna Wetterlind
Swedish University of Agricultural Sciences
Rikard Westbom and Daniel Olsson
Eurofins Food & Agro Testing Sweden AB
Can small NIR calibrations at the
farm scale be an alternative to
large calibrations for farm soil
mapping of soil type?

2. NIR for farm soil mapping of soil
organic matter and clay content
Standard procedure today:
 Top soil sampling – one sample per ha
 Plant available nutrients and pH analyzed on
every sample
 Texture and SOM on every third sample – if
at all

3. Our mission:
 SOM on every sample with NIR
 Texture on every sample with NIR
 Plant available nutrients and pH also with NIR
 Evaluate small farm scale calibrations for
implementation in a commercial lab
 How small can calibrations be?
 Keep it simple!
 Clay and SOM presented here

4. Wetterlind
& Stenberg
2010, EJSS
61
Minasney
2013,
Pedometron
Bränneberg
RMSEP=0.72
SEP=0.12
3.0
2.5
2.0
1.5
1.0
Hacksta
RMSEP=0.49
SEP=0.40
4
3
2
1
0
Large scale calibration for predictions at the farm scale

5. Seven farm sites in Sweden – 41-206 ha
picked from the production line at Eurofins
1 sample per ha

6. Methods:
 Samples – air dried and sieved to 2 mm
 SOM – loss of ignition corrected for structural
water
 Clay – sedimentation (<2µ)
 NIR – 1300-2500nm, NIRSystem 5000
(FOSS) – the system in use at the Eurofins
lab
 Everything analyzed on all samples

7. Calibratons:
 First derivative spectral pre-treatment
 Farm-wise PLS
1. Every third for calibration to predict the
remaining
2. Kennard-Stone selected calibration samples
in steps of 10 up to N/2 to predict the
remaining

8. Linköping SOM
Calibration samples
10 20 30 40 50 60 70 80
RMSE
0.0
0.2
0.4
0.6
0.8
1.0
Eskilstuna SOM
Calibration samples
10 20 30 40 50 60 70 80
RMSE
0.0
0.2
0.4
0.6
0.8
1.0
Askersund SOM
Calibration samples
10 20 30 40 50 60 70 80
RMSE
0.0
0.2
0.4
0.6
0.8
2.0
Mariestad SOM
Calibration samples
10 20 30 40 50 60 70 80
RMSE
0.0
0.2
0.4
0.6
1.0
1.2
Sjöbo SOM
Calibration samples
10 20 30 40 50 60 70 80
RMSE
0.0
0.2
0.4
0.6
2.5 Storfors SOM
Calibration samples
10 20 30 40 50 60 70 80 90
RMSE
0.0
0.2
0.4
0.6
0.8
1.0
Västerås SOM
Calibration samples
10 20 30 40 50 60 70 80
RMSE
0.0
0.2
0.4
0.6
0.8
1.0
0.59
1.6
0.58
1.5
0.83
2.5
0.82
2.4
0.78
2.2
0.83
2.4
0.95
4.6
• Every third
• Kennard-
Stone
SOM results
RMSE of predictions at the seven sites

9. Eskilstuna Clay
Calibration samples
10 20 30 40 50 60 70 80
RMSE
0
1
2
3
4
5
Linköping Clay
Calibration samples
10 20 30 40 50 60 70 80
RMSE
0
2
4
6
8
Mariestad Clay
Calibration samples
10 20 30 40 50 60 70 80
RMSE
0
1
2
3
4
5
Askersund Clay
Calibration samples
10 20 30 40 50 60 70 80
RMSE
0
1
2
3
4
5
Sjöbo Clay
Calibration samples
10 20 30 40 50 60 70 80
RMSE
0
1
2
3
4
5
Storfors Clay
Calibration samples
10 20 30 40 50 60 70 80
RMSE
0
1
2
3
4
5
Västerås Clay
Calibration samples
10 20 30 40 50 60 70 80
RMSE
0
1
2
3
4
5
0.86
2.7
0.68
1.8
0.92
3.5
0.95
4.3
0.91
3.4
0.88
2.9
0.81
2.3
Clay results
RMSE of predictions at the seven sites
• Every third
• Kennard-
Stone

10. Outlier detection
Important – especially in small calibrations
• Outliers in the calibration step – Manageble
• Outliers in the prediciton step – Not preferable
• Most methods removed too many samples
• Calibration step – X-Y relationship if CV bad (u-t
scores)
• Prediction step leverage >1 for prediction samples

11. Example 1: Clay at Linköping
Outlier detection

12. Example 2: SOM at Askersund
Outlier detection

13. KS40 KS40-6 outliers
Effect of removing calibration outliers
for SOM predictions at Askersund

14. Askersund SOM
SOM (%)
0 2 4 6 8
PredictedSOM(%)
0
2
4
6
8
Askersund Clay
Clay (%)
0 5 10 15 20 25 30
PredictedClay(%)
0
5
10
15
20
25
30
Eskilstuna SOM
SOM (%)
0 2 4 6
PredictedSOM(%)
0
2
4
6
Eskilstuna Clay
Clay (%)
40 50 60 70
PredictedClay(%)
40
50
60
70
Mariestad SOM
SOM (%)
0 2 4 6 8 10
PredictedSOM(%)
0
2
4
6
8
10
Mariestad Clay
Clay (%)
0 20 40 60 80
PredictedClay(%)
0
20
40
60
80
Linköping SOM
SOM (%)
0 2 4 6 8 10
PredictedSOM(%)
0
2
4
6
8
10
Linköping Clay
Clay (%)
0 20 40 60 80
PredictedClay(%)
0
20
40
60
80
R2 = 0.59
RMSEP = 0.66
R2 = 0.68
RMSEP = 2.9
R2 = 0.86
RMSEP = 2.0
R2 = 0.58
RMSEP = 0.53
R2 = 0.82
RMSEP = 0.67
R2 = 0.95
RMSEP = 3.3
R2 = 0.83
RMSEP = 0.52
R2 = 0.92
RMSEP = 4.1
Predictions with KS40 of SOM
and Clay at 4 sites

15. Sjöbo SOM
SOM (%)
0 2 4 6 8 10
PredictedSOM(%)
0
2
4
6
8
10
Sjöbo Clay
Clay (%)
0 10 20 30 40 50 60
PredictedClay(%)
0
10
20
30
40
50
60
Storfors SOM
SOM (%)
0 2 4 6 8 10
PredictedSOM(%)
0
2
4
6
8
10
Storfors Clay
Clay (%)
0 20 40 60
PredictedClay(%)
0
20
40
60
Västerås SOM
SOM (%)
0 2 4 6 8
PredictedSOM(%)
0
2
4
6
8
Västerås Clay
Clay (%)
20 30 40 50 60
PredictedClay(%)
20
30
40
50
60
R2 = 0.78
RMSEP = 0.44
R2 = 0.88
RMSEP = 4.0
R2 = 0.91
RMSEP = 2.2
R2 = 0.83
RMSEP = 0.58
R2 = 0.95
RMSEP = 0.32
R2 = 0.81
RMSEP = 2.4
Predictions with KS40 of SOM
and Clay at 3 sites

16. Relationship between SD and RMSE of 26
published large scale calibrations – SOM
Standard Deviation (% SOM)
0 1 2 3 4 5 6 7
RMSE(%SOM)
0.0
0.5
1.0
1.5
2.0
2.5
RMSE=0.20+0.31*SD
r2
=0.85

17. Standard Deviation (% SOM)
0 1 2 3 4 5 6 7
RMSE(%SOM)
0.0
0.5
1.0
1.5
2.0
2.5
RMSE=0.20+0.31*SD
r2
=0.85
Some published field and farm scale pre-
dictions in relation to the large scale
regressionline – SOM

18. Standard Deviation (% SOM)
0 1 2 3 4 5 6 7
RMSE(%SOM)
0.0
0.5
1.0
1.5
2.0
2.5
RMSE=0.20+0.31*SD
r2
=0.85
The present farm scale pre-dictions in relation
to the large scale regressionline – SOM

19. Standard deviation (%)
0 5 10 15 20 25 30
RMSE(%)
0
2
4
6
8
10
12
14
RMSE = -0.02 + 0.42 * SD
r ² = 0.77
The present farm scale predictions in relation
to the large scale regressionline and
calibrations – Clay
RPD (SD/RMSE) is
the same along the
line (~2.3)
Are the often used
RPD thresholds
relevant?

20. Lime requirement at Askersund (206 ha)
1/ha 1/3 ha
1/ha predicted
from KS40
About 9 tons ha-1
on average

21. Diff. all – 1/3 ha Diff. all – pred.
from KS40
Lime requirement at Askersund (206 ha)
7% 4%

22. Lime requirement at Västerås (175 ha)
1/ha 1/3 ha
1/ha predicted
from KS40
About 16 tons
ha-1 on average

23. Lime requirement at Västerås (175 ha)
Diff. all –
1/3 ha
Diff. all – pred.
from KS40
16% 2%

24. Conclusions
• It works – it seems
• Worthwhile for the labs to consider
• Some more thinking about outlier detection
• Calculations on the economy needs to be done