3rd Meeting of the Global Soil Laboratory Network (GLOSOLAN)
Ms. Fenny van Egmond
ISRIC
Harmonizing spectroscopy: progresses,
needs and way forward
3rd Meeting of the Global Soil Laboratory Network (GLOSOLAN)
Currently there are several limitations to global
assessment:
(i) some data simply do not exist for all places on Earth
(ii) while specific data may be widely available, it is
often collected and reported using different methods,
diverse standards, and/or using incompatible scales.
Recap: Why GLOSOLAN?
3rd Meeting of the Global Soil Laboratory Network (GLOSOLAN)
All scales
We need soil data
and information
Slide courtesy: Peter Wilson (CSIRO)
Global Soil Services
3rd Meeting of the Global Soil Laboratory Network (GLOSOLAN)
Purpose: why?
New data needed for:
• Monitoring
• Mapping
• National and international reporting
• Advice on soil fertility, sustainable land
management, land use planning, Land Degredation
Neutrality, Sustainable Development Goals
indicators, etc.
• Additional aid for validation reference measurements
3rd Meeting of the Global Soil Laboratory Network (GLOSOLAN)
Spectroscopy
• We focus now on the visible, near infrared and mid
infrared part of the electromagnetic spectrum
• 300-2500 nm and 4000-600 cm-1
3rd Meeting of the Global Soil Laboratory Network (GLOSOLAN)
Basic principle
• Interaction of light by soil is indication of some of
its properties
• Build a suitable calibration set: spectral library
(wet chemistry plus spectra of same samples)
• Model the relationship using
PLSR/ machine learning
• Apply this model to spectra
of a new sample to predict
its soil properties
3rd Meeting of the Global Soil Laboratory Network (GLOSOLAN)
Advantages
• Cost effective (SOC: ~ 1/3 of dry combustion price)
• (relatively) easy to implement
• Many properties in one go: Texture, SOC, TC, TN, pH,
CEC, extractable Ca, P sorption, some nutrients/others
• Fast
• Non-destructive
• Reproducible
• Different platforms possible: lab, field
• Input for digital soil mapping using airborne, satellite
data/sensors, possibly in future more options
3rd Meeting of the Global Soil Laboratory Network (GLOSOLAN)
Drawbacks
• Indirect: prediction models needed
• Bias and quality differences between equipment of
different manufacturers
• Model performance varies per property and
depends on the available calibration models inputs
(spectral library)
• Sensitive to heterogeneity and sample preparation
• Noise due to moisture and roughness (Zalidis)
3rd Meeting of the Global Soil Laboratory Network (GLOSOLAN)
Some history of spectroscopy
• Laboratory
• Field: proximal
• Earth Observation:
satellites, airplane (EO)
• Spectral libraries:
wet chemistry
(reference) + spectra
3rd Meeting of the Global Soil Laboratory Network (GLOSOLAN)
Problem space
Standardisation and harmonisation in collection, storage and
exchange is needed
Slide courtesy: Peter Wilson (CSIRO)
3rd Meeting of the Global Soil Laboratory Network (GLOSOLAN)
Spectroscopy history
• Over 20 years of attempts, discussions, initiatives for
building (protocols for) a Global Soil Spectral Library
• Many different protocols: need for standards, data sharing,
calibration transfer
• Coming from research, now operational in big labs
• Active global research community
• Computation power, optical benches and analysis
algorithms increasingly powerful
• Field and lab instruments getting better and more available
• Earth Observation more and more powerful and available
3rd Meeting of the Global Soil Laboratory Network (GLOSOLAN)
Existing spectral libraries
• ICRAF-ISRIC: VNIR, MIR 4,438 samples, 785 soil profiles, 58
countries
• African Soil Spectral Library (AfSIS, ICRAF): MIR, VNIR Africa
> 100.000 spectra
• USDA: MIR USA plus: > 80.000 samples, NIR 20.000 samples
• Australia: 10-15 labs using MIR with different protocols
• LUCAS: Europe 2013: VNIR 20,000 topsoil (0 - 20 cm)
samples
• France, Czech Republic, Denmark, Brazil, Sweden, likely
many more
• Viscarra Rossel (2016): VNIR global soil spectral library paper
3rd Meeting of the Global Soil Laboratory Network (GLOSOLAN)
Field instruments
• On the go systems: eg. Veris
• Handheld: eg. Ocean Optics, Scio
• NIR – MIR
• Various quality
• Effort for NIR field measurement
standardisation in ISO led by
Eyal Ben Dor
3rd Meeting of the Global Soil Laboratory Network (GLOSOLAN)
Earth Observation
• ESA World Soil Consultation Meeting
• Upcoming missions: including hyperspectral
• Aim for soil products in Copernicus Land Service
• Active community
3rd Meeting of the Global Soil Laboratory Network (GLOSOLAN)
Challenges
• How do we compare spectral databases from different
laboratories and spanning a wide range of soil types?
This requires harmonization of wet chemistry legacy
data as well.
• Is it feasible and economically viable to implement soil
spectroscopy for assessing the status of soil at country
and continental scales?
• Can we improve the prediction accuracy of soil
spectroscopy by joining spectral databases in order to
increase the density of samples and their spatial
coverage?
• What are the missing conditions to turn soil
spectroscopy into an operational technique?
Source: white paper resulting from 2013 workshop on soil spectroscopy
3rd Meeting of the Global Soil Laboratory Network (GLOSOLAN)
Why do this in GLOSOLAN?
• Handling standardisation (SOP’s and legacy data)
• Extra lab method that is applied more and more
• Allows more new data:
• Supports monitoring efforts (eg SOC, others)
• Rapidly increase point density -> better maps
• New applications possible (eg. more detailed advise)
• Regional capacity building and knowledge exchange to
predict soil properties for classification, interpretation,
soil assessment
• (meta) data storage and harmonisation of methods and
data is crucial for Pillar 4/GLOSIS
3rd Meeting of the Global Soil Laboratory Network (GLOSOLAN)
Link to GLOSOLAN objectives
• Standardisation of methods needed
• Wet chemistry + spectra + infrastructure +
organisational structure =
GLOSOLAN + Spectra WG/community + GLOSIS
+ GSP =
synergy and unique opportunity =
1 + 1 + 1 + 1 = 5
3rd Meeting of the Global Soil Laboratory Network (GLOSOLAN)
2019: Establishment of a working group on lab soil
spectroscopy and set up of a work plan including:
- Inventory on the information and capacity GLOSOLAN
members need/ have on spectroscopy
- Development of guidelines on lab spectroscopy
measurements
- Identification of metadata shortlist
Future aim: include lab spectral data in GLOSIS as Global Soil
Spectral Library using agreed upon guidelines and calibration
transfer algorithms for spectra.
Leaders: ICRAF, ISRIC, USDA
GLOSOLAN WG
3rd Meeting of the Global Soil Laboratory Network (GLOSOLAN)
Possible future work:
• Integrate spectral data into GLOSIS
• By doing so: Provide a data and exchange platform for the
broader soil (spectral) community
• Provide simple tools to work with spectral data to make it
more accessible to multiple users and labs
• Follow up on the needs that are mapped before: e.g. include
capacity building, indicator development, applications, use
case development, integration of these lab data and earth
observation data to assist thematic products, etc.
• Develop and disseminate use cases to show the use and
cost/benefit of this type of data to encourage countries to
extend their data holdings
GLOSOLAN Soil Spectroscopy WG
3rd Meeting of the Global Soil Laboratory Network (GLOSOLAN)
• Inventory of current capacities, needs and future
aims of labs and data providers/users on soil
spectroscopy
• Part 1. Laboratories and procedures
Part 2. Spectral data provisioning
Part 3. Spectral data analysis
Part 4. The Global Soil Information System (GLOSIS)
and soil spectral data
• https://forms.gle/sLsckokCf8V1CtU66
Questionnaire
3rd Meeting of the Global Soil Laboratory Network (GLOSOLAN)
Guidelines document
• Introduction to soil spectroscopy
• Spectral instruments
• Soil sample preparation and reading
• Spectra quality assurance and control
• Calibration of soil properties
• Laboratory information systems
• Spectral software
• Spectral prediction performance
• Digital soil mapping application
• Soil testing and advisory services application
• Soil carbon accounting example application
• Reporting for global spectral library development
• Future developments
3rd Meeting of the Global Soil Laboratory Network (GLOSOLAN)
Metadata
• ISO 19115 (ISO, 2003)
• INSPIRE compliant
• Should at least address:
• What
• Who
• Where
• When
• How
• Why
• Whose
For example:
• Bandwidth
• Nr of scans
• Brand and model
• Date
3rd Meeting of the Global Soil Laboratory Network (GLOSOLAN)
• Joint effort
• Seeing is believing
• Attractive alternative to conventional analysis
• Soil sample archives, computational and research
developments provide a unique opportunity
• Unique opportunity:
• For underrepresented areas on the map,
• for developing countries to get to cutting edge methods
overnight,
• to overcome differences between groups, to enable new
applications,
• to increase cooperation between pillars
Invitation to contribute/support
3rd Meeting of the Global Soil Laboratory Network (GLOSOLAN)
Harmonise and collect
Source: ISRIC WOSIS snapshot 2019
3rd Meeting of the Global Soil Laboratory Network (GLOSOLAN)
Thanks for your attention!
Also on behalf of Rich Ferguson,
Keith Shepherd
Let’s discuss!

Item 21 spectroscopy

  • 1.
    3rd Meeting ofthe Global Soil Laboratory Network (GLOSOLAN) Ms. Fenny van Egmond ISRIC Harmonizing spectroscopy: progresses, needs and way forward
  • 2.
    3rd Meeting ofthe Global Soil Laboratory Network (GLOSOLAN) Currently there are several limitations to global assessment: (i) some data simply do not exist for all places on Earth (ii) while specific data may be widely available, it is often collected and reported using different methods, diverse standards, and/or using incompatible scales. Recap: Why GLOSOLAN?
  • 3.
    3rd Meeting ofthe Global Soil Laboratory Network (GLOSOLAN) All scales We need soil data and information Slide courtesy: Peter Wilson (CSIRO) Global Soil Services
  • 4.
    3rd Meeting ofthe Global Soil Laboratory Network (GLOSOLAN) Purpose: why? New data needed for: • Monitoring • Mapping • National and international reporting • Advice on soil fertility, sustainable land management, land use planning, Land Degredation Neutrality, Sustainable Development Goals indicators, etc. • Additional aid for validation reference measurements
  • 5.
    3rd Meeting ofthe Global Soil Laboratory Network (GLOSOLAN) Spectroscopy • We focus now on the visible, near infrared and mid infrared part of the electromagnetic spectrum • 300-2500 nm and 4000-600 cm-1
  • 6.
    3rd Meeting ofthe Global Soil Laboratory Network (GLOSOLAN) Basic principle • Interaction of light by soil is indication of some of its properties • Build a suitable calibration set: spectral library (wet chemistry plus spectra of same samples) • Model the relationship using PLSR/ machine learning • Apply this model to spectra of a new sample to predict its soil properties
  • 7.
    3rd Meeting ofthe Global Soil Laboratory Network (GLOSOLAN) Advantages • Cost effective (SOC: ~ 1/3 of dry combustion price) • (relatively) easy to implement • Many properties in one go: Texture, SOC, TC, TN, pH, CEC, extractable Ca, P sorption, some nutrients/others • Fast • Non-destructive • Reproducible • Different platforms possible: lab, field • Input for digital soil mapping using airborne, satellite data/sensors, possibly in future more options
  • 8.
    3rd Meeting ofthe Global Soil Laboratory Network (GLOSOLAN) Drawbacks • Indirect: prediction models needed • Bias and quality differences between equipment of different manufacturers • Model performance varies per property and depends on the available calibration models inputs (spectral library) • Sensitive to heterogeneity and sample preparation • Noise due to moisture and roughness (Zalidis)
  • 9.
    3rd Meeting ofthe Global Soil Laboratory Network (GLOSOLAN) Some history of spectroscopy • Laboratory • Field: proximal • Earth Observation: satellites, airplane (EO) • Spectral libraries: wet chemistry (reference) + spectra
  • 10.
    3rd Meeting ofthe Global Soil Laboratory Network (GLOSOLAN) Problem space Standardisation and harmonisation in collection, storage and exchange is needed Slide courtesy: Peter Wilson (CSIRO)
  • 11.
    3rd Meeting ofthe Global Soil Laboratory Network (GLOSOLAN) Spectroscopy history • Over 20 years of attempts, discussions, initiatives for building (protocols for) a Global Soil Spectral Library • Many different protocols: need for standards, data sharing, calibration transfer • Coming from research, now operational in big labs • Active global research community • Computation power, optical benches and analysis algorithms increasingly powerful • Field and lab instruments getting better and more available • Earth Observation more and more powerful and available
  • 12.
    3rd Meeting ofthe Global Soil Laboratory Network (GLOSOLAN) Existing spectral libraries • ICRAF-ISRIC: VNIR, MIR 4,438 samples, 785 soil profiles, 58 countries • African Soil Spectral Library (AfSIS, ICRAF): MIR, VNIR Africa > 100.000 spectra • USDA: MIR USA plus: > 80.000 samples, NIR 20.000 samples • Australia: 10-15 labs using MIR with different protocols • LUCAS: Europe 2013: VNIR 20,000 topsoil (0 - 20 cm) samples • France, Czech Republic, Denmark, Brazil, Sweden, likely many more • Viscarra Rossel (2016): VNIR global soil spectral library paper
  • 13.
    3rd Meeting ofthe Global Soil Laboratory Network (GLOSOLAN) Field instruments • On the go systems: eg. Veris • Handheld: eg. Ocean Optics, Scio • NIR – MIR • Various quality • Effort for NIR field measurement standardisation in ISO led by Eyal Ben Dor
  • 14.
    3rd Meeting ofthe Global Soil Laboratory Network (GLOSOLAN) Earth Observation • ESA World Soil Consultation Meeting • Upcoming missions: including hyperspectral • Aim for soil products in Copernicus Land Service • Active community
  • 15.
    3rd Meeting ofthe Global Soil Laboratory Network (GLOSOLAN) Challenges • How do we compare spectral databases from different laboratories and spanning a wide range of soil types? This requires harmonization of wet chemistry legacy data as well. • Is it feasible and economically viable to implement soil spectroscopy for assessing the status of soil at country and continental scales? • Can we improve the prediction accuracy of soil spectroscopy by joining spectral databases in order to increase the density of samples and their spatial coverage? • What are the missing conditions to turn soil spectroscopy into an operational technique? Source: white paper resulting from 2013 workshop on soil spectroscopy
  • 16.
    3rd Meeting ofthe Global Soil Laboratory Network (GLOSOLAN) Why do this in GLOSOLAN? • Handling standardisation (SOP’s and legacy data) • Extra lab method that is applied more and more • Allows more new data: • Supports monitoring efforts (eg SOC, others) • Rapidly increase point density -> better maps • New applications possible (eg. more detailed advise) • Regional capacity building and knowledge exchange to predict soil properties for classification, interpretation, soil assessment • (meta) data storage and harmonisation of methods and data is crucial for Pillar 4/GLOSIS
  • 17.
    3rd Meeting ofthe Global Soil Laboratory Network (GLOSOLAN) Link to GLOSOLAN objectives • Standardisation of methods needed • Wet chemistry + spectra + infrastructure + organisational structure = GLOSOLAN + Spectra WG/community + GLOSIS + GSP = synergy and unique opportunity = 1 + 1 + 1 + 1 = 5
  • 18.
    3rd Meeting ofthe Global Soil Laboratory Network (GLOSOLAN) 2019: Establishment of a working group on lab soil spectroscopy and set up of a work plan including: - Inventory on the information and capacity GLOSOLAN members need/ have on spectroscopy - Development of guidelines on lab spectroscopy measurements - Identification of metadata shortlist Future aim: include lab spectral data in GLOSIS as Global Soil Spectral Library using agreed upon guidelines and calibration transfer algorithms for spectra. Leaders: ICRAF, ISRIC, USDA GLOSOLAN WG
  • 19.
    3rd Meeting ofthe Global Soil Laboratory Network (GLOSOLAN) Possible future work: • Integrate spectral data into GLOSIS • By doing so: Provide a data and exchange platform for the broader soil (spectral) community • Provide simple tools to work with spectral data to make it more accessible to multiple users and labs • Follow up on the needs that are mapped before: e.g. include capacity building, indicator development, applications, use case development, integration of these lab data and earth observation data to assist thematic products, etc. • Develop and disseminate use cases to show the use and cost/benefit of this type of data to encourage countries to extend their data holdings GLOSOLAN Soil Spectroscopy WG
  • 20.
    3rd Meeting ofthe Global Soil Laboratory Network (GLOSOLAN) • Inventory of current capacities, needs and future aims of labs and data providers/users on soil spectroscopy • Part 1. Laboratories and procedures Part 2. Spectral data provisioning Part 3. Spectral data analysis Part 4. The Global Soil Information System (GLOSIS) and soil spectral data • https://forms.gle/sLsckokCf8V1CtU66 Questionnaire
  • 21.
    3rd Meeting ofthe Global Soil Laboratory Network (GLOSOLAN) Guidelines document • Introduction to soil spectroscopy • Spectral instruments • Soil sample preparation and reading • Spectra quality assurance and control • Calibration of soil properties • Laboratory information systems • Spectral software • Spectral prediction performance • Digital soil mapping application • Soil testing and advisory services application • Soil carbon accounting example application • Reporting for global spectral library development • Future developments
  • 22.
    3rd Meeting ofthe Global Soil Laboratory Network (GLOSOLAN) Metadata • ISO 19115 (ISO, 2003) • INSPIRE compliant • Should at least address: • What • Who • Where • When • How • Why • Whose For example: • Bandwidth • Nr of scans • Brand and model • Date
  • 23.
    3rd Meeting ofthe Global Soil Laboratory Network (GLOSOLAN) • Joint effort • Seeing is believing • Attractive alternative to conventional analysis • Soil sample archives, computational and research developments provide a unique opportunity • Unique opportunity: • For underrepresented areas on the map, • for developing countries to get to cutting edge methods overnight, • to overcome differences between groups, to enable new applications, • to increase cooperation between pillars Invitation to contribute/support
  • 24.
    3rd Meeting ofthe Global Soil Laboratory Network (GLOSOLAN) Harmonise and collect Source: ISRIC WOSIS snapshot 2019
  • 25.
    3rd Meeting ofthe Global Soil Laboratory Network (GLOSOLAN) Thanks for your attention! Also on behalf of Rich Ferguson, Keith Shepherd Let’s discuss!

Editor's Notes

  • #20 Could leave this one out if you prefer. We haven’t discussed this yet in detail although the majority is supported by Keith and Rich I think.