Open Soil Proles


Tomislav Hengl
ISRIC  World Soil Information, Wageningen University




                                                        GSM2011.org, June 2024th 2011
GSIF components


 1. Cyber infrastructure for input, analysis and visualization of
    data.
 2. Global databases (legacy data, gridded covariates) that
    are main inputs to global soil mapping.
 3. Software tools (modules and packages) and manuals for
    creation of geoinformation, for instance, according to the
    GlobalSoilMap.net specications.
 4. Standards and protocols for data entry, map generation and
    data sharing.




                                                 GSM2011.org, June 2024th 2011
Hypothesis




 Probably the most valuable legacy soil survey data

       are the eld records of soil properties
          i.e.   soil prole observations




                                       GSM2011.org, June 2024th 2011
Denition




Soil eld record1: any georeferenced observation
               or measurement of soil properties
              (anything measured in soil medium)




  1
      Compare with e.g.   http://www.geosamples.org
                                                      GSM2011.org, June 2024th 2011
Book chapter




               GSM2011.org, June 2024th 2011
Proposed implementation




 1. Build an open (real time Public access) data portal that
    can be used to collect and maintain world soil eld
    observations




                                            GSM2011.org, June 2024th 2011
Proposed implementation




 1. Build an open (real time Public access) data portal that
    can be used to collect and maintain world soil eld
    observations
 2. Use it to facilitate global soil mapping activities




                                            GSM2011.org, June 2024th 2011
Proposed implementation




 1. Build an open (real time Public access) data portal that
    can be used to collect and maintain world soil eld
    observations
 2. Use it to facilitate global soil mapping activities
 3. Collect millions of points. . .




                                            GSM2011.org, June 2024th 2011
Basic principles




 1. Soil surveyors / contributors remain owners of the data
    (read/write rights)




                                                GSM2011.org, June 2024th 2011
Basic principles




 1. Soil surveyors / contributors remain owners of the data
    (read/write rights)
 2. Anyone can contribute




                                                GSM2011.org, June 2024th 2011
Basic principles




 1. Soil surveyors / contributors remain owners of the data
    (read/write rights)
 2. Anyone can contribute
 3. Anyone can use




                                                GSM2011.org, June 2024th 2011
Basic principles




 1. Soil surveyors / contributors remain owners of the data
    (read/write rights)
 2. Anyone can contribute
 3. Anyone can use
 4. (ISRIC will maintain and improve it)




                                                GSM2011.org, June 2024th 2011
GBIF




       GSM2011.org, June 2024th 2011
Global Surface Summary of the Day (GSOD)




                                  GSM2011.org, June 2024th 2011
There are probably millions of soil records           out there




Figure: USDA NCSS Characterization Database, CSIRO National Soil
Archive, ISRIC WISE, SPADE, Iran National soil prole database,
Canadian Soil Information System, and African soil proles.

                                                GSM2011.org, June 2024th 2011
Lower Saxony



                       Only in Lower Saxony


              (federal state in northern Germany)


                    there are about 1.3 millions

                    of soil proles and cores.            2



 2
     Rainer et al. (2011): Soil carbon balance in sensitive ecosystems in Europe.

                                                          GSM2011.org, June 2024th 2011
Endangered species




Legacy soil eld records are being lost as we speak.


              There is a considerable

  danger of being lost to eective use forever (!)




                                        GSM2011.org, June 2024th 2011
Types of soil observations (records)




 1. Freeform Soil Records (FFSR)
 2. Soil Prole Records in Original Form (SPROF)
 3. Harmonized Soil Prole Records (HSPR)




                                          GSM2011.org, June 2024th 2011
FFSR


  Only a very limited number of basic site columns are
  absolutely required.
  Freeform soil records do not need to describe a whole soil
  prole (0200 cm) nor do they require any pedological or
  taxonomic data.
  Variable types (columns) are extendible.
  Entries are not restricted to standard measurement techniques.
  Metadata is attached to each new record and is a part of the
  record.




                                              GSM2011.org, June 2024th 2011
SPROF



  Soil Prole Records in Original Form (SPROF)

are soil prole records that describe complete prole

              but are   unharmonized
(do not comply with a single international standard)




                                      GSM2011.org, June 2024th 2011
HSPR




  Are complete (core columns).
  Are ready for use (mapping and analysis).
  Contain soil variables required to produce maps
  according to GlobalSoilMap.net specications.
  Do not carry any metadata (metadata is standard).




                                          GSM2011.org, June 2024th 2011
Data entry scheme
                                      New entry




                                      Complete
                        NO                                YES
                                       profile?




                             Soil Profile Record     NO
                              in Original Form              Harmonized      YES
                                  (SPROF)                    values?



      Freeform soil           Fill in the             Harmonize           Harmonized Soil
         record               missing                 to referent          Profile Record
         (FFSR)                values                   method                (HSPR)


                                                        SOIL
     Open soil record                               REFERENCE             Open soil profile
                                                      LIBRARY
        OSRL0                                                                 OSPL0
                                            Open Soil Profiles
                                            (ISRIC hosted enterprise DB)



                                                                         GSM2011.org, June 2024th 2011
Test-bed data portal




                       GSM2011.org, June 2024th 2011
Data display / sharing



For convenience of automated use and for consistency, we propose
the following attributes for HSPD:
       One row = one record3
       Column naming convention
       Uncertainty columns
       Access via URL or via ODBC (MySQL)




  3
      Although in the background we work with a relational DB.

                                                         GSM2011.org, June 2024th 2011
3D variables


A proposed standard naming convention. E.g. pH estimated in 1:5
water solution for the initial soil depth layer becomes:

 variable type           estimation model        depth sequence




    P       H        I         H     O       5          _    A




        variable type                 estimation model




        T        A       X   N           U       S    D     A     GSM2011.org, June 2024th 2011
variable type           estimation model        depth sequence
Site observations

    P       H        I         H     O       5          .    A
Site information, such as soil classication by the USDA system can
be coded as:

        variable type                 estimation model




        T        A       X   N           U       S    D     A




                                                                  GSM2011.org, June 2024th 2011
Summary points


   If meteorologists, ecologists have managed to build
   global data sets of eld records why can't we?




                                          GSM2011.org, June 2024th 2011
Summary points


   If meteorologists, ecologists have managed to build
   global data sets of eld records why can't we?
   If a government agency does not intend to sell their soil
   data, why not contribute to OSP (distributed hosting)?




                                           GSM2011.org, June 2024th 2011
Summary points


   If meteorologists, ecologists have managed to build
   global data sets of eld records why can't we?
   If a government agency does not intend to sell their soil
   data, why not contribute to OSP (distributed hosting)?
   Before we can use the OSP for soil mapping, we need to
   convert FFSR and SPROF to HSPR




                                           GSM2011.org, June 2024th 2011
Summary points


   If meteorologists, ecologists have managed to build
   global data sets of eld records why can't we?
   If a government agency does not intend to sell their soil
   data, why not contribute to OSP (distributed hosting)?
   Before we can use the OSP for soil mapping, we need to
   convert FFSR and SPROF to HSPR
   There is an enormous value in legacy soil eld data
   (forgotten goldmine)




                                           GSM2011.org, June 2024th 2011
Next steps




   Prepare entry templates for FFSR, SPROF and HSPR
   Focus on National soil prole datasets
   Invite people (IUSS, DSM, any soil surveyor) to contribute
   Popularize it




                                              GSM2011.org, June 2024th 2011

Open Soil Profiles - testbed data portal for storing soil profile data

  • 1.
    Open Soil Proles TomislavHengl ISRIC World Soil Information, Wageningen University GSM2011.org, June 2024th 2011
  • 2.
    GSIF components 1.Cyber infrastructure for input, analysis and visualization of data. 2. Global databases (legacy data, gridded covariates) that are main inputs to global soil mapping. 3. Software tools (modules and packages) and manuals for creation of geoinformation, for instance, according to the GlobalSoilMap.net specications. 4. Standards and protocols for data entry, map generation and data sharing. GSM2011.org, June 2024th 2011
  • 3.
    Hypothesis Probably themost valuable legacy soil survey data are the eld records of soil properties i.e. soil prole observations GSM2011.org, June 2024th 2011
  • 4.
    Denition Soil eld record1:any georeferenced observation or measurement of soil properties (anything measured in soil medium) 1 Compare with e.g. http://www.geosamples.org GSM2011.org, June 2024th 2011
  • 5.
    Book chapter GSM2011.org, June 2024th 2011
  • 6.
    Proposed implementation 1.Build an open (real time Public access) data portal that can be used to collect and maintain world soil eld observations GSM2011.org, June 2024th 2011
  • 7.
    Proposed implementation 1.Build an open (real time Public access) data portal that can be used to collect and maintain world soil eld observations 2. Use it to facilitate global soil mapping activities GSM2011.org, June 2024th 2011
  • 8.
    Proposed implementation 1.Build an open (real time Public access) data portal that can be used to collect and maintain world soil eld observations 2. Use it to facilitate global soil mapping activities 3. Collect millions of points. . . GSM2011.org, June 2024th 2011
  • 9.
    Basic principles 1.Soil surveyors / contributors remain owners of the data (read/write rights) GSM2011.org, June 2024th 2011
  • 10.
    Basic principles 1.Soil surveyors / contributors remain owners of the data (read/write rights) 2. Anyone can contribute GSM2011.org, June 2024th 2011
  • 11.
    Basic principles 1.Soil surveyors / contributors remain owners of the data (read/write rights) 2. Anyone can contribute 3. Anyone can use GSM2011.org, June 2024th 2011
  • 12.
    Basic principles 1.Soil surveyors / contributors remain owners of the data (read/write rights) 2. Anyone can contribute 3. Anyone can use 4. (ISRIC will maintain and improve it) GSM2011.org, June 2024th 2011
  • 13.
    GBIF GSM2011.org, June 2024th 2011
  • 14.
    Global Surface Summaryof the Day (GSOD) GSM2011.org, June 2024th 2011
  • 15.
    There are probablymillions of soil records out there Figure: USDA NCSS Characterization Database, CSIRO National Soil Archive, ISRIC WISE, SPADE, Iran National soil prole database, Canadian Soil Information System, and African soil proles. GSM2011.org, June 2024th 2011
  • 16.
    Lower Saxony Only in Lower Saxony (federal state in northern Germany) there are about 1.3 millions of soil proles and cores. 2 2 Rainer et al. (2011): Soil carbon balance in sensitive ecosystems in Europe. GSM2011.org, June 2024th 2011
  • 17.
    Endangered species Legacy soileld records are being lost as we speak. There is a considerable danger of being lost to eective use forever (!) GSM2011.org, June 2024th 2011
  • 18.
    Types of soilobservations (records) 1. Freeform Soil Records (FFSR) 2. Soil Prole Records in Original Form (SPROF) 3. Harmonized Soil Prole Records (HSPR) GSM2011.org, June 2024th 2011
  • 19.
    FFSR Onlya very limited number of basic site columns are absolutely required. Freeform soil records do not need to describe a whole soil prole (0200 cm) nor do they require any pedological or taxonomic data. Variable types (columns) are extendible. Entries are not restricted to standard measurement techniques. Metadata is attached to each new record and is a part of the record. GSM2011.org, June 2024th 2011
  • 20.
    SPROF SoilProle Records in Original Form (SPROF) are soil prole records that describe complete prole but are unharmonized (do not comply with a single international standard) GSM2011.org, June 2024th 2011
  • 21.
    HSPR Arecomplete (core columns). Are ready for use (mapping and analysis). Contain soil variables required to produce maps according to GlobalSoilMap.net specications. Do not carry any metadata (metadata is standard). GSM2011.org, June 2024th 2011
  • 22.
    Data entry scheme New entry Complete NO YES profile? Soil Profile Record NO in Original Form Harmonized YES (SPROF) values? Freeform soil Fill in the Harmonize Harmonized Soil record missing to referent Profile Record (FFSR) values method (HSPR) SOIL Open soil record REFERENCE Open soil profile LIBRARY OSRL0 OSPL0 Open Soil Profiles (ISRIC hosted enterprise DB) GSM2011.org, June 2024th 2011
  • 23.
    Test-bed data portal GSM2011.org, June 2024th 2011
  • 24.
    Data display /sharing For convenience of automated use and for consistency, we propose the following attributes for HSPD: One row = one record3 Column naming convention Uncertainty columns Access via URL or via ODBC (MySQL) 3 Although in the background we work with a relational DB. GSM2011.org, June 2024th 2011
  • 25.
    3D variables A proposedstandard naming convention. E.g. pH estimated in 1:5 water solution for the initial soil depth layer becomes: variable type estimation model depth sequence P H I H O 5 _ A variable type estimation model T A X N U S D A GSM2011.org, June 2024th 2011
  • 26.
    variable type estimation model depth sequence Site observations P H I H O 5 . A Site information, such as soil classication by the USDA system can be coded as: variable type estimation model T A X N U S D A GSM2011.org, June 2024th 2011
  • 27.
    Summary points If meteorologists, ecologists have managed to build global data sets of eld records why can't we? GSM2011.org, June 2024th 2011
  • 28.
    Summary points If meteorologists, ecologists have managed to build global data sets of eld records why can't we? If a government agency does not intend to sell their soil data, why not contribute to OSP (distributed hosting)? GSM2011.org, June 2024th 2011
  • 29.
    Summary points If meteorologists, ecologists have managed to build global data sets of eld records why can't we? If a government agency does not intend to sell their soil data, why not contribute to OSP (distributed hosting)? Before we can use the OSP for soil mapping, we need to convert FFSR and SPROF to HSPR GSM2011.org, June 2024th 2011
  • 30.
    Summary points If meteorologists, ecologists have managed to build global data sets of eld records why can't we? If a government agency does not intend to sell their soil data, why not contribute to OSP (distributed hosting)? Before we can use the OSP for soil mapping, we need to convert FFSR and SPROF to HSPR There is an enormous value in legacy soil eld data (forgotten goldmine) GSM2011.org, June 2024th 2011
  • 31.
    Next steps Prepare entry templates for FFSR, SPROF and HSPR Focus on National soil prole datasets Invite people (IUSS, DSM, any soil surveyor) to contribute Popularize it GSM2011.org, June 2024th 2011