TataKelola dan KamSiber Kecerdasan Buatan v022.pdf
Data Management System of the FAO Statistics Divison
1. Data Management System
of the FAO Statistics Divison
Monday, February 20, 2017FAO Statistics Division
Overview and main concepts
2. Monday, February 20, 2017FAO Statistics Division
Main ESS Systems
FAOSTAT CountrySTAT
Statistical Working System (SWS)
National Statistical Offices (NSOs)
Ministries of Agriculture (MAs)
3. Monday, February 20, 2017FAO Statistics Division
FAOSTAT data collection overview
Information is collected through questionnaires regularly dispatched to the countries via
electronic or regular mail.
The questionnaires have a standard formats, although they are not always uniform
across the FAOSTAT domains
Questionnaires are checked and validated by statistical clerks using different procedures
(including Excel working sheets).
Cleaned and validated data is then converted to a format compatible with the FAOSTAT
Working System.
Once in the FAOSTAT Working System, data is checked and validated a second time
before it is transferred to the FAOSTAT production database for publication.
4. Monday, February 20, 2017FAO Statistics Division
Structure of the FAOSTAT questionnaire
The questionnaire consists of an Excel Workbook with various sheets. There are:
A Cover Page: containing information such as:
• Contact info: Reporter name, title, administration and office, address, tel., fax, e-mail
• Brief description of the questionnaire: content and key information for its compilation
• Contacts in FAO
Domain-specific worksheets inclusive of the following elements:
• Codes: HS, FAO (FCL), others (depending on the domains)
• Code descriptions: e.g. Commodities, Land Categories, others
• Yearly time series: these can consist of one or more values per year. Fertilizer Production
expressed in metric tons of nutrient content, for example, includes three values for different types of
fertilizer (N, P2O5, K2O). The number of years is constant in each worksheet but varies across the
questionnaires (e.g. Machinery: 7 years; Fertilizer: 4 years; Land use: 5 years)
• Formulas or other ad-hoc columns: whenever needed, other columns are included in the
questionnaire (e.g. Fertilized Production includes columns for Plant Nutrient Content in %; the Land
use and Irrigation includes a column to describe formulas)
• Notes and remarks
Metadata: a set of metadata elements standardized for all questionnaires. It includes:
• Focal point for contact,
• Availability of data
• Sources of basic data
• Frequency of data collection
• Dissemination method
• Timeliness of dissemination
• General observations on the Questionnaire
Additional note
Instructions on how to use the questionnaire
5. Monday, February 20, 2017FAO Statistics Division
FAOSTAT data validation
This operation requires thorough revision of the questionnaires and the verification of the consistency and completeness of the
data; the compliancy with the standards used by FAO is also verified.
The consistency and reliability of the data is thoroughly check by the ESS officers and statistical clerks. The main elements that
needs validation are listed below
Measurement units:
At times, conversion from local measurement units to the international systems is needed. The questionnaires may report values
expressed in sacks, bags, or other local measurement units. In these cases the statistical clerks use different procedures to check
and validate data
Standardization:
Local commodities or varieties might not be classified using the standard coding system. Those commodities are registered by the
statistical clerks as remarks, but are not included in the database (these are normally minor commodities)
In some cases, alternative classification systems are used to describe the commodities (e.g. Eurostat classification); conversion is
needed to include the data in the database
Metadata and country contacts:
Metadata information is also provided with the questionnaire and this data is converted to a proper metadata format.
Key tools:
• Cross-validation with external datasets
• Verification of incongruence of totals or calculated fields
• Interpolation of missing data
• Data checking and validation tools for generating the Food Balance Sheet
• Creation of working tables and charts
All methods applied by the statistical clerks to check and validate the data are quite solid and well tested and despite the large
number of questionnaires received every year they carry out the job in relatively short time.
6. Monday, February 20, 2017FAO Statistics Division
CountrySTAT data management
Aims at building country ownership of the system
NSOs are entirely responsible for the data updates
No request for data is made by FAO to NSO to feed CountrySTAT, as in
FAOSTAT
TWGs are the tools to build capacity on methods, standards and indicators that
are most relevant and convenient to monitor the status of agriculture and food
security
CountrySTAT data is accessed by FAOSTAT Statistical Working System via
web services or direct queries
FAOSTAT data and CountrySTAT data are harmonized through software
programs that convert national classifications to FAOSTAT international
classifications (HS, CPC, etc.) and vice versa